-- Logs begin at Sat 2022-09-17 20:01:16 UTC, end at Sat 2022-09-17 22:18:56 UTC. -- Sep 17 20:32:40.322277 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Sep 17 20:32:40.332045 ncdv-master-1 devstack@keystone.service[99154]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 17 20:32:40 2022] *** Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: nodename: ncdv-master-1 Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: machine: x86_64 Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: clock source: unix Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: pcre jit disabled Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: detected number of CPU cores: 4 Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: current working directory: / Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: detected binary path: /usr/bin/uwsgi-core Sep 17 20:32:40.334686 ncdv-master-1 devstack@keystone.service[99154]: your processes number limit is 47680 Sep 17 20:32:40.335319 ncdv-master-1 devstack@keystone.service[99154]: your memory page size is 4096 bytes Sep 17 20:32:40.335319 ncdv-master-1 devstack@keystone.service[99154]: detected max file descriptor number: 2048 Sep 17 20:32:40.335319 ncdv-master-1 devstack@keystone.service[99154]: lock engine: pthread robust mutexes Sep 17 20:32:40.335319 ncdv-master-1 devstack@keystone.service[99154]: thunder lock: enabled Sep 17 20:32:40.335319 ncdv-master-1 devstack@keystone.service[99154]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 17 20:32:40.335319 ncdv-master-1 devstack@keystone.service[99154]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 17 20:32:40.378300 ncdv-master-1 devstack@keystone.service[99154]: Python main interpreter initialized at 0x5632267299c0 Sep 17 20:32:40.378300 ncdv-master-1 devstack@keystone.service[99154]: python threads support enabled Sep 17 20:32:40.378300 ncdv-master-1 devstack@keystone.service[99154]: your server socket listen backlog is limited to 100 connections Sep 17 20:32:40.378300 ncdv-master-1 devstack@keystone.service[99154]: your mercy for graceful operations on workers is 90 seconds Sep 17 20:32:40.378639 ncdv-master-1 devstack@keystone.service[99154]: mapped 403077 bytes (393 KB) for 2 cores Sep 17 20:32:40.378713 ncdv-master-1 devstack@keystone.service[99154]: *** Operational MODE: preforking *** Sep 17 20:32:40.378949 ncdv-master-1 devstack@keystone.service[99154]: *** uWSGI is running in multiple interpreter mode *** Sep 17 20:32:40.378949 ncdv-master-1 devstack@keystone.service[99154]: spawned uWSGI master process (pid: 99154) Sep 17 20:32:40.379144 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Sep 17 20:32:40.381070 ncdv-master-1 devstack@keystone.service[99154]: spawned uWSGI worker 1 (pid: 99166, cores: 1) Sep 17 20:32:40.381070 ncdv-master-1 devstack@keystone.service[99154]: spawned uWSGI worker 2 (pid: 99167, cores: 1) Sep 17 20:32:40.381070 ncdv-master-1 devstack@keystone.service[99154]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 17 20:32:44.940393 ncdv-master-1 devstack@keystone.service[99166]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:32:44.940393 ncdv-master-1 devstack@keystone.service[99166]: from cryptography.utils import int_from_bytes Sep 17 20:32:44.941297 ncdv-master-1 devstack@keystone.service[99166]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:32:44.941297 ncdv-master-1 devstack@keystone.service[99166]: from cryptography.utils import int_from_bytes Sep 17 20:32:44.942083 ncdv-master-1 devstack@keystone.service[99167]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:32:44.942083 ncdv-master-1 devstack@keystone.service[99167]: from cryptography.utils import int_from_bytes Sep 17 20:32:44.942608 ncdv-master-1 devstack@keystone.service[99167]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:32:44.942608 ncdv-master-1 devstack@keystone.service[99167]: from cryptography.utils import int_from_bytes Sep 17 20:32:44.991163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 20:32:44.991360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 20:32:44.991443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 20:32:44.991534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 20:32:44.991613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] command line args: [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 20:32:44.991691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] command line args: [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 20:32:44.991768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 20:32:44.991845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 20:32:44.991923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ================================================================================ {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 20:32:44.992003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ================================================================================ {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 20:32:44.992176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] admin_token = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.992298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] admin_token = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.992376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] config_dir = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.992447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] config_dir = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.992595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.992732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.992732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] config_source = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.992946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] config_source = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] debug = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] debug = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.993901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] list_limit = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] list_limit = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] log_config_append = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] log_config_append = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.994930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] log_dir = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] log_dir = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] log_file = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] log_file = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.995944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.996961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.997986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.998109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.998203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.998407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.998589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.998968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] publish_errors = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] publish_errors = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:44.999990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] standard_threads = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] standard_threads = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.000870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] use_journal = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] use_journal = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] use_json = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] use_json = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] use_stderr = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] use_stderr = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] use_syslog = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] use_syslog = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.001949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.002090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:32:45.002171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.002250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.002426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.002550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.002636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.002721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.002903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.003962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.004919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.005951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.006998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.007984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.008911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.009940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.010957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.011891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.012967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.013922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.014954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.015875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.016838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.017955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.018942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.019960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.020903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.021870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.022984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.023923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.024973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.025980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.026959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] auth.external = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] auth.external = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.027929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] auth.password = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] auth.password = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] auth.token = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] auth.token = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.028896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.029847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] credential.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] credential.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.030955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.031922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.032001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.032154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.032241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.032401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.032827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.032982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.033063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.033228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.033323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.033497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.033580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.033763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.033842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.034969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.035917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.036941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.037913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.038853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.039949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.040959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.041970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.042903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.043894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.044844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.045978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.046993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.047939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.048878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.049853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.050972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.051983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.052896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.053901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.054966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.055999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.056921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.057893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.058976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.059054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.059238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.059370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.059496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.059630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.059758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.059881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.060981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.061106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.061239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] role.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.061368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] role.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.061571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] role.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.061571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.061811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] role.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.061811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.062960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.063842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.064972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.065122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.065245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.065370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.065511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.065636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.065782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.065908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.066943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.067980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.068998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.069122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.069257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.069382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.069535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.069661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.069786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.069911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.070931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.071949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.072971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.073095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.073219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.073352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.073520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.073646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.073771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.073895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.074021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.074156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.074289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.074431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.074577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.074703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.074893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.connection = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.connection = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.075905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.076935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.077964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 20:32:45.078905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:32:45.078905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 20:32:45.106719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.106947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.107217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.107367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.107588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.107777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.115683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.116295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.144521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.144737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.145310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.145569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.196662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.196957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.244448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.244691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.245292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.245520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.309902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.310103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.310332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.310428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.310584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.310719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.310871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.310966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.311102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.311182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.311245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.311450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.333618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.333846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.333970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.334975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.335056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.335206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.346051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.346178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.358670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.358885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.358983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.359984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.360166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:32:45.376656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.376853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:32:45.377210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.377331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.377411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:32:45.377514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:32:45.377668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.377753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.377956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.378082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:32:45.378265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.378455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.378553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.378693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.378693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.378947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.387924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.388206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.388473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.388669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.388764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.388893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.389021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.389275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.395667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.396261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.398544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.398828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.399111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.399306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.399401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.399531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.399656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.399779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.399903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.400944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.401073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.401201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.401346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.401491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.401785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.425957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.426468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.427406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.427673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.428051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.428338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.432344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.432625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.432906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.433072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.433376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.433681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.439613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.439899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.440164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.440453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.444014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.444310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.444594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.444736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.444863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.444948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.445172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.445293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.445434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.445586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.445737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.445858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.446047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.446192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.446278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.446574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.461633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.461833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.462236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.462516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.465753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.466450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.468994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.469290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.469712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.470008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.473516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.474228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.476661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.477425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.479875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.480189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.480475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.480599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.480829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.480925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.481087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.481214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.481406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.481578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.481768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.482084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.497324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.497574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.497872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.497999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.498078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.498291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.498571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.498832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.504963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.505692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.506533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.507221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.508070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.508343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.508775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.509054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.511004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.511283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.511594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.511735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.511922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.512051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.512290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.512430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.512648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.512980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.523426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.523706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.523990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.524114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.524264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.524392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.524640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.524833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.524928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.525072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.525198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.525360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.525614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.525907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.540984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.541845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.543915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.544808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.546849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.547726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.548271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.549170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.549767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.550049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.550456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.550675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.550957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.551351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.556777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.557725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.558297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.559281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.561229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.562168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.562474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.562763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.563046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.563417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.563707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.563995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.570868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.571158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.571442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.571849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.572141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.572421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.578370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.578654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.578937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.579210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.579405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:32:45.579500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.579721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.579849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.580047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.580172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.580325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.580458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.580631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.580765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.580944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.581073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.581201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.581344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.581500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.581640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.581768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.581896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.582024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.582150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.582286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.582518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.582783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.583051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:32:45.614889 ncdv-master-1 devstack@keystone.service[99166]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5632267299c0 pid: 99166 (default app) Sep 17 20:32:45.616208 ncdv-master-1 devstack@keystone.service[99167]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5632267299c0 pid: 99167 (default app) Sep 17 20:32:45.618195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3ee0-7532-4205-8cd7-93cc59ef8bee None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:45.618486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3ee0-7532-4205-8cd7-93cc59ef8bee None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:45.618754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3ee0-7532-4205-8cd7-93cc59ef8bee None None] PATH_INFO: `/v3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:45.622968 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Sat Sep 17 20:32:45 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:32:52.828258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3e8af-53e0-454d-8b7a-967800566aa5 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:52.828837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3e8af-53e0-454d-8b7a-967800566aa5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:52.828837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3e8af-53e0-454d-8b7a-967800566aa5 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:52.835739 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:32:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:32:52.842263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:52.842839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:52.842839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:52.930292 ncdv-master-1 devstack@keystone.service[99166]: INFO dbcounter [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] Registered counter for database keystone Sep 17 20:32:52.951470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG oslo_db.sqlalchemy.engines [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99166) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:32:52.961738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writer thread running {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 20:32:53.143132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG passlib.handlers.bcrypt [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99166) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 20:32:53.143529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG passlib.handlers.bcrypt [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99166) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 20:32:53.202986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:32:53.204509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:32:53.345125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:32:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 17 20:32:53.860758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3e8af-53e0-454d-8b7a-967800566aa5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:32:53.947001 ncdv-master-1 devstack@keystone.service[99167]: INFO dbcounter [None req-0dd3e8af-53e0-454d-8b7a-967800566aa5 None None] Registered counter for database keystone Sep 17 20:32:53.968953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG oslo_db.sqlalchemy.engines [None req-0dd3e8af-53e0-454d-8b7a-967800566aa5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99167) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:32:53.980419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writer thread running {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 20:32:54.098195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3e8af-53e0-454d-8b7a-967800566aa5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:32:54.103576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fff9ef-9186-45be-a466-8f74f093efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:32:54.104199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fff9ef-9186-45be-a466-8f74f093efd7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:54.104449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fff9ef-9186-45be-a466-8f74f093efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:54.104740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fff9ef-9186-45be-a466-8f74f093efd7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:54.113489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fff9ef-9186-45be-a466-8f74f093efd7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:32:54.165185 ncdv-master-1 devstack@keystone.service[99167]: WARNING keystone.common.rbac_enforcer.enforcer [None req-43fff9ef-9186-45be-a466-8f74f093efd7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 20:32:54.165532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fff9ef-9186-45be-a466-8f74f093efd7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:32:54.175730 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Sat Sep 17 20:32:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:32:54.185618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:32:54.207151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743cade-e696-4f73-a882-ed3b4db6866d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:32:54.215986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8f69e-99fd-4725-8fe5-786fbc6fd6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:32:54.217595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f69e-99fd-4725-8fe5-786fbc6fd6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:54.217875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f69e-99fd-4725-8fe5-786fbc6fd6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:54.218142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f69e-99fd-4725-8fe5-786fbc6fd6d2 admin admin] PATH_INFO: `/v3/projects/c8633d9055db49bfa42074c66a0581c2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:54.219821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8f69e-99fd-4725-8fe5-786fbc6fd6d2 admin admin] RBAC: Authorizing `identity:get_project(project_id=c8633d9055db49bfa42074c66a0581c2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:32:54.272195 ncdv-master-1 devstack@keystone.service[99166]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c4b8f69e-99fd-4725-8fe5-786fbc6fd6d2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 20:32:54.272696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8f69e-99fd-4725-8fe5-786fbc6fd6d2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:32:54.280433 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Sat Sep 17 20:32:54 2022] GET /identity/v3/projects/c8633d9055db49bfa42074c66a0581c2 => generated 344 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:32:56.708173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdb8b7-f115-4cb4-a3f4-4eb5a5f9bddf None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:56.708632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdb8b7-f115-4cb4-a3f4-4eb5a5f9bddf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:56.708732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdb8b7-f115-4cb4-a3f4-4eb5a5f9bddf None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:56.712832 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:32:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:32:56.719964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927feb6-75a7-4b5a-8d88-2093e3cd1748 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:56.720394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927feb6-75a7-4b5a-8d88-2093e3cd1748 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:56.720496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927feb6-75a7-4b5a-8d88-2093e3cd1748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:56.751720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2927feb6-75a7-4b5a-8d88-2093e3cd1748 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:32:56.753273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2927feb6-75a7-4b5a-8d88-2093e3cd1748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:32:56.763656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:32:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 17 20:32:57.083681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdb8b7-f115-4cb4-a3f4-4eb5a5f9bddf None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:32:57.105414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdb8b7-f115-4cb4-a3f4-4eb5a5f9bddf None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:32:57.110818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d88b7-0f0c-415a-8527-9d605a7aa14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:32:57.111428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d88b7-0f0c-415a-8527-9d605a7aa14b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:57.111708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d88b7-0f0c-415a-8527-9d605a7aa14b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:57.111979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d88b7-0f0c-415a-8527-9d605a7aa14b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:57.120552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d88b7-0f0c-415a-8527-9d605a7aa14b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:32:57.122429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d88b7-0f0c-415a-8527-9d605a7aa14b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:32:57.132230 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Sat Sep 17 20:32:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:32:57.141908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927feb6-75a7-4b5a-8d88-2093e3cd1748 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:32:57.154915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927feb6-75a7-4b5a-8d88-2093e3cd1748 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:32:57.161667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1bc30-1912-4969-ad8e-fdd934478a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:32:57.162313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1bc30-1912-4969-ad8e-fdd934478a09 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:57.162657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1bc30-1912-4969-ad8e-fdd934478a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:57.162928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1bc30-1912-4969-ad8e-fdd934478a09 admin admin] PATH_INFO: `/v3/users/fef6fab9131d4c1e89e0107005eb3caa` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:57.164488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1bc30-1912-4969-ad8e-fdd934478a09 admin admin] RBAC: Authorizing `identity:get_user(user_id=fef6fab9131d4c1e89e0107005eb3caa)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:32:57.166423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1bc30-1912-4969-ad8e-fdd934478a09 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:32:57.167980 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Sat Sep 17 20:32:57 2022] GET /identity/v3/users/fef6fab9131d4c1e89e0107005eb3caa => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:32:59.673860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d0f8-e654-4014-83bd-1aab0164dffb None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:59.674441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d0f8-e654-4014-83bd-1aab0164dffb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:59.674441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d0f8-e654-4014-83bd-1aab0164dffb None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:59.675688 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:32:59.676532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388622-e797-4164-9de6-c0ef837d253b None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:59.677006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388622-e797-4164-9de6-c0ef837d253b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:59.677081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388622-e797-4164-9de6-c0ef837d253b None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:59.678451 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:32:59.680664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:59.680909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:59.681176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:59.683339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ff7f-b563-406e-9c99-27d15260c3bc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:32:59.683624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ff7f-b563-406e-9c99-27d15260c3bc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:32:59.683899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ff7f-b563-406e-9c99-27d15260c3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:32:59.703740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5f60ff7f-b563-406e-9c99-27d15260c3bc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:32:59.704942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f60ff7f-b563-406e-9c99-27d15260c3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:32:59.724500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG passlib.handlers.bcrypt [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99167) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 20:32:59.724901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG passlib.handlers.bcrypt [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99167) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 20:32:59.783288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:32:59.785298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:32:59.900884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 7/12] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:32:59.903588 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 6/13] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:00.218073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ff7f-b563-406e-9c99-27d15260c3bc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:00.220794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:00.243377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:00.322942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:00.326319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200769d7-940e-4a6c-bb32-4569a6768dd9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:00.327065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ff7f-b563-406e-9c99-27d15260c3bc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:00.361664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:00.362281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:00.362558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:00.362829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:00.364472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:00.365620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:00.365893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:00.366160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:00.377603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:00.386004 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:00.387068 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:00.388019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:00.392143 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:00.392899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:00.429416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:00.430943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:00.453654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:00.489279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71352843-af69-420b-aad0-f9c53dc0f75c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:00.524892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:00.525506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:00.525785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:00.526055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:00.526709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:00.527008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:00.527323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 8/15] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:00.528452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:00.533219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:00.540196 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 8/16] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:00.545740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:00.640994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfeea07-c8e8-41fa-ae54-0cd03d9b2e89 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:00.666307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:00.679990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:00.680621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:00.680897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:00.681167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:00.684418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:00.693405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:00.694953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:33:00.718587 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:00.719710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:33:00.720724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:33:00.720724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:33:00.720724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:00.720724 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:00.725545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:00.766915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4da94-be2b-4b46-81e0-b3b21e8ab433 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:00.801342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:00.802028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:00.802301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:00.802577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] PATH_INFO: `/v3/users/fef6fab9131d4c1e89e0107005eb3caa` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:00.804006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] RBAC: Authorizing `identity:get_user(user_id=fef6fab9131d4c1e89e0107005eb3caa)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:00.805515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:00.807268 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/users/fef6fab9131d4c1e89e0107005eb3caa => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:00.812779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:00.827330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81993c-5c16-4a2d-a15c-53ea60c36e13 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:00.862432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:00.863055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:00.863331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:00.863629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:00.864280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:00.864582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:00.866029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:00.882834 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:00.907880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:00.919199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a09e2-3949-4a23-b74d-f3d8e92b036d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:00.958351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d639a35-cf47-45cc-a827-d7bc3b1ca6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:00.959012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639a35-cf47-45cc-a827-d7bc3b1ca6f1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:00.959287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639a35-cf47-45cc-a827-d7bc3b1ca6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:00.959556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639a35-cf47-45cc-a827-d7bc3b1ca6f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:00.960733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d639a35-cf47-45cc-a827-d7bc3b1ca6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:00.962276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d639a35-cf47-45cc-a827-d7bc3b1ca6f1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:00.963864 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:00.970188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:01.007398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:01.079780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3535fe4-959d-417d-ae6f-508ffb11a520 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:01.119340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62af9d-f548-4ba2-869c-6511999188c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:01.119962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62af9d-f548-4ba2-869c-6511999188c7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:01.120238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62af9d-f548-4ba2-869c-6511999188c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:01.120510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62af9d-f548-4ba2-869c-6511999188c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:01.121241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62af9d-f548-4ba2-869c-6511999188c7 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.domain.id=default, user.id=fef6fab9131d4c1e89e0107005eb3caa) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:01.121557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62af9d-f548-4ba2-869c-6511999188c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:01.124164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62af9d-f548-4ba2-869c-6511999188c7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:01.130367 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 11/21] 10.222.0.26 () {56 vars in 1322 bytes} [Sat Sep 17 20:33:00 2022] GET /identity/v3/role_assignments?user.id=fef6fab9131d4c1e89e0107005eb3caa&scope.domain.id=default&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:03.808004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe66fbb-8806-4e44-87e8-c7aa946ac847 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:03.808554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe66fbb-8806-4e44-87e8-c7aa946ac847 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:03.808554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe66fbb-8806-4e44-87e8-c7aa946ac847 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:03.809927 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:03.831526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0156-75c8-49a8-8aef-4e964c06ea50 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:03.832027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0156-75c8-49a8-8aef-4e964c06ea50 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:03.832120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0156-75c8-49a8-8aef-4e964c06ea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:03.858477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-703e0156-75c8-49a8-8aef-4e964c06ea50 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:03.859700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-703e0156-75c8-49a8-8aef-4e964c06ea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:03.942941 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 12/23] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:04.377376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9f699-1e58-4767-a545-ed9c544bbbb7 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:04.377791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9f699-1e58-4767-a545-ed9c544bbbb7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:04.377959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9f699-1e58-4767-a545-ed9c544bbbb7 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:04.379480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 12/24] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:04.384848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9903-deeb-403b-a251-bdf59ff892f2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:04.385129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9903-deeb-403b-a251-bdf59ff892f2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:04.385409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9903-deeb-403b-a251-bdf59ff892f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:04.410558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5eec9903-deeb-403b-a251-bdf59ff892f2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:04.411769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5eec9903-deeb-403b-a251-bdf59ff892f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:04.439366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:04.488987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852a272-70fd-4be5-a2e0-8881e3bd24f9 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:04.489343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852a272-70fd-4be5-a2e0-8881e3bd24f9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:04.489548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852a272-70fd-4be5-a2e0-8881e3bd24f9 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:04.491084 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 13/25] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:04.496893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308ec2-08d6-4861-af52-981e23064d01 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:04.497196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308ec2-08d6-4861-af52-981e23064d01 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:04.497439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308ec2-08d6-4861-af52-981e23064d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:04.514348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 13/26] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:04.525834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ec308ec2-08d6-4861-af52-981e23064d01 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:04.527048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ec308ec2-08d6-4861-af52-981e23064d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:04.601964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9903-deeb-403b-a251-bdf59ff892f2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:04.635812 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 14/27] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:04.698883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b2d2-a059-47b0-aa48-1426a8d5567a None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:04.699163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b2d2-a059-47b0-aa48-1426a8d5567a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:04.699436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b2d2-a059-47b0-aa48-1426a8d5567a None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:04.700791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 15/28] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:04.705972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788090d-cf48-4516-a2b5-bee7a0633f7f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:04.706254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788090d-cf48-4516-a2b5-bee7a0633f7f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:04.706528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788090d-cf48-4516-a2b5-bee7a0633f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:04.750941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4788090d-cf48-4516-a2b5-bee7a0633f7f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:04.752133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4788090d-cf48-4516-a2b5-bee7a0633f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:04.761528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9903-deeb-403b-a251-bdf59ff892f2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:04.824860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:04.825563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:04.825847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:04.826236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:04.848951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:04.850547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:33:04.876564 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:04.877544 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:33:04.879133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:33:04.879133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:33:04.879133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:04.879133 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 14/29] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:04.884526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:04.905878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:04.943936 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 16/30] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:05.019484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788090d-cf48-4516-a2b5-bee7a0633f7f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:05.041412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dc101-b9fb-4823-bb6c-07def1224502 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:05.076910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:05.094174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:05.094825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.095094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.095365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.096025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:05.096332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:05.097825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:05.114564 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 15/31] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:05.122661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:05.187709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788090d-cf48-4516-a2b5-bee7a0633f7f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:05.224496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:05.225111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.225387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.225685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.226354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:05.227750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:05.230379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:05.267228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f5da8-4e8f-4027-a9c3-e02c30210c11 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:05.307474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580d405-9efc-40f6-a087-ee220c7bb593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:05.308100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d405-9efc-40f6-a087-ee220c7bb593 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.308376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d405-9efc-40f6-a087-ee220c7bb593 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.308647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580d405-9efc-40f6-a087-ee220c7bb593 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.309314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580d405-9efc-40f6-a087-ee220c7bb593 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:05.310867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580d405-9efc-40f6-a087-ee220c7bb593 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:05.364539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 17/32] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 17 20:33:05 2022] POST /identity/v3/roles => generated 225 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:05.365311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:05.463042 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 16/33] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:33:05 2022] POST /identity/v3/projects => generated 301 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:33:05.465167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240584f0-461d-4d41-aa39-937e871ebd51 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.465776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240584f0-461d-4d41-aa39-937e871ebd51 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.466327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240584f0-461d-4d41-aa39-937e871ebd51 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.469862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:05.472811 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 17/34] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:05.473981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240584f0-461d-4d41-aa39-937e871ebd51 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:05.490947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3be3-99b3-4b5a-989b-a2142e0fde84 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:05.534115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c8e1d-0fc1-4dbf-b72e-bd4f80696f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:05.534882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8e1d-0fc1-4dbf-b72e-bd4f80696f96 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.535164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8e1d-0fc1-4dbf-b72e-bd4f80696f96 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.535437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8e1d-0fc1-4dbf-b72e-bd4f80696f96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.550476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c8e1d-0fc1-4dbf-b72e-bd4f80696f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-fa9c8e1d-0fc1-4dbf-b72e-bd4f80696f96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:05.555504 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:05.556443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:05.557256 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 18/35] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:05.562874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0af1-d4e9-4cb2-ad15-904a53f5265d None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.562874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0af1-d4e9-4cb2-ad15-904a53f5265d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.562874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0af1-d4e9-4cb2-ad15-904a53f5265d None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.562874 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 19/36] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:05.562874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ccf5-1844-4caa-ab2c-d84f0d6f2d47 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.562874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ccf5-1844-4caa-ab2c-d84f0d6f2d47 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.563363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ccf5-1844-4caa-ab2c-d84f0d6f2d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.584166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9be3ccf5-1844-4caa-ab2c-d84f0d6f2d47 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:05.585907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9be3ccf5-1844-4caa-ab2c-d84f0d6f2d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:05.654374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240584f0-461d-4d41-aa39-937e871ebd51 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:05.680175 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 20/37] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:05.681938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1ac51-efaa-41f1-8f31-b48244e5cdec None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.682441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1ac51-efaa-41f1-8f31-b48244e5cdec None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.682806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1ac51-efaa-41f1-8f31-b48244e5cdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.687628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:05.712630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e5c1ac51-efaa-41f1-8f31-b48244e5cdec None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:05.713817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:05.713938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e5c1ac51-efaa-41f1-8f31-b48244e5cdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:05.714748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.715250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.715686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.717082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:05.719412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:05.741622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 18/38] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 17 20:33:05 2022] POST /identity/v3/roles => generated 219 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:05.741622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:05.806990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:05.817672 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 21/39] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:05.895953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23fedb-3541-4087-8f22-eb45b741760a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:05.939290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:05.939941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:05.940245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:05.940524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:05.941225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:05.942222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:05.943914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:05.959780 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 19/40] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:05.965860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1ac51-efaa-41f1-8f31-b48244e5cdec None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:06.047678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:06.082293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1ac51-efaa-41f1-8f31-b48244e5cdec None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:06.109658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:06.128990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:06.129678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:06.129964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:06.130238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] PATH_INFO: `/v3/users/fef6fab9131d4c1e89e0107005eb3caa` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:06.131374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] RBAC: Authorizing `identity:get_user(user_id=fef6fab9131d4c1e89e0107005eb3caa)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:06.132947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:06.134743 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 22/41] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:05 2022] GET /identity/v3/users/fef6fab9131d4c1e89e0107005eb3caa => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:06.140499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:06.184696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b814e-0dce-485c-9f80-405ccbf22776 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:06.206254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:06.218464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:06.219015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:06.219296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:06.219570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:06.220326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:06.221762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:06.236192 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 20/42] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 17 20:33:06 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:06.236790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:06.247614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828828-25bc-4785-b5b3-2a38e004925d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:06.289509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:06.290165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:06.290561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:06.290836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:06.292031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:06.293541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:06.300126 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 23/43] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:06.301412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:06.351436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa92a18-b175-4450-bd75-83c0642a3654 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:06.354268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:06.403249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:06.404724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:06.405529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:06.406037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:06.408018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:06.409839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:06.411627 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 21/44] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:06.416133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f80cea-b42b-4c8c-91cf-db953834b44c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:06.417722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:06.460354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfe8bd-69da-4f79-8d08-801f1cd097db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:06.461023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfe8bd-69da-4f79-8d08-801f1cd097db None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:06.461329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfe8bd-69da-4f79-8d08-801f1cd097db None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:06.461647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfe8bd-69da-4f79-8d08-801f1cd097db None admin] PATH_INFO: `/v3/domains/default/users/fef6fab9131d4c1e89e0107005eb3caa/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:06.464725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfe8bd-69da-4f79-8d08-801f1cd097db None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fef6fab9131d4c1e89e0107005eb3caa, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:06.467145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfe8bd-69da-4f79-8d08-801f1cd097db None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:06.469586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:06.487366 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 24/45] 10.222.0.26 () {58 vars in 1361 bytes} [Sat Sep 17 20:33:06 2022] PUT /identity/v3/domains/default/users/fef6fab9131d4c1e89e0107005eb3caa/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:33:06.547632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe980b5-ff83-47c9-81e3-2f21c14e7b2d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:06.583819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a96e72-ca10-4cbf-b5bc-90311c4011fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:06.584443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a96e72-ca10-4cbf-b5bc-90311c4011fa None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:06.584725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a96e72-ca10-4cbf-b5bc-90311c4011fa None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:06.585005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a96e72-ca10-4cbf-b5bc-90311c4011fa None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:06.585703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a96e72-ca10-4cbf-b5bc-90311c4011fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:06.587236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a96e72-ca10-4cbf-b5bc-90311c4011fa None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:06.617258 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 22/46] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:33:06 2022] POST /identity/v3/projects => generated 312 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:08.871261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9675-c5c1-4c2c-8116-5079fb41ddef None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:08.871805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9675-c5c1-4c2c-8116-5079fb41ddef None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:08.871805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9675-c5c1-4c2c-8116-5079fb41ddef None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:08.873112 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 25/47] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:08.878733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c62c-6022-428e-9e73-5267d3fa5bf1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:08.879368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c62c-6022-428e-9e73-5267d3fa5bf1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:08.879368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0c62c-6022-428e-9e73-5267d3fa5bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:08.899296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-20a0c62c-6022-428e-9e73-5267d3fa5bf1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:08.900826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-20a0c62c-6022-428e-9e73-5267d3fa5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:08.920102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:08.969412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:08.976369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f88a5c-1e1f-4ca8-b1d7-f1f2a9b4395f None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:08.976655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f88a5c-1e1f-4ca8-b1d7-f1f2a9b4395f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:08.976924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f88a5c-1e1f-4ca8-b1d7-f1f2a9b4395f None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:08.978279 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 26/49] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:08.983508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d899-8e8b-48a8-af4f-a4100461d180 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:08.983788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d899-8e8b-48a8-af4f-a4100461d180 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:08.984058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d899-8e8b-48a8-af4f-a4100461d180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:09.002524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b637d899-8e8b-48a8-af4f-a4100461d180 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:09.003697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b637d899-8e8b-48a8-af4f-a4100461d180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:09.069432 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 24/50] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:09.288368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f88a5c-1e1f-4ca8-b1d7-f1f2a9b4395f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:09.386584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637d899-8e8b-48a8-af4f-a4100461d180 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:09.391647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f88a5c-1e1f-4ca8-b1d7-f1f2a9b4395f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:09.427786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:09.428418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:09.428671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:09.428939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:09.439533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:09.444300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:09.445428 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:09.446475 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 27/51] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:09.451566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:09.472659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:09.489470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637d899-8e8b-48a8-af4f-a4100461d180 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:09.505074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:09.523142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:09.523778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:09.524057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:09.524330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:09.525431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:09.526922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:09.528501 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 25/52] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:09.534397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:09.553309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df403b-7fd4-46cc-8cc0-28d19f672565 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:09.586624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:09.587233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:09.587517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:09.587818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:09.588466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:09.588768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:09.590197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:09.598092 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 28/53] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:09.603714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:09.630626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733593ab-1ba0-4c41-9405-34e5e6359d5b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:09.664006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:09.664613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:09.664888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:09.665156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:09.665833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:09.667333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:09.702404 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 26/54] 10.222.0.26 () {60 vars in 1140 bytes} [Sat Sep 17 20:33:09 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:33:09.712708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca53a1-45ce-4e32-8158-c80dbaff6aac None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:09.733619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:09.752782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:09.753440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:09.753752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:09.754025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] PATH_INFO: `/v3/users/fef6fab9131d4c1e89e0107005eb3caa` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:09.755246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] RBAC: Authorizing `identity:get_user(user_id=fef6fab9131d4c1e89e0107005eb3caa)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:09.756727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:09.758348 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 29/55] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:09 2022] GET /identity/v3/users/fef6fab9131d4c1e89e0107005eb3caa => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:09.764504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:09.770036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:09.876619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301a80-9453-4587-a3d3-654f918d5ced None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:09.912621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce799ab0-f910-4ab1-8ced-a7887cf2d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:09.913261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce799ab0-f910-4ab1-8ced-a7887cf2d63f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:09.913601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce799ab0-f910-4ab1-8ced-a7887cf2d63f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:09.913871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce799ab0-f910-4ab1-8ced-a7887cf2d63f None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:09.914972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce799ab0-f910-4ab1-8ced-a7887cf2d63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:09.916451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce799ab0-f910-4ab1-8ced-a7887cf2d63f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:09.918396 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 27/56] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:09.924663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:10.027557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5d4ee-414c-4683-ba28-4e2762fcae60 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:10.061194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfc223-5e4d-48c5-bc92-9f795453f7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:10.061835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfc223-5e4d-48c5-bc92-9f795453f7e5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:10.062111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfc223-5e4d-48c5-bc92-9f795453f7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:10.062382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfc223-5e4d-48c5-bc92-9f795453f7e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:10.063103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfc223-5e4d-48c5-bc92-9f795453f7e5 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.domain.id=default, user.id=fef6fab9131d4c1e89e0107005eb3caa) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:10.063389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfc223-5e4d-48c5-bc92-9f795453f7e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:10.064850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfc223-5e4d-48c5-bc92-9f795453f7e5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:10.072890 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 30/57] 10.222.0.26 () {56 vars in 1322 bytes} [Sat Sep 17 20:33:09 2022] GET /identity/v3/role_assignments?user.id=fef6fab9131d4c1e89e0107005eb3caa&scope.domain.id=default&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:12.078129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c80c20-ebbd-4a0e-bbd1-dabbe9016d30 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:12.078920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c80c20-ebbd-4a0e-bbd1-dabbe9016d30 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:12.078920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c80c20-ebbd-4a0e-bbd1-dabbe9016d30 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:12.081021 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 28/58] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:12.087301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a924769-15a3-469a-b608-a578e8598129 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:12.087975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a924769-15a3-469a-b608-a578e8598129 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:12.088721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a924769-15a3-469a-b608-a578e8598129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:12.115633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4a924769-15a3-469a-b608-a578e8598129 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:12.116849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a924769-15a3-469a-b608-a578e8598129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:12.163246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:12.188144 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 31/59] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:12.509692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c80c20-ebbd-4a0e-bbd1-dabbe9016d30 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:12.591066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:12.629787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c80c20-ebbd-4a0e-bbd1-dabbe9016d30 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:12.672898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fcf13-0782-43e3-a047-297ca2695aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:12.673976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fcf13-0782-43e3-a047-297ca2695aea None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:12.674453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fcf13-0782-43e3-a047-297ca2695aea None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:12.674906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fcf13-0782-43e3-a047-297ca2695aea None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:12.676987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fcf13-0782-43e3-a047-297ca2695aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:12.679372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fcf13-0782-43e3-a047-297ca2695aea None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:12.681984 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 29/60] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:12.690686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a924769-15a3-469a-b608-a578e8598129 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:12.799038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a924769-15a3-469a-b608-a578e8598129 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:12.834058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8b682-7ac5-450a-987f-420a63fcebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:12.834666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b682-7ac5-450a-987f-420a63fcebb5 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:12.834949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b682-7ac5-450a-987f-420a63fcebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:12.835219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8b682-7ac5-450a-987f-420a63fcebb5 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:12.835867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8b682-7ac5-450a-987f-420a63fcebb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:12.837325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8b682-7ac5-450a-987f-420a63fcebb5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:12.866420 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 32/61] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:33:12 2022] POST /identity/v3/users => generated 275 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:16.409512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e2ef-894f-4fe7-82ca-87cf3229322b None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:16.409512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e2ef-894f-4fe7-82ca-87cf3229322b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:16.410130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e2ef-894f-4fe7-82ca-87cf3229322b None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:16.410930 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 30/62] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:16.416499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b896150-faa7-40fe-8919-032b17c3b43a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:16.417303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b896150-faa7-40fe-8919-032b17c3b43a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:16.417792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b896150-faa7-40fe-8919-032b17c3b43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:16.438868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0b896150-faa7-40fe-8919-032b17c3b43a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:16.440407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0b896150-faa7-40fe-8919-032b17c3b43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:16.477170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:16.535193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 33/63] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:16.731632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ecc62-2e2a-47da-9564-ba42ce83a7ea None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:16.732313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ecc62-2e2a-47da-9564-ba42ce83a7ea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:16.732805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ecc62-2e2a-47da-9564-ba42ce83a7ea None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:16.734482 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 31/64] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:16.740619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af459a65-f701-4222-9de2-9f61dac0fbec None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:16.740898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af459a65-f701-4222-9de2-9f61dac0fbec None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:16.741330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af459a65-f701-4222-9de2-9f61dac0fbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:16.763099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-af459a65-f701-4222-9de2-9f61dac0fbec None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:16.764586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af459a65-f701-4222-9de2-9f61dac0fbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:16.851186 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 34/65] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:16.913090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ecc62-2e2a-47da-9564-ba42ce83a7ea None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.000848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473985-a4be-4217-b0b8-d57addf5a16f None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.001480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473985-a4be-4217-b0b8-d57addf5a16f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.001953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473985-a4be-4217-b0b8-d57addf5a16f None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.008153 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 35/66] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:17.009541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce118d-81b1-4efb-a1d7-6ef94a7bbcf8 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.010079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce118d-81b1-4efb-a1d7-6ef94a7bbcf8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.010564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce118d-81b1-4efb-a1d7-6ef94a7bbcf8 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.012128 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 36/67] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:17.013568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd59d-7448-4395-9bb8-e27c69e46111 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.014041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd59d-7448-4395-9bb8-e27c69e46111 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.014677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd59d-7448-4395-9bb8-e27c69e46111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.034588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4eccd59d-7448-4395-9bb8-e27c69e46111 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:17.036203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4eccd59d-7448-4395-9bb8-e27c69e46111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:17.053534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ecc62-2e2a-47da-9564-ba42ce83a7ea None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.088534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.089566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.089979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.090500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.099848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.104197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:17.111385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:17.112242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:17.113011 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:17.115584 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 32/68] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:17.118377 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 37/69] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:17.119921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187304f3-9131-45bb-a93b-c9f03ca11888 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.120669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187304f3-9131-45bb-a93b-c9f03ca11888 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.121107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187304f3-9131-45bb-a93b-c9f03ca11888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.121573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.140479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-187304f3-9131-45bb-a93b-c9f03ca11888 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:17.142006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-187304f3-9131-45bb-a93b-c9f03ca11888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:17.161451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.216142 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 38/70] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:17.227125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187304f3-9131-45bb-a93b-c9f03ca11888 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.298989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f208a8-847e-42a3-b64d-ddc9b88f9bcd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.341616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d258c0-6888-412d-854c-8a8c4e8c7f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.343022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d258c0-6888-412d-854c-8a8c4e8c7f24 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.343505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d258c0-6888-412d-854c-8a8c4e8c7f24 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.343951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d258c0-6888-412d-854c-8a8c4e8c7f24 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.345222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d258c0-6888-412d-854c-8a8c4e8c7f24 None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:17.345726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d258c0-6888-412d-854c-8a8c4e8c7f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:17.347769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d258c0-6888-412d-854c-8a8c4e8c7f24 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:17.350987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187304f3-9131-45bb-a93b-c9f03ca11888 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.358376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 33/71] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:17.359800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed88e88-5124-4efd-a0c4-ffccb8725487 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.360276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed88e88-5124-4efd-a0c4-ffccb8725487 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.360738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed88e88-5124-4efd-a0c4-ffccb8725487 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.362440 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 34/72] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:17.364908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed88e88-5124-4efd-a0c4-ffccb8725487 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.401088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a402d-6a71-4c43-91f7-8e5463ad71f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.401839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a402d-6a71-4c43-91f7-8e5463ad71f9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.402113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a402d-6a71-4c43-91f7-8e5463ad71f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.402380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a402d-6a71-4c43-91f7-8e5463ad71f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.415063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a402d-6a71-4c43-91f7-8e5463ad71f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-809a402d-6a71-4c43-91f7-8e5463ad71f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:17.421357 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:17.422706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:17.423494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 39/73] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:17.424147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52e60a-593c-46f2-b22b-3983581abcb6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.425100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52e60a-593c-46f2-b22b-3983581abcb6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.425100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52e60a-593c-46f2-b22b-3983581abcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.442138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.445536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4c52e60a-593c-46f2-b22b-3983581abcb6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:17.446774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4c52e60a-593c-46f2-b22b-3983581abcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:17.457955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.484673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed88e88-5124-4efd-a0c4-ffccb8725487 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.517170 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 40/74] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:17.521643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52e60a-593c-46f2-b22b-3983581abcb6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.533623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.534812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.535162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.535760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.552464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:17.554025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:17.555749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 35/75] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:17.556581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.622329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52e60a-593c-46f2-b22b-3983581abcb6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.655096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa890a2-559e-4e2d-9c74-dfaed0f12342 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.659725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.660342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.660616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.660888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.661581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:17.661884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:17.663675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:17.677955 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 41/76] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:17.677955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.679512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.687509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.701004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.701681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.701964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.702234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.712591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:17.717542 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:17.718370 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:17.719133 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 36/77] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:17.719761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.787657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cf2e9-c325-4cac-9b81-038b7840e84f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.822076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab122489-7d16-4797-8a27-e3cc983292fc None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.826310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.826966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.827229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.827502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.838065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:17.843307 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:17.844417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 42/78] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:17.845168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:17.857410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.858049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.858330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.858601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.905558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.912077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:17.926120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:17.927667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:17.949710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b50845-4ab3-4fdc-94cf-58ff44092135 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:17.983110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:17.983744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:17.984018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:17.984284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] PATH_INFO: `/v3/users/fef6fab9131d4c1e89e0107005eb3caa` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:17.985479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] RBAC: Authorizing `identity:get_user(user_id=fef6fab9131d4c1e89e0107005eb3caa)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:17.986973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:17.988756 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 43/79] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/users/fef6fab9131d4c1e89e0107005eb3caa => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:17.989352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.031297 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 37/80] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:18.032748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.088245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2199ac3-520d-4d6d-a548-7eacea757b17 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.110438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.120889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.121542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.121811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.122083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.122784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:18.123088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.124496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.125823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.130714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9b831-3e64-4c12-9a06-b1982f61227f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.133567 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 44/81] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:33:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.138866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.165371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.166033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.166303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.166577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.167681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.169157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.170864 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 38/82] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.171654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.240282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844c81d-5d97-4c68-a615-5e2086347141 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.273945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.274549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.274828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.275096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.275739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:18.276039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.277416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.279726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8da93-414b-418e-97e5-9982c3da713b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.286037 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 45/83] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.286844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.321513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.322137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.322417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.322687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.340012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.349207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.391745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.392382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34311e-c0ae-4ce7-8512-3b4062d28e47 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.393220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.427655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.428289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.428572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.428842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.430036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.431477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.433120 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 46/84] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.438162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.489327 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 39/85] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:18.494471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.535468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21825d-8599-4181-9409-5703192c0a36 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.541037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.562008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.570618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.571253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.571527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.571800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.572969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.574454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.576107 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 47/86] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.576870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.597135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6c8b-6a58-44bd-b40a-99ccec26f150 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.630971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.631617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.631893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.632163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.632820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.634356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.661148 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 40/87] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:33:18 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:18.662015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.691808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b953e3-aacc-441d-8350-1291026b0c4b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.749943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.750651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.750906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.751180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.752420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.754698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.761317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 48/88] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.761317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.797289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ed12d-6e03-445f-b2d4-e4dc70663542 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.802046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.853539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:18.862495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.863145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.863425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.863878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.865600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=a451c98d481d4228a63985e55b2d8871) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:18.865917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.867455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.876028 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 41/89] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/role_assignments?user.id=a451c98d481d4228a63985e55b2d8871&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.883001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:18.935384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673cfd1-d5a4-40c1-a7d6-c0fc1cf5f3cd None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:18.989000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:18.989854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:18.990140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:18.990416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:18.991604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:18.993291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:18.995055 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 49/90] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:18.995839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:19.060319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335613c-9389-4318-a846-98570f0e88c6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:19.118272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:19.118917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:19.119200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:19.119478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:19.148764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a26a0-2ab7-417b-a0b9-63b7ed7fa4bb None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:19.167001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:19.192181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:19.192992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:19.193388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:19.193940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:19.205830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:19.209754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:19.211820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:19.271331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:19.272996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:19.343770 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 42/91] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:19.344832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:19.416473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 50/92] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:19.417087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:19.469817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:19.473859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:19.519240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ae3d3-daed-4b2d-aa63-21eac2b22e31 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:19.584379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:19.585243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:19.585711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:19.586115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:19.586984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=fef6fab9131d4c1e89e0107005eb3caa) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:19.587427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:19.589044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:19.593681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734867b8-ab1f-4114-ac75-77bf86c14314 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:19.605622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 43/93] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:19 2022] GET /identity/v3/role_assignments?user.id=fef6fab9131d4c1e89e0107005eb3caa&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:19.606824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:19.652089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:19.653116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:19.653652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:19.654196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:19.656254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:19.658312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:19.660407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 51/94] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:19 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:19.666451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:19.795249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ac313-c4fc-49e6-8618-56e1bfb52101 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:19.851731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b9d36-f91b-4d5d-a174-4b443ed1f192 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:19.856369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:19.861860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:19.863447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:19.864362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:19.864844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:19.865302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] PATH_INFO: `/v3/projects/3c868b98d526469f89292a8c01d12dcc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:19.867194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] RBAC: Authorizing `identity:get_project(project_id=3c868b98d526469f89292a8c01d12dcc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:19.868998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:19.870941 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 44/95] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:19 2022] GET /identity/v3/projects/3c868b98d526469f89292a8c01d12dcc => generated 312 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:19.877953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:19.958389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60611603-4c23-4f9a-9a9a-1f39c3c22449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:19.958962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611603-4c23-4f9a-9a9a-1f39c3c22449 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:19.959794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611603-4c23-4f9a-9a9a-1f39c3c22449 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:19.960083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611603-4c23-4f9a-9a9a-1f39c3c22449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:19.961866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60611603-4c23-4f9a-9a9a-1f39c3c22449 None admin] RBAC: Adding query filter params (role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=a451c98d481d4228a63985e55b2d8871) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:19.962192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60611603-4c23-4f9a-9a9a-1f39c3c22449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:19.963713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60611603-4c23-4f9a-9a9a-1f39c3c22449 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:19.970941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 52/96] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:19 2022] GET /identity/v3/role_assignments?user.id=a451c98d481d4228a63985e55b2d8871&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=1b7585de1b6442509e3788682caf8774 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:20.061546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d08f-3023-4a4d-a106-7daa85388afb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:20.127119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070884f-4a34-42b0-9711-3d771efbe973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:20.128187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070884f-4a34-42b0-9711-3d771efbe973 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:20.128692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070884f-4a34-42b0-9711-3d771efbe973 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:20.129157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070884f-4a34-42b0-9711-3d771efbe973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:20.130166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070884f-4a34-42b0-9711-3d771efbe973 None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=3c868b98d526469f89292a8c01d12dcc, user.id=a451c98d481d4228a63985e55b2d8871) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:20.130688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070884f-4a34-42b0-9711-3d771efbe973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:20.132536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070884f-4a34-42b0-9711-3d771efbe973 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:20.142200 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 45/97] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:19 2022] GET /identity/v3/role_assignments?user.id=a451c98d481d4228a63985e55b2d8871&scope.project.id=3c868b98d526469f89292a8c01d12dcc&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:21.846707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1451-352b-4373-b30c-de5276acab24 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:21.847737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1451-352b-4373-b30c-de5276acab24 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:21.848245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1451-352b-4373-b30c-de5276acab24 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:21.849857 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 53/98] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:21.863190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2d89d-33ad-44dd-9b5c-43d9d508737b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:21.864152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2d89d-33ad-44dd-9b5c-43d9d508737b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:21.864615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2d89d-33ad-44dd-9b5c-43d9d508737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:21.890572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b0d2d89d-33ad-44dd-9b5c-43d9d508737b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:21.892125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b0d2d89d-33ad-44dd-9b5c-43d9d508737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:21.937269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663387b-e968-4826-95c2-7a6f2cc99723 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:21.937942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663387b-e968-4826-95c2-7a6f2cc99723 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:21.938409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663387b-e968-4826-95c2-7a6f2cc99723 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:21.940049 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 54/99] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:21.948514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada746f-a999-42f4-a1a5-523ea9403883 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:21.949110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada746f-a999-42f4-a1a5-523ea9403883 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:21.949604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada746f-a999-42f4-a1a5-523ea9403883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:21.976609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bada746f-a999-42f4-a1a5-523ea9403883 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:21.978234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bada746f-a999-42f4-a1a5-523ea9403883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:22.029841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:22.050151 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 46/100] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:22.095716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 55/101] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:22.678952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2d89d-33ad-44dd-9b5c-43d9d508737b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:22.770683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada746f-a999-42f4-a1a5-523ea9403883 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:22.789849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2d89d-33ad-44dd-9b5c-43d9d508737b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:22.834205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:22.834871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:22.835164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:22.835441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:22.858077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:22.862950 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:22.864446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:22.865240 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 47/102] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:22.871909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:22.933020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada746f-a999-42f4-a1a5-523ea9403883 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:22.949903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:22.976621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5eb21-086a-492e-b129-055c8f3eb5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:22.977264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5eb21-086a-492e-b129-055c8f3eb5f8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:22.977576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5eb21-086a-492e-b129-055c8f3eb5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:22.977848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5eb21-086a-492e-b129-055c8f3eb5f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:22.985878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:22.988829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5eb21-086a-492e-b129-055c8f3eb5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-06f5eb21-086a-492e-b129-055c8f3eb5f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:22.993875 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:22.994849 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:22.995618 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 56/103] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:22.997187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2325f-7e51-4814-b5b0-46cb7e367ada None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:22.997187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2325f-7e51-4814-b5b0-46cb7e367ada None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:22.997352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2325f-7e51-4814-b5b0-46cb7e367ada None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.003198 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 57/104] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:23.005006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2325f-7e51-4814-b5b0-46cb7e367ada None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:23.088385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6c275-3068-47e6-86a9-b2825e4c44fc None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:23.135916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2325f-7e51-4814-b5b0-46cb7e367ada None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:23.174922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536278b8-16a8-46df-bbef-c39d2f9f1467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:23.175552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536278b8-16a8-46df-bbef-c39d2f9f1467 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.175832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536278b8-16a8-46df-bbef-c39d2f9f1467 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.176100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536278b8-16a8-46df-bbef-c39d2f9f1467 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.176778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536278b8-16a8-46df-bbef-c39d2f9f1467 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:23.177077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536278b8-16a8-46df-bbef-c39d2f9f1467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:23.178515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536278b8-16a8-46df-bbef-c39d2f9f1467 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:23.183503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:23.184061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.184399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.184675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.185383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:23.185709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:23.187100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:23.200382 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 48/105] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:23.201758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf358b-5cc0-4b4b-bd62-6c6724a9709f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.204681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf358b-5cc0-4b4b-bd62-6c6724a9709f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.205176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf358b-5cc0-4b4b-bd62-6c6724a9709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.208505 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 58/106] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:23.209207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:23.240272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:23.246494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-11bf358b-5cc0-4b4b-bd62-6c6724a9709f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:23.247750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-11bf358b-5cc0-4b4b-bd62-6c6724a9709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:23.330407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5431ea7-8264-4ed2-9790-8ab735aebb8b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:23.365295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:23.365965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.366241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.366514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] PATH_INFO: `/v3/users/f4436c57d6334db7ba03902795ac6f08` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.383720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] RBAC: Authorizing `identity:get_user(user_id=f4436c57d6334db7ba03902795ac6f08)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:23.385321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:23.387316 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 59/107] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:23 2022] GET /identity/v3/users/f4436c57d6334db7ba03902795ac6f08 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:23.388507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:23.454110 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 49/108] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:23.454910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf358b-5cc0-4b4b-bd62-6c6724a9709f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:23.466605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:23.504416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:23.504738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5da24-e60e-4139-b952-5efbd3636c9c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:23.554511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b5ee4-957b-48f8-9b6b-fe5224137163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:23.555153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b5ee4-957b-48f8-9b6b-fe5224137163 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.555430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b5ee4-957b-48f8-9b6b-fe5224137163 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.555702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b5ee4-957b-48f8-9b6b-fe5224137163 None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.556068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf358b-5cc0-4b4b-bd62-6c6724a9709f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:23.556881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b5ee4-957b-48f8-9b6b-fe5224137163 None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:23.559749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b5ee4-957b-48f8-9b6b-fe5224137163 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:23.561613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 60/109] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:23 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:23.565604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923e78d-ef9d-49e6-af59-695887f4fd99 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.565921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923e78d-ef9d-49e6-af59-695887f4fd99 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.566197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923e78d-ef9d-49e6-af59-695887f4fd99 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.567683 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 61/110] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:23.568597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709ea0-b9e0-498d-988d-be31f24767fc None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.568872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709ea0-b9e0-498d-988d-be31f24767fc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.569140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709ea0-b9e0-498d-988d-be31f24767fc None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.573942 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 62/111] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:23.573942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c313674-d5eb-4a91-8909-11cb19c22aa8 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.573942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c313674-d5eb-4a91-8909-11cb19c22aa8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.573942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c313674-d5eb-4a91-8909-11cb19c22aa8 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.573942 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 63/112] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:23.573942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c313674-d5eb-4a91-8909-11cb19c22aa8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:23.602704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fb561-223f-42cd-9641-346643518763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:23.603326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fb561-223f-42cd-9641-346643518763 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.603603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fb561-223f-42cd-9641-346643518763 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.603874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fb561-223f-42cd-9641-346643518763 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.672933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fb561-223f-42cd-9641-346643518763 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:23.674813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fb561-223f-42cd-9641-346643518763 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:23.684995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c313674-d5eb-4a91-8909-11cb19c22aa8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:23.725201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a49cd-18ae-4976-a65f-f821d66541ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:23.726079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a49cd-18ae-4976-a65f-f821d66541ed None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.726505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a49cd-18ae-4976-a65f-f821d66541ed None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.726781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a49cd-18ae-4976-a65f-f821d66541ed None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.728016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a49cd-18ae-4976-a65f-f821d66541ed None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:23.729493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a49cd-18ae-4976-a65f-f821d66541ed None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:23.729616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:23.731589 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 64/113] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:23 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:23.737481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832f7a-42a0-4bb8-bedf-493b20cbaec5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.737771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832f7a-42a0-4bb8-bedf-493b20cbaec5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.738069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e832f7a-42a0-4bb8-bedf-493b20cbaec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.759289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9e832f7a-42a0-4bb8-bedf-493b20cbaec5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:23.760480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e832f7a-42a0-4bb8-bedf-493b20cbaec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:23.762978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:23.796074 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 50/114] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:23.796074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624aa84-b824-47f1-ac99-2fb5c5af1c31 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.796074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624aa84-b824-47f1-ac99-2fb5c5af1c31 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.796074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624aa84-b824-47f1-ac99-2fb5c5af1c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.810802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d624aa84-b824-47f1-ac99-2fb5c5af1c31 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:23.811983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d624aa84-b824-47f1-ac99-2fb5c5af1c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:23.834371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 65/115] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:23.835608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7828f0-d7ba-42a4-96c9-b57d60f8f871 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:23.835889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7828f0-d7ba-42a4-96c9-b57d60f8f871 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:23.836157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7828f0-d7ba-42a4-96c9-b57d60f8f871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:23.852931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cc7828f0-d7ba-42a4-96c9-b57d60f8f871 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:23.854121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cc7828f0-d7ba-42a4-96c9-b57d60f8f871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:23.890493 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 51/116] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:23.901567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624aa84-b824-47f1-ac99-2fb5c5af1c31 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:23.922232 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 66/117] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:23.923090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7828f0-d7ba-42a4-96c9-b57d60f8f871 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.031003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.040307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.075617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624aa84-b824-47f1-ac99-2fb5c5af1c31 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.099102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7828f0-d7ba-42a4-96c9-b57d60f8f871 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.136998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.137690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.137971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.138296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/a451c98d481d4228a63985e55b2d8871/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.140511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=54c0489c5be04e868df9a06af5b5ed5b, user_id=a451c98d481d4228a63985e55b2d8871, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:24.142027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:24.157768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.158457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.158739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.159012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.161119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:24.162616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:24.164199 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 67/118] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:24.164964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.166055 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 52/119] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:23 2022] PUT /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/a451c98d481d4228a63985e55b2d8871/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:33:24.174038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.323609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496c783-7665-4ac0-aa7c-4269dfdb34c2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.331624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7d9bf-f2e4-4bc1-9d23-e9db225371d6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.361804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.362460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.362734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.363002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.365332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:24.366900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:24.368815 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 68/120] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:24 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:24.369404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.374541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.380452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.381111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.381391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.381692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.386377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.400190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:24.406345 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:24.407769 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 53/121] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:24.408713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.494761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3838b-1172-4ea7-b41c-e52b4683f2fb None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.511317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb2b3c-9ba4-4d3e-a4bd-6dd101c063d7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.530311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.530917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.531191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.531459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.532470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:24.533970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:24.546416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.547048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.547322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.547588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.559663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:24.564416 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:24.569561 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 54/122] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:24.570567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.579734 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 69/123] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:33:24 2022] POST /identity/v3/projects => generated 302 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:24.580636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.648551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.654581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.715166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4934c6-12c4-489e-87c8-769dfad6e72b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.721147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384639f-decf-4388-af2f-0b7a0e2d7769 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.754379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.755243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.756573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.756858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.775025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.775957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.776381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.776795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.777933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] RBAC: Adding query filter params (role.id=e6b825f310f149b38baa30e8b96142fb, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=f4436c57d6334db7ba03902795ac6f08) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:24.778521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:24.778907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:24.780500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:24.787616 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 70/124] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:24 2022] GET /identity/v3/role_assignments?user.id=f4436c57d6334db7ba03902795ac6f08&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=e6b825f310f149b38baa30e8b96142fb => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:24.788929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:24.793031 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:24.794048 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:24.794891 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:24.798860 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 55/125] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:33:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:24.802823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:24.949678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31623d-970f-4988-bb7b-896d484df1f2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.965311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555b14d-2cdb-45e1-a334-9acf9c3a71b7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:24.975734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.993019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:24.995592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:24.996304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:24.996650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:24.996985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:24.997860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:24.998234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:24.999759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.010099 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 71/126] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:25.011126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.021449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:25.022115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:25.022395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:25.022664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:25.023344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:25.023647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:25.025084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.041506 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 56/127] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:25.042338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.168361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddf4f7-1ae5-4a5b-8df4-227a5b5438a7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:25.194606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd48da-bbf7-4e79-9124-3b7bc8ca3a95 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:25.229237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:25.229958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:25.230237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:25.230508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:25.231186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:25.231485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:25.232914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.243824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:25.244471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:25.244750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:25.245020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:25.247070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:25.248619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.252038 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 57/128] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:25 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:25.252659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.253853 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 72/129] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:33:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:25.260941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.344593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:25.345258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:25.401490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6bdaa-2550-47f8-8c2d-0e4db1c7ec5e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:25.437496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:25.437719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:25.437813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:25.438086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] PATH_INFO: `/v3/projects/3c868b98d526469f89292a8c01d12dcc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:25.439223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] RBAC: Authorizing `identity:get_project(project_id=3c868b98d526469f89292a8c01d12dcc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:25.440765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.442699 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 73/130] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:25 2022] GET /identity/v3/projects/3c868b98d526469f89292a8c01d12dcc => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:25.443829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.459063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcec86-f176-42db-bfff-0df0a07b6373 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:25.498757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:25.499684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:25.500133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:25.500584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] PATH_INFO: `/v3/users/fef6fab9131d4c1e89e0107005eb3caa` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:25.501914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] RBAC: Authorizing `identity:get_user(user_id=fef6fab9131d4c1e89e0107005eb3caa)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:25.503791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.505705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 58/131] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:25 2022] GET /identity/v3/users/fef6fab9131d4c1e89e0107005eb3caa => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:25.506901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.660449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d757b22-22bb-4cc4-9b7c-5da0a844c757 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:25.679641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809be2e-0582-4dfe-97f7-66c0a3277dea None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:25.697483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:25.711055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:25.731016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:25.731664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:25.731981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:25.732257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:25.735896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:25.738227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.739849 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 74/132] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:25 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:25.745579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.756198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:25.756820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:25.757154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:25.757433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] PATH_INFO: `/v3/projects/3c868b98d526469f89292a8c01d12dcc/users/a451c98d481d4228a63985e55b2d8871/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:25.762101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] RBAC: Authorizing `identity:create_grant(project_id=3c868b98d526469f89292a8c01d12dcc, user_id=a451c98d481d4228a63985e55b2d8871, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:25.763784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:25.788728 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 59/133] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:25 2022] PUT /identity/v3/projects/3c868b98d526469f89292a8c01d12dcc/users/a451c98d481d4228a63985e55b2d8871/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:33:25.789684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:25.926125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdabd5-dbc2-4836-ab65-d9c07506b73e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:25.992875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d37f9-c469-469e-9420-cc1a276ca9b7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:26.006398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:26.007049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:26.007278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:26.007558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:26.008842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:26.010359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:26.017484 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 60/134] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:25 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:26.018295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:26.074694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:26.075811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:26.076284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:26.076764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:26.078223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:26.079922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:26.087373 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 75/135] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:25 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:26.094221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:26.099438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:26.167148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0d87d-40c2-4177-a872-7e69694f381a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:26.208074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ba7fb-d32a-4f8a-ba51-73f8ac0c81d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:26.209067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba7fb-d32a-4f8a-ba51-73f8ac0c81d6 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:26.209357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba7fb-d32a-4f8a-ba51-73f8ac0c81d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:26.209667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba7fb-d32a-4f8a-ba51-73f8ac0c81d6 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/a451c98d481d4228a63985e55b2d8871/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:26.211168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:26.214087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ba7fb-d32a-4f8a-ba51-73f8ac0c81d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=54c0489c5be04e868df9a06af5b5ed5b, user_id=a451c98d481d4228a63985e55b2d8871, role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:26.215684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ba7fb-d32a-4f8a-ba51-73f8ac0c81d6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:26.247046 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 61/136] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:26 2022] PUT /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/a451c98d481d4228a63985e55b2d8871/roles/1b7585de1b6442509e3788682caf8774 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:33:26.290357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e9238-beef-4080-ba47-b8f6bead4a04 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:26.334022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573947e-f501-46f7-9c86-b67a5e473577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:26.335090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573947e-f501-46f7-9c86-b67a5e473577 None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:26.335596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573947e-f501-46f7-9c86-b67a5e473577 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:26.336037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573947e-f501-46f7-9c86-b67a5e473577 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/fef6fab9131d4c1e89e0107005eb3caa/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:26.338331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573947e-f501-46f7-9c86-b67a5e473577 None admin] RBAC: Authorizing `identity:create_grant(project_id=54c0489c5be04e868df9a06af5b5ed5b, user_id=fef6fab9131d4c1e89e0107005eb3caa, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:26.339995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573947e-f501-46f7-9c86-b67a5e473577 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:26.355950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 76/137] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:26 2022] PUT /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/fef6fab9131d4c1e89e0107005eb3caa/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:33:27.104799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dead3-2fbd-4def-ac92-83039b403cfe None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:27.105300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dead3-2fbd-4def-ac92-83039b403cfe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:27.105300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dead3-2fbd-4def-ac92-83039b403cfe None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:27.106651 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 62/138] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:27.112458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f783-f16f-49b2-a2cd-ef92b2d39f1a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:27.112943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f783-f16f-49b2-a2cd-ef92b2d39f1a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:27.113403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f783-f16f-49b2-a2cd-ef92b2d39f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:27.142573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-98b9f783-f16f-49b2-a2cd-ef92b2d39f1a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:27.144119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98b9f783-f16f-49b2-a2cd-ef92b2d39f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:27.228390 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 77/139] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:27.727650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dead3-2fbd-4def-ac92-83039b403cfe None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:27.853023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dead3-2fbd-4def-ac92-83039b403cfe None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:27.855283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:27.893701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ad815-5faa-4fcf-acdc-93609fe5b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:27.894776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ad815-5faa-4fcf-acdc-93609fe5b22d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:27.895243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ad815-5faa-4fcf-acdc-93609fe5b22d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:27.895756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ad815-5faa-4fcf-acdc-93609fe5b22d None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:27.906483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ad815-5faa-4fcf-acdc-93609fe5b22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-525ad815-5faa-4fcf-acdc-93609fe5b22d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:27.912168 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:27.913002 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:27.913808 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:27.916278 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 63/140] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:27.925482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f783-f16f-49b2-a2cd-ef92b2d39f1a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:27.995759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:28.085003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f783-f16f-49b2-a2cd-ef92b2d39f1a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:28.120725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c9644-cec2-49dc-a48a-4b62b468ff60 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:28.121767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c9644-cec2-49dc-a48a-4b62b468ff60 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:28.122242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c9644-cec2-49dc-a48a-4b62b468ff60 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:28.123911 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 64/141] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:28.126963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1d46-1418-4a93-9e4f-98e9b7d5b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:28.127586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1d46-1418-4a93-9e4f-98e9b7d5b702 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:28.128019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1d46-1418-4a93-9e4f-98e9b7d5b702 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:28.128125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1d46-1418-4a93-9e4f-98e9b7d5b702 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:28.128764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d1d46-1418-4a93-9e4f-98e9b7d5b702 None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:28.129061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d1d46-1418-4a93-9e4f-98e9b7d5b702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:28.129822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda8f2-5192-412c-a179-e1d1ceffcf8a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:28.130235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda8f2-5192-412c-a179-e1d1ceffcf8a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:28.130487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d1d46-1418-4a93-9e4f-98e9b7d5b702 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:28.130792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cda8f2-5192-412c-a179-e1d1ceffcf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:28.139810 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 78/142] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:28.141166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183caeb3-ec1f-4cfe-a74e-31980225e061 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:28.141450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183caeb3-ec1f-4cfe-a74e-31980225e061 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:28.141746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183caeb3-ec1f-4cfe-a74e-31980225e061 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:28.143032 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 79/143] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:28.148555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb9df9-83eb-40ef-a3ee-1ac4ceb47cf8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:28.148838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb9df9-83eb-40ef-a3ee-1ac4ceb47cf8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:28.149110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb9df9-83eb-40ef-a3ee-1ac4ceb47cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:28.168020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e2fb9df9-83eb-40ef-a3ee-1ac4ceb47cf8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:28.169187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-25cda8f2-5192-412c-a179-e1d1ceffcf8a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:28.169650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e2fb9df9-83eb-40ef-a3ee-1ac4ceb47cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:28.170609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25cda8f2-5192-412c-a179-e1d1ceffcf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:28.289966 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 80/144] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:28.292307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb9df9-83eb-40ef-a3ee-1ac4ceb47cf8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:28.299847 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 65/145] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:28.347275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:28.429192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb9df9-83eb-40ef-a3ee-1ac4ceb47cf8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:28.468918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:28.469585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:28.469864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:28.470133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:28.471270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:28.472732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:28.474531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 81/146] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:28 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:28.481253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cda8f2-5192-412c-a179-e1d1ceffcf8a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:28.596876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:28.607917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cda8f2-5192-412c-a179-e1d1ceffcf8a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:28.642660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:28.643629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:28.644098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:28.644556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:28.729842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:28.731746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:28.825996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:28.833332 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 66/147] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:28.839580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:28.949766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:28.974942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:28.981742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abc0a6-6dbc-41d5-ac6e-f89676356194 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:29.034246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:29.034954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.035155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.035425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.036632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:29.038910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:29.048705 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 82/148] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:28 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:29.048705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:29.075059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d1b7c-593d-47ba-b8ba-3900472d4762 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:29.110789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:29.111890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.112360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.112894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.114360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:29.116142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:29.117942 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 67/149] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:29.118969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:29.188617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:29.229867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1226075-4652-46ec-b86b-8355a2be5ef9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:29.232027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abe850-9a45-482b-b1f3-0da0dfc92c4d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:29.267457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:29.268435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.268936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.269404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.270406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=a451c98d481d4228a63985e55b2d8871) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:29.270896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:29.272639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:29.279103 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 68/150] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:29 2022] GET /identity/v3/role_assignments?user.id=a451c98d481d4228a63985e55b2d8871&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:29.280096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:29.300207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:29.300772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.301040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.301298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.339225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:29.344700 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:29.345667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:29.346439 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 83/151] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:29.354421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:29.426191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:29.539594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9314a6-5ee7-41c5-a641-e078512f6a68 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:29.555800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df76f-bff3-4360-9c0a-546580842224 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:29.589996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:29.598237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73100455-0872-4f14-92b0-a86c58bb682b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:29.599269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73100455-0872-4f14-92b0-a86c58bb682b None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.599775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73100455-0872-4f14-92b0-a86c58bb682b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.600233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73100455-0872-4f14-92b0-a86c58bb682b None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.601245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73100455-0872-4f14-92b0-a86c58bb682b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:29.603166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73100455-0872-4f14-92b0-a86c58bb682b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:29.603365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd00d8-b182-4cc1-b611-7d56d7c8eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:29.603948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd00d8-b182-4cc1-b611-7d56d7c8eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.604224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd00d8-b182-4cc1-b611-7d56d7c8eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.604498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd00d8-b182-4cc1-b611-7d56d7c8eaf1 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.605189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd00d8-b182-4cc1-b611-7d56d7c8eaf1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:29.605515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd00d8-b182-4cc1-b611-7d56d7c8eaf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:29.607028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd00d8-b182-4cc1-b611-7d56d7c8eaf1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:29.627970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 84/152] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:29.629520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a7d8-de64-49ed-ac8a-456d8c66bdbd None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.629809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a7d8-de64-49ed-ac8a-456d8c66bdbd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.630078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a7d8-de64-49ed-ac8a-456d8c66bdbd None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.631280 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 85/153] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:29.637145 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 69/154] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:33:29 2022] POST /identity/v3/users => generated 283 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:29.638175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a7d8-de64-49ed-ac8a-456d8c66bdbd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:29.639295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670499-45a9-4f29-8586-1817e3be1842 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.639775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670499-45a9-4f29-8586-1817e3be1842 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.640260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670499-45a9-4f29-8586-1817e3be1842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.661668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-29670499-45a9-4f29-8586-1817e3be1842 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:29.663383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-29670499-45a9-4f29-8586-1817e3be1842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:29.756962 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 70/155] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:29.851628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:29.874268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a7d8-de64-49ed-ac8a-456d8c66bdbd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:29.915517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6023-4036-42d7-8474-492e65d35667 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.916163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6023-4036-42d7-8474-492e65d35667 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.916312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:29.916637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6023-4036-42d7-8474-492e65d35667 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.916956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.917233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.917519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] PATH_INFO: `/v3/users/f4436c57d6334db7ba03902795ac6f08` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.918671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] RBAC: Authorizing `identity:get_user(user_id=f4436c57d6334db7ba03902795ac6f08)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:29.920230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:29.920360 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 71/156] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:29.922308 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 86/157] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:29 2022] GET /identity/v3/users/f4436c57d6334db7ba03902795ac6f08 => generated 289 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:29.926521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1e915-ebba-4991-9182-e3a383bc987c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:29.926797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1e915-ebba-4991-9182-e3a383bc987c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:29.927067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1e915-ebba-4991-9182-e3a383bc987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:29.927483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:29.967230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2ea1e915-ebba-4991-9182-e3a383bc987c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:29.968417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2ea1e915-ebba-4991-9182-e3a383bc987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:30.049782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56006b06-97d8-43bf-a76c-95efe0a808b9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:30.077539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 72/158] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:30.102591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb36c4d-d532-413c-a488-ac1016893658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:30.103574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36c4d-d532-413c-a488-ac1016893658 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.104103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36c4d-d532-413c-a488-ac1016893658 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.104613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36c4d-d532-413c-a488-ac1016893658 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.105999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb36c4d-d532-413c-a488-ac1016893658 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:30.107862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb36c4d-d532-413c-a488-ac1016893658 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:30.109801 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 87/159] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:29 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:30.118515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1e915-ebba-4991-9182-e3a383bc987c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:30.164149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:30.191915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4628bd0-a230-4ad2-85b8-7a9fd6d38858 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.192535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4628bd0-a230-4ad2-85b8-7a9fd6d38858 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.192998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4628bd0-a230-4ad2-85b8-7a9fd6d38858 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.194978 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 88/160] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:30.202839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2c585-74d9-4548-a939-87e54b738d22 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.203145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2c585-74d9-4548-a939-87e54b738d22 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.203544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2c585-74d9-4548-a939-87e54b738d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.229925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-aaa2c585-74d9-4548-a939-87e54b738d22 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:30.231115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aaa2c585-74d9-4548-a939-87e54b738d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:30.238631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1e915-ebba-4991-9182-e3a383bc987c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:30.251737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:30.279284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:30.279958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.280235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.280509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/f4436c57d6334db7ba03902795ac6f08/roles/e6b825f310f149b38baa30e8b96142fb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.282828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] RBAC: Authorizing `identity:create_grant(project_id=54c0489c5be04e868df9a06af5b5ed5b, user_id=f4436c57d6334db7ba03902795ac6f08, role_id=e6b825f310f149b38baa30e8b96142fb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:30.284367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:30.306376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 73/161] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:30 2022] PUT /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/users/f4436c57d6334db7ba03902795ac6f08/roles/e6b825f310f149b38baa30e8b96142fb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:33:30.307039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:30.336183 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 89/162] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:30.433820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf8077-3456-45a4-be30-003c2aeb44be None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:30.436030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:30.455286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2c585-74d9-4548-a939-87e54b738d22 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:30.491046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:30.491698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.491974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.492243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.506420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:30.512140 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:30.513147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:30.514022 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 74/163] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:30.521487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:30.624769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2c585-74d9-4548-a939-87e54b738d22 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:30.667829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015500d-2599-4190-be02-f64e982b412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:30.670694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015500d-2599-4190-be02-f64e982b412a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.670694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015500d-2599-4190-be02-f64e982b412a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.670694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015500d-2599-4190-be02-f64e982b412a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.680601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a35ca8-01c1-42e4-a6f7-3c51c1765b00 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:30.684438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015500d-2599-4190-be02-f64e982b412a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f015500d-2599-4190-be02-f64e982b412a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:30.689643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:30.690544 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:30.691310 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 90/164] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:30.699576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015500d-2599-4190-be02-f64e982b412a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:30.719582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:30.744345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:30.744973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.745244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.745652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.746797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:30.748425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:30.749022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:30.761970 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 75/165] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:30.768229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:30.817669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:30.863822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015500d-2599-4190-be02-f64e982b412a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:30.901211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26e7b9-bd39-4075-ac6e-7578f296c2fe None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:30.901733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:30.902417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.902690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.902996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.903666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:30.904084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:30.907519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:30.916545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 91/166] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:30.917385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:30.964036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:30.964534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:30.964811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:30.965082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:30.967672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:30.969159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:30.971029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 76/167] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:30 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:30.971674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:31.015112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:31.040036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd7ce5-49d7-4d62-956d-fa8a813a2d26 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:31.095437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:31.096705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:31.097887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:31.098526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:31.119441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:31.120112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:31.127980 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:31.128994 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:31.129834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:31.131186 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 92/168] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:33:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:31.131751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:31.141911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a177-e2c9-4b89-9253-90c63c85a324 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:31.198424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:31.199441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:31.199923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:31.200385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] PATH_INFO: `/v3/users/fef6fab9131d4c1e89e0107005eb3caa` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:31.201805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] RBAC: Authorizing `identity:get_user(user_id=fef6fab9131d4c1e89e0107005eb3caa)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:31.203965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:31.206125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 77/169] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:30 2022] GET /identity/v3/users/fef6fab9131d4c1e89e0107005eb3caa => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:31.207317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:31.296590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d669ffa-59ae-4795-a02e-ac2ccf099954 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:31.346492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:31.347134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:31.347381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:31.347631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:31.359176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:31.376170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cefa86-856d-4dda-93ac-eb358e69bb52 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:31.419028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:31.420643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:31.457181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:31.458182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:31.458759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:31.459234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:31.460095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:31.460588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:31.462213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:31.484986 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 78/170] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:33:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:31.486196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:31.530103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:31.611114 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 93/171] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:31.611915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:31.685872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:31.720559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccca6f6-c233-4bd2-bde2-3d862a34a77e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:31.780034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c6069-cb70-4326-b3f6-ad0dff45d10c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:31.808738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:31.809758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:31.810321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:31.810779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:31.816271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:31.816271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:31.816271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:31.816271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] PATH_INFO: `/v3/users/a451c98d481d4228a63985e55b2d8871` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:31.817349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] RBAC: Authorizing `identity:get_user(user_id=a451c98d481d4228a63985e55b2d8871)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:31.819118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:31.821158 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 94/172] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:31 2022] GET /identity/v3/users/a451c98d481d4228a63985e55b2d8871 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:31.829506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:31.935455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:31.936945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:31.959334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a393b06-dc0c-41e2-85f5-4a6b09446a2e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:31.966528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:31.988577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:32.039456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:32.040204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:32.040487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:32.040757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] PATH_INFO: `/v3/projects/3c868b98d526469f89292a8c01d12dcc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:32.044716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] RBAC: Authorizing `identity:get_project(project_id=3c868b98d526469f89292a8c01d12dcc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:32.046205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:32.048167 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 95/173] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:31 2022] GET /identity/v3/projects/3c868b98d526469f89292a8c01d12dcc => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:32.048944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:32.148027 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 79/174] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:32.149238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:32.247807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd28b-dff0-4dc5-9ba4-4cef11e8f77e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:32.306182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:32.307303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:32.307847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:32.308123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:32.343512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae0353-5b38-4100-a171-8cdd03a6ddce None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:32.355075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:32.382401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:32.383888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:32.384563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:32.438179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:32.438808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:32.439085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:32.439356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:32.440099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=3c868b98d526469f89292a8c01d12dcc, user.id=a451c98d481d4228a63985e55b2d8871) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:32.440402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:32.441914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:32.448096 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 80/175] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:32 2022] GET /identity/v3/role_assignments?user.id=a451c98d481d4228a63985e55b2d8871&scope.project.id=3c868b98d526469f89292a8c01d12dcc&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:32.454348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:32.513770 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 96/176] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:32.523543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:32.601138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:32.635141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dabd57-f164-42ca-86b2-bfbf2c20290b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:32.691105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd71fc3-d665-4dbc-9b5b-72c10b80e6f2 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:32.723388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:32.724102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:32.724385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:32.724655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:32.726289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:32.728707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:32.732279 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 97/177] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:32 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:32.739318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:32.775618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:32.776180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:32.776459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:32.776733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:32.778682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:32.780178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:32.785126 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 81/178] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:32 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:32.794837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:32.925906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:32.953851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ca71b-a8de-48da-82c1-9e1a51f062f1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:33.027779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10148e9-fa63-4473-bed9-452e62d7cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:33.028337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10148e9-fa63-4473-bed9-452e62d7cb8c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:33.028631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10148e9-fa63-4473-bed9-452e62d7cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:33.028961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10148e9-fa63-4473-bed9-452e62d7cb8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:33.029732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10148e9-fa63-4473-bed9-452e62d7cb8c None admin] RBAC: Adding query filter params (role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=a451c98d481d4228a63985e55b2d8871) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:33.030042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10148e9-fa63-4473-bed9-452e62d7cb8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:33.030282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402b4e8-c40d-44a3-bbbc-535a8424cb81 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:33.031510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10148e9-fa63-4473-bed9-452e62d7cb8c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:33.051897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 98/179] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:32 2022] GET /identity/v3/role_assignments?user.id=a451c98d481d4228a63985e55b2d8871&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=1b7585de1b6442509e3788682caf8774 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:33.084913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d97e02-5da4-49ff-9810-dced0bdef2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:33.085579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d97e02-5da4-49ff-9810-dced0bdef2ef None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:33.085864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d97e02-5da4-49ff-9810-dced0bdef2ef None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:33.086118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d97e02-5da4-49ff-9810-dced0bdef2ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:33.086869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d97e02-5da4-49ff-9810-dced0bdef2ef None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=fef6fab9131d4c1e89e0107005eb3caa) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:33.087169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d97e02-5da4-49ff-9810-dced0bdef2ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:33.088686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d97e02-5da4-49ff-9810-dced0bdef2ef None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:33.102067 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 82/180] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:32 2022] GET /identity/v3/role_assignments?user.id=fef6fab9131d4c1e89e0107005eb3caa&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:33.680306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd6347-0333-4a43-8bf0-4bb68169418c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:33.680864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd6347-0333-4a43-8bf0-4bb68169418c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:33.680864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd6347-0333-4a43-8bf0-4bb68169418c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:33.682250 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 99/181] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:33.693591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff28c9-82a2-4b2a-a03d-3aa5985a3ee6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:33.694526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff28c9-82a2-4b2a-a03d-3aa5985a3ee6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:33.694987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff28c9-82a2-4b2a-a03d-3aa5985a3ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:33.717206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-04ff28c9-82a2-4b2a-a03d-3aa5985a3ee6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:33.718801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-04ff28c9-82a2-4b2a-a03d-3aa5985a3ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:33.796305 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 83/182] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:33.847851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd55c1a-5ca4-4d89-b714-54e95d132ee4 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:33.848470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd55c1a-5ca4-4d89-b714-54e95d132ee4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:33.848950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd55c1a-5ca4-4d89-b714-54e95d132ee4 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:33.864174 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 100/183] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:33 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:33.870876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60c79f-3813-475f-a302-6bfe87ff6c89 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:33.871158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60c79f-3813-475f-a302-6bfe87ff6c89 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:33.871427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60c79f-3813-475f-a302-6bfe87ff6c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:33.894087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8f60c79f-3813-475f-a302-6bfe87ff6c89 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:33.895266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f60c79f-3813-475f-a302-6bfe87ff6c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:33.950081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:33.985965 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 84/184] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:34.154550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd55c1a-5ca4-4d89-b714-54e95d132ee4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.159294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:34.162241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbc579-5f34-46a3-865f-b64c89157d46 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.162754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbc579-5f34-46a3-865f-b64c89157d46 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.163209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbc579-5f34-46a3-865f-b64c89157d46 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.164740 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 85/185] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:34.171241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64005d-ecf3-404c-8044-f954eefcf5ae None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.171761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64005d-ecf3-404c-8044-f954eefcf5ae None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.172283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64005d-ecf3-404c-8044-f954eefcf5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.191677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1d64005d-ecf3-404c-8044-f954eefcf5ae None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:34.193265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1d64005d-ecf3-404c-8044-f954eefcf5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:34.265904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd55c1a-5ca4-4d89-b714-54e95d132ee4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.278996 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 86/186] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:34.303812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01841c90-6816-4bf5-91fd-b83f4682925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.304816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01841c90-6816-4bf5-91fd-b83f4682925f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.305285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01841c90-6816-4bf5-91fd-b83f4682925f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.305788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01841c90-6816-4bf5-91fd-b83f4682925f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.316764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01841c90-6816-4bf5-91fd-b83f4682925f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:34.320977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64005d-ecf3-404c-8044-f954eefcf5ae None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-01841c90-6816-4bf5-91fd-b83f4682925f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:34.321947 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:34.322861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:34.324001 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 101/187] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:34.324966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b4841-6774-4745-8e3e-4d2f54055107 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.324966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b4841-6774-4745-8e3e-4d2f54055107 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.324966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b4841-6774-4745-8e3e-4d2f54055107 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.326098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 102/188] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:34.328842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b4841-6774-4745-8e3e-4d2f54055107 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.393319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:34.425530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64005d-ecf3-404c-8044-f954eefcf5ae None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.433976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b4841-6774-4745-8e3e-4d2f54055107 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.434440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:34.462536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ec773-24c4-4074-b580-791045636dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.463173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ec773-24c4-4074-b580-791045636dca None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.463447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ec773-24c4-4074-b580-791045636dca None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.463895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ec773-24c4-4074-b580-791045636dca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.468866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4e6e7-3a2a-4adb-8e04-1aa590d718ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.469497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e6e7-3a2a-4adb-8e04-1aa590d718ee None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.469781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e6e7-3a2a-4adb-8e04-1aa590d718ee None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.470046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e6e7-3a2a-4adb-8e04-1aa590d718ee None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.470700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4e6e7-3a2a-4adb-8e04-1aa590d718ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:34.471017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4e6e7-3a2a-4adb-8e04-1aa590d718ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:34.472388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4e6e7-3a2a-4adb-8e04-1aa590d718ee None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:34.474403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ec773-24c4-4074-b580-791045636dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-131ec773-24c4-4074-b580-791045636dca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:34.479126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:34.480185 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:34.481179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:34.489074 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 103/189] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:34.489314 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 87/190] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:34.489448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bab2b-f3ba-461f-ae18-52860823982c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.489448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bab2b-f3ba-461f-ae18-52860823982c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.489448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bab2b-f3ba-461f-ae18-52860823982c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.490097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ec773-24c4-4074-b580-791045636dca None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.506221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-562bab2b-f3ba-461f-ae18-52860823982c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:34.507429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-562bab2b-f3ba-461f-ae18-52860823982c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:34.576001 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 104/191] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:34.577131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bab2b-f3ba-461f-ae18-52860823982c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.597680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ec773-24c4-4074-b580-791045636dca None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.635465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.636071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.636344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.636609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.637275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] RBAC: Adding query filter params (name=reader) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:34.637610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:34.639104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:34.647457 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 88/192] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:34.648235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.649083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:34.678496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:34.703873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bab2b-f3ba-461f-ae18-52860823982c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.740531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.741206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.741488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.741755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] PATH_INFO: `/v3/users/3e75b0a0e7f1488a91b60f65ab670d64` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.755147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] RBAC: Authorizing `identity:get_user(user_id=3e75b0a0e7f1488a91b60f65ab670d64)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:34.755829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd5699-ff43-443c-84cc-d2f22122165c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.756604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:34.758699 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 105/193] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/users/3e75b0a0e7f1488a91b60f65ab670d64 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:34.759075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.792482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.793118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.793512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.793794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.795039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:34.796793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:34.798536 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 89/194] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:34.801609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.875480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e385a-b68f-42fe-8ddd-96fba2faee59 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.890200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:34.902819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:34.910409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244780f5-624a-4864-aa14-44fea7a1e54c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:34.912051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.912692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.912963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.913236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] PATH_INFO: `/v3/users/f4436c57d6334db7ba03902795ac6f08` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:34.914398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] RBAC: Authorizing `identity:get_user(user_id=f4436c57d6334db7ba03902795ac6f08)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:34.915841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:34.917441 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 106/195] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/users/f4436c57d6334db7ba03902795ac6f08 => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:34.918199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:34.947478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:34.948091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:34.948371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:34.948633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.012195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3737e-4b88-4ced-b853-fc4337965749 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.013942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.015413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.046215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.046817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.047086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.047353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.047992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.049416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.070656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 107/196] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:33:34 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:35.071648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.110539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:35.119312 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 90/197] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:35.120278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.133316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:35.205102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37368a-7c72-43c7-ab13-be8368f1408e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.230415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee2906-56f4-4f9d-a1dd-1dce540a9b28 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.244040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.244692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.245107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.245524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.257488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:35.262431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:35.263480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:35.264282 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 108/198] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:35.264934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.275129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.275766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.276036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.276304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.356414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:35.364905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.367343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.375534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:35.379923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd649f3f-21e3-4e46-b81a-d6fd0b2a56e0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.433112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.433802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.434083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.434346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.436644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.438138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.439791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 109/199] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:35.443043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.494929 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 91/200] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:35.495791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.564729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b777d-ab51-4fa6-b5a6-9b67b35a189b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.596114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:35.601239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.601892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.602167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.602435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.603110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:35.603408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.604223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71341-c201-4b0b-9b48-f933ca2c9c2f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.604846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.614660 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 110/201] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:35.619902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.644070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.645062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.645586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.646059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.647132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=3e75b0a0e7f1488a91b60f65ab670d64) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:35.647660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.649434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.650306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:35.658262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 92/202] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/role_assignments?user.id=3e75b0a0e7f1488a91b60f65ab670d64&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:35.659368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.762341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47599f31-fe2a-4d17-a2c5-ba10d79a93e9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.773788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b2dbd-d9c2-44b5-86af-c6ecdfe5835e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:35.814402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.815560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.815840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.816113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] PATH_INFO: `/v3/users/3e75b0a0e7f1488a91b60f65ab670d64` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.823227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] RBAC: Authorizing `identity:get_user(user_id=3e75b0a0e7f1488a91b60f65ab670d64)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.823954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.825851 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 93/203] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/users/3e75b0a0e7f1488a91b60f65ab670d64 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:35.829320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:35.830027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:35.830302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:35.830572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:35.832080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.833074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:35.841936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:35.844022 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 111/204] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:35.850152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:35.890721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:36.005133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10657c4-f52d-4355-a1c2-d4c4283d38f6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:36.017536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b9c91-7f5d-4424-807a-aa274ef53e15 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:36.021775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:36.070959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:36.071587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:36.071866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:36.072138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:36.093799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:36.094420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:36.094695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:36.094963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:36.095706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] RBAC: Adding query filter params (role.id=e6b825f310f149b38baa30e8b96142fb, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b, user.id=f4436c57d6334db7ba03902795ac6f08) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:36.096008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:36.100647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:36.111895 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 112/205] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/role_assignments?user.id=f4436c57d6334db7ba03902795ac6f08&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=e6b825f310f149b38baa30e8b96142fb => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:36.193067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:36.194571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:36.241202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:36.349263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 94/206] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:33:36.355030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:36.488867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1d314-ab77-4863-8111-43b9bfff6967 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:36.533804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39a2ae-67b5-44f1-b99e-3ed6b0f9d1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:36.534459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a2ae-67b5-44f1-b99e-3ed6b0f9d1a1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:36.534733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a2ae-67b5-44f1-b99e-3ed6b0f9d1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:36.535002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39a2ae-67b5-44f1-b99e-3ed6b0f9d1a1 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:36.538081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39a2ae-67b5-44f1-b99e-3ed6b0f9d1a1 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:36.540224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39a2ae-67b5-44f1-b99e-3ed6b0f9d1a1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:36.542324 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 113/207] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:36 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:36.548089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:36.658731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cab7c-0cc7-4127-878d-ba1f316ff43c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:36.688629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:36.697132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa23307-6dab-4581-9409-6696721c1c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:36.697999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa23307-6dab-4581-9409-6696721c1c67 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:36.698419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa23307-6dab-4581-9409-6696721c1c67 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:36.698833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa23307-6dab-4581-9409-6696721c1c67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:36.699711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa23307-6dab-4581-9409-6696721c1c67 None admin] RBAC: Adding query filter params (role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=3e75b0a0e7f1488a91b60f65ab670d64) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:36.700179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa23307-6dab-4581-9409-6696721c1c67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:36.701892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa23307-6dab-4581-9409-6696721c1c67 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:36.707989 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 95/208] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:36 2022] GET /identity/v3/role_assignments?user.id=3e75b0a0e7f1488a91b60f65ab670d64&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=1b7585de1b6442509e3788682caf8774 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:37.797081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61815e-3940-4383-8058-03423397b424 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:37.797633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61815e-3940-4383-8058-03423397b424 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:37.797633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61815e-3940-4383-8058-03423397b424 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:37.799271 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 114/209] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:37.804704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4562f-0336-44c7-b8c6-696b6a9e3997 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:37.805580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4562f-0336-44c7-b8c6-696b6a9e3997 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:37.806058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4562f-0336-44c7-b8c6-696b6a9e3997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:37.821076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36e93f-a8aa-4797-8b83-d708fd3626d0 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:37.821548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36e93f-a8aa-4797-8b83-d708fd3626d0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:37.822038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36e93f-a8aa-4797-8b83-d708fd3626d0 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:37.823579 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 115/210] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:37.826458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-25f4562f-0336-44c7-b8c6-696b6a9e3997 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:37.827657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25f4562f-0336-44c7-b8c6-696b6a9e3997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:37.833968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60047a05-14fb-4cb3-81e0-9455e7d1b892 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:37.834509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60047a05-14fb-4cb3-81e0-9455e7d1b892 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:37.834980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60047a05-14fb-4cb3-81e0-9455e7d1b892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:37.853986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-60047a05-14fb-4cb3-81e0-9455e7d1b892 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:37.855573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-60047a05-14fb-4cb3-81e0-9455e7d1b892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:37.883415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:37.915700 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 96/211] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:37.939351 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 116/212] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:38.248897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4562f-0336-44c7-b8c6-696b6a9e3997 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:38.279590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60047a05-14fb-4cb3-81e0-9455e7d1b892 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:38.376111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4562f-0336-44c7-b8c6-696b6a9e3997 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:38.386850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60047a05-14fb-4cb3-81e0-9455e7d1b892 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:38.401761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:38.414869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90768f63-faa4-4d90-ba17-b7c274e0196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:38.415511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90768f63-faa4-4d90-ba17-b7c274e0196f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:38.415791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90768f63-faa4-4d90-ba17-b7c274e0196f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:38.416067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90768f63-faa4-4d90-ba17-b7c274e0196f None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:38.417192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90768f63-faa4-4d90-ba17-b7c274e0196f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:38.418859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90768f63-faa4-4d90-ba17-b7c274e0196f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:38.420660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 97/213] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:38.425578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644bae2-53d2-4c76-8f02-cf40d5251491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:38.426221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644bae2-53d2-4c76-8f02-cf40d5251491 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:38.426320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e2b38-af4e-4ea6-ae25-28e9da4cb4ab None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:38.426501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644bae2-53d2-4c76-8f02-cf40d5251491 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:38.426597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e2b38-af4e-4ea6-ae25-28e9da4cb4ab None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:38.426773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644bae2-53d2-4c76-8f02-cf40d5251491 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:38.426865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e2b38-af4e-4ea6-ae25-28e9da4cb4ab None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:38.429658 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 98/214] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:38.429658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e2b38-af4e-4ea6-ae25-28e9da4cb4ab None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:38.439504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644bae2-53d2-4c76-8f02-cf40d5251491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-e644bae2-53d2-4c76-8f02-cf40d5251491 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:38.445096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:38.446251 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:38.447271 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:38.449324 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 117/215] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:38.454834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c278f-f08e-4c77-8224-ba3d8614057d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:38.455117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c278f-f08e-4c77-8224-ba3d8614057d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:38.455392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c278f-f08e-4c77-8224-ba3d8614057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:38.473790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-695c278f-f08e-4c77-8224-ba3d8614057d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:38.474981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-695c278f-f08e-4c77-8224-ba3d8614057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:38.488888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:38.546316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e2b38-af4e-4ea6-ae25-28e9da4cb4ab None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:38.547028 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 118/216] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:38.547891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c278f-f08e-4c77-8224-ba3d8614057d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:38.606661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:38.607306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:38.607585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:38.607855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:38.608520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:38.610018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:38.635449 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 99/217] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:33:38 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:38.680887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c278f-f08e-4c77-8224-ba3d8614057d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:38.725855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:38.726746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:38.727177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:38.727782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:38.728645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:38.729130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:38.730782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:38.740829 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 119/218] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:38.751717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:38.807558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:38.870394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3d945-e084-41e5-9ab7-100d97e02e6a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:38.912264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:38.912913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:38.913193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:38.913595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] PATH_INFO: `/v3/users/e967e5d9815e426bbf271a1bee6f03e1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:38.919350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:38.927978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] RBAC: Authorizing `identity:get_user(user_id=e967e5d9815e426bbf271a1bee6f03e1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:38.929496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:38.931501 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 100/219] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:38 2022] GET /identity/v3/users/e967e5d9815e426bbf271a1bee6f03e1 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:38.935191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:38.937492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.047520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9522cf7-0bb8-4005-b80d-6e6744eecfe6 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:39.065419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982dbd7-843b-4f20-a6ef-b7b411b45e99 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:39.072689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:39.090242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:39.091168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.091448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.091718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.108048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:39.113671 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:39.114597 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:39.115922 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:39.116831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609afdb7-5b38-4859-af86-ea0ea4fd3e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:39.116831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609afdb7-5b38-4859-af86-ea0ea4fd3e40 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.116831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609afdb7-5b38-4859-af86-ea0ea4fd3e40 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.116831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609afdb7-5b38-4859-af86-ea0ea4fd3e40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.117128 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 120/220] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:39.121092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.202013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:39.205795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609afdb7-5b38-4859-af86-ea0ea4fd3e40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:39.207533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609afdb7-5b38-4859-af86-ea0ea4fd3e40 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:39.241029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ae7b4-d403-42ab-8cdd-22cba1fb5da6 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:39.285057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb504c-c209-484b-9ad2-8b921de7d55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:39.285853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb504c-c209-484b-9ad2-8b921de7d55f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.286329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb504c-c209-484b-9ad2-8b921de7d55f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.286682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb504c-c209-484b-9ad2-8b921de7d55f None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.287514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb504c-c209-484b-9ad2-8b921de7d55f None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:39.287883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb504c-c209-484b-9ad2-8b921de7d55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:39.289405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb504c-c209-484b-9ad2-8b921de7d55f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:39.299575 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 121/221] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:39.300917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940f0c-2497-4b6b-9531-4aa96606d5f1 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.301268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940f0c-2497-4b6b-9531-4aa96606d5f1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.301630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940f0c-2497-4b6b-9531-4aa96606d5f1 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.303218 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 122/222] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:39.308185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940f0c-2497-4b6b-9531-4aa96606d5f1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.331450 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 101/223] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:39.333036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb76bc9-1a9d-4093-b2b3-528bd84797cd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.333503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb76bc9-1a9d-4093-b2b3-528bd84797cd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.333953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb76bc9-1a9d-4093-b2b3-528bd84797cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.341499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:39.354667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5eb76bc9-1a9d-4093-b2b3-528bd84797cd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:39.356288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5eb76bc9-1a9d-4093-b2b3-528bd84797cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:39.437506 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 102/224] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:39.438736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940f0c-2497-4b6b-9531-4aa96606d5f1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:39.442303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb76bc9-1a9d-4093-b2b3-528bd84797cd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.490826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:39.497779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:39.498643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.499085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.499557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] PATH_INFO: `/v3/users/3e75b0a0e7f1488a91b60f65ab670d64` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.501385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] RBAC: Authorizing `identity:get_user(user_id=3e75b0a0e7f1488a91b60f65ab670d64)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:39.503018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:39.505076 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 123/225] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:39 2022] GET /identity/v3/users/3e75b0a0e7f1488a91b60f65ab670d64 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:39.517128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.576932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb76bc9-1a9d-4093-b2b3-528bd84797cd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:39.614027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:39.614959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.615412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.615853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.617172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:39.618836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:39.621167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 103/226] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:39 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:39.628427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.633785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:39.748377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a4a37-d0ee-4814-80d3-fb33072213bf None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:39.795057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f586-fd25-45f6-ada4-f46b4b486314 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:39.825071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:39.825821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.826103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.826377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.828718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:39.830571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:39.837565 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 124/227] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:39 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:39.844325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.845404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:39.846143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:39.846491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:39.846835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:39.847642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=e967e5d9815e426bbf271a1bee6f03e1) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:39.848032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:39.849607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:39.861763 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 104/228] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:39 2022] GET /identity/v3/role_assignments?user.id=e967e5d9815e426bbf271a1bee6f03e1&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:39.862738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:39.925367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:39.966752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4fdfc-2dd9-45bb-b506-1576b8d8860f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:40.008060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:40.009070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:40.009580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:40.010103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:40.023320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:40.024385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:40.030961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:40.032313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:40.033112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:40.036112 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 125/229] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:33:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:40.049784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:40.099299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0137bc-6a2e-4872-b663-2b8655f8e197 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:40.177052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:40.177695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:40.177985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:40.178239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/3e75b0a0e7f1488a91b60f65ab670d64/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:40.180242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb544378edc4844a8bfe4c58c6b8b04, user_id=3e75b0a0e7f1488a91b60f65ab670d64, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:40.181909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:40.201180 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 105/230] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:39 2022] PUT /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/3e75b0a0e7f1488a91b60f65ab670d64/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:33:40.268784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59796d-c1cc-43f7-9db8-fd4e1c0601a1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:40.308673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:40.323873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:40.324897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:40.325371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:40.326029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:40.326921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:40.327427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:40.329158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:40.342429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 126/231] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:33:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:40.348515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:40.489185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dfbdd-ed9c-4772-be0a-db26067287e6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:40.523324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:40.537183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:40.537860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:40.538140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:40.538414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] PATH_INFO: `/v3/users/3e75b0a0e7f1488a91b60f65ab670d64` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:40.539565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] RBAC: Authorizing `identity:get_user(user_id=3e75b0a0e7f1488a91b60f65ab670d64)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:40.541168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:40.542811 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 106/232] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:40 2022] GET /identity/v3/users/3e75b0a0e7f1488a91b60f65ab670d64 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:40.548727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:40.674745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b478e-63be-4b20-b251-d55e9e83019b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:40.709409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d638-4ddc-4ae2-b73f-c7bee61ba38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:40.710077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d638-4ddc-4ae2-b73f-c7bee61ba38a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:40.710351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d638-4ddc-4ae2-b73f-c7bee61ba38a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:40.710619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d638-4ddc-4ae2-b73f-c7bee61ba38a None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:40.711744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d638-4ddc-4ae2-b73f-c7bee61ba38a None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:40.713311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d638-4ddc-4ae2-b73f-c7bee61ba38a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:40.717445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 127/233] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:40 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:40.723180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:40.844236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31727290-e1c0-49d5-b889-fb828dc00c4f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:40.878531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3a7cd-638b-4b6a-abba-d36dfe1482e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:40.879292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3a7cd-638b-4b6a-abba-d36dfe1482e4 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:40.879475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3a7cd-638b-4b6a-abba-d36dfe1482e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:40.879752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3a7cd-638b-4b6a-abba-d36dfe1482e4 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/3e75b0a0e7f1488a91b60f65ab670d64/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:40.881966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3a7cd-638b-4b6a-abba-d36dfe1482e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb544378edc4844a8bfe4c58c6b8b04, user_id=3e75b0a0e7f1488a91b60f65ab670d64, role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:40.883434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3a7cd-638b-4b6a-abba-d36dfe1482e4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:40.904844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 107/234] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:40 2022] PUT /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/3e75b0a0e7f1488a91b60f65ab670d64/roles/1b7585de1b6442509e3788682caf8774 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:33:41.502051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cf742-6c5c-4040-98e6-d79f5a3c3aa9 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:41.503080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cf742-6c5c-4040-98e6-d79f5a3c3aa9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:41.503601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cf742-6c5c-4040-98e6-d79f5a3c3aa9 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:41.505156 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 128/235] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:41.510735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fab213-03a6-43c3-af4b-fc712d600eef None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:41.511246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fab213-03a6-43c3-af4b-fc712d600eef None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:41.511711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fab213-03a6-43c3-af4b-fc712d600eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:41.530686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-55fab213-03a6-43c3-af4b-fc712d600eef None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:41.532568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-55fab213-03a6-43c3-af4b-fc712d600eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:41.583541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:41.603245 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 108/236] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:41.821964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca24f7-772e-4071-b5b3-26f3b7dac323 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:41.822621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca24f7-772e-4071-b5b3-26f3b7dac323 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:41.823097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca24f7-772e-4071-b5b3-26f3b7dac323 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:41.824649 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 129/237] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:41.830200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dd075-71f6-4c30-8282-56efb3fa4559 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:41.830693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dd075-71f6-4c30-8282-56efb3fa4559 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:41.831178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dd075-71f6-4c30-8282-56efb3fa4559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:41.849696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-908dd075-71f6-4c30-8282-56efb3fa4559 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:41.851129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-908dd075-71f6-4c30-8282-56efb3fa4559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:41.931135 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 109/238] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:42.004378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca24f7-772e-4071-b5b3-26f3b7dac323 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:42.076389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:42.120952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca24f7-772e-4071-b5b3-26f3b7dac323 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:42.163253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:42.163895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:42.164172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:42.164444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:42.174905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:42.179863 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:42.180740 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:42.181552 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 130/239] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:42.187157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dd075-71f6-4c30-8282-56efb3fa4559 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:42.307874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dd075-71f6-4c30-8282-56efb3fa4559 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:42.326660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:42.347355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:42.347980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:42.348260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:42.348531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:42.349259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:42.349598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:42.351051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:42.359982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 110/240] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:42.365874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:42.414886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:42.479850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c9678-b305-48e3-953b-2b2114c1e410 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:42.501798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:42.516699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:42.518174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:42.518766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:42.519177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] PATH_INFO: `/v3/users/4996e526602e44c58c2bab9a329950cf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:42.535926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] RBAC: Authorizing `identity:get_user(user_id=4996e526602e44c58c2bab9a329950cf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:42.537776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:42.539789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 131/241] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:42 2022] GET /identity/v3/users/4996e526602e44c58c2bab9a329950cf => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:42.545976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:42.656058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb5bf-ce6a-483b-b4e0-d0681cf7db2f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:42.666609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f767382-865f-484b-b9e3-70d1f174a012 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:42.715649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:42.716628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:42.717046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:42.717481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:42.718877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:42.720611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:42.722784 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 111/242] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:42.726439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:42.727302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:42.728194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:42.728960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:42.729726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:42.816644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:42.824429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:42.844343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:42.846652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:42.954230 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 132/243] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:33:42.959739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:42.964315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69523186-4ccf-4da6-a8e2-cb8e60a86f6e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:43.046327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c6bc9-ad80-4e32-b121-48108b021910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:43.047194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c6bc9-ad80-4e32-b121-48108b021910 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.047693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c6bc9-ad80-4e32-b121-48108b021910 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.048102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c6bc9-ad80-4e32-b121-48108b021910 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.048898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c6bc9-ad80-4e32-b121-48108b021910 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:43.050509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c6bc9-ad80-4e32-b121-48108b021910 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:43.080515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 112/244] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:33:42 2022] POST /identity/v3/users => generated 293 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:43.083046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7376de-3bd1-4bce-a461-fee63ec8fbca None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.083535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7376de-3bd1-4bce-a461-fee63ec8fbca None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.084105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7376de-3bd1-4bce-a461-fee63ec8fbca None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.086114 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 113/245] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:43.092728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4db38-e34e-42b5-8ce4-5e706aa0fbbc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.093277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4db38-e34e-42b5-8ce4-5e706aa0fbbc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.093804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4db38-e34e-42b5-8ce4-5e706aa0fbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.099535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:43.119869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d5e4db38-e34e-42b5-8ce4-5e706aa0fbbc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:43.121869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d5e4db38-e34e-42b5-8ce4-5e706aa0fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:43.136324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807b0ef-d942-4978-a7b4-2edbb95e1b72 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:43.203884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 114/246] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:43.228551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97adcefa-83b7-4e77-94af-e90ee3f2b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:43.229206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adcefa-83b7-4e77-94af-e90ee3f2b94c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.229508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adcefa-83b7-4e77-94af-e90ee3f2b94c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.229789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adcefa-83b7-4e77-94af-e90ee3f2b94c None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.234149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97adcefa-83b7-4e77-94af-e90ee3f2b94c None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:43.235724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97adcefa-83b7-4e77-94af-e90ee3f2b94c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:43.237950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 133/247] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:42 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:43.244184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4db38-e34e-42b5-8ce4-5e706aa0fbbc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:43.279305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:43.356439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4db38-e34e-42b5-8ce4-5e706aa0fbbc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:43.393036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689dc08-dafe-4fd7-9cb3-4212547e55c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:43.394104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689dc08-dafe-4fd7-9cb3-4212547e55c0 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.394576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689dc08-dafe-4fd7-9cb3-4212547e55c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.395087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689dc08-dafe-4fd7-9cb3-4212547e55c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.396046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689dc08-dafe-4fd7-9cb3-4212547e55c0 None admin] RBAC: Adding query filter params (role.id=e6b825f310f149b38baa30e8b96142fb, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=4996e526602e44c58c2bab9a329950cf) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:43.396545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689dc08-dafe-4fd7-9cb3-4212547e55c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:43.398239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689dc08-dafe-4fd7-9cb3-4212547e55c0 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:43.404206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 115/248] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:43 2022] GET /identity/v3/role_assignments?user.id=4996e526602e44c58c2bab9a329950cf&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=e6b825f310f149b38baa30e8b96142fb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:43.470727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59d34-4094-48fe-9bc0-5d8e44e2f933 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.471078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59d34-4094-48fe-9bc0-5d8e44e2f933 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.471275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f59d34-4094-48fe-9bc0-5d8e44e2f933 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.472639 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 134/249] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:43.478176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00daf3-eef1-4b46-884c-078ac6aea256 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.478800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00daf3-eef1-4b46-884c-078ac6aea256 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.479286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00daf3-eef1-4b46-884c-078ac6aea256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.500802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4c00daf3-eef1-4b46-884c-078ac6aea256 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:43.502301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4c00daf3-eef1-4b46-884c-078ac6aea256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:43.584476 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 116/250] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:43.598346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59d34-4094-48fe-9bc0-5d8e44e2f933 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:43.726281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f59d34-4094-48fe-9bc0-5d8e44e2f933 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:43.766794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:43.767348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.767620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.767887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.778862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:43.784079 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:43.784983 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:43.785850 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 135/251] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:43.791058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00daf3-eef1-4b46-884c-078ac6aea256 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:43.820446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:43.899695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00daf3-eef1-4b46-884c-078ac6aea256 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:43.935890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:43.936513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:43.936790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:43.937057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:43.937865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:43.939088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:43.939942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:43.941499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:43.949667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:43.950693 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 117/252] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:43.959413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:44.063660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a988c3-00d1-4fe3-ab66-c932e6ceb75b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:44.064742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:44.081291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e3358-5c6a-4a03-b159-c788817282cb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:44.106312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4d14d-ddee-4541-8aba-ec468a7466a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:44.107329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4d14d-ddee-4541-8aba-ec468a7466a5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.107744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4d14d-ddee-4541-8aba-ec468a7466a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.108240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4d14d-ddee-4541-8aba-ec468a7466a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.116651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772a0c9-b027-4932-86f3-37fa8f8c5288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:44.117208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772a0c9-b027-4932-86f3-37fa8f8c5288 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.117502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772a0c9-b027-4932-86f3-37fa8f8c5288 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.117780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772a0c9-b027-4932-86f3-37fa8f8c5288 None admin] PATH_INFO: `/v3/users/e967e5d9815e426bbf271a1bee6f03e1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.118923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772a0c9-b027-4932-86f3-37fa8f8c5288 None admin] RBAC: Authorizing `identity:get_user(user_id=e967e5d9815e426bbf271a1bee6f03e1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:44.120398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772a0c9-b027-4932-86f3-37fa8f8c5288 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:44.121898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4d14d-ddee-4541-8aba-ec468a7466a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:44.122110 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 118/253] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:43 2022] GET /identity/v3/users/e967e5d9815e426bbf271a1bee6f03e1 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:44.123621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb46d0-78ae-4af8-ae0e-67e5b7786119 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.123913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb46d0-78ae-4af8-ae0e-67e5b7786119 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.124179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb46d0-78ae-4af8-ae0e-67e5b7786119 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.125526 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 119/254] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-16f4d14d-ddee-4541-8aba-ec468a7466a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:44.128370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:44.129258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:33:44.130151 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:44.130770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb46d0-78ae-4af8-ae0e-67e5b7786119 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:44.132831 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 136/255] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:33:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:44.138700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f3ff2-768f-4be9-b4e6-23b5f9138d9a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.139268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f3ff2-768f-4be9-b4e6-23b5f9138d9a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.139730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f3ff2-768f-4be9-b4e6-23b5f9138d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.162745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e26f3ff2-768f-4be9-b4e6-23b5f9138d9a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:44.164375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e26f3ff2-768f-4be9-b4e6-23b5f9138d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:44.290758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb46d0-78ae-4af8-ae0e-67e5b7786119 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:44.291417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:44.301194 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 137/256] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:44.302374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f3ff2-768f-4be9-b4e6-23b5f9138d9a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:44.352757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:44.353329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.353684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.353957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.355130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:44.356710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:44.360010 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 120/257] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:44 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:44.364568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:44.412662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:44.468300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f3ff2-768f-4be9-b4e6-23b5f9138d9a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:44.527735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f5ba8-c773-4d1a-ad54-4060f9c3b93d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:44.534268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:44.534914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.535197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.535466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.536150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:44.536454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:44.537883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:44.551162 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 138/258] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:44.562577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:44.616749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:44.617430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.617737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.618012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/e967e5d9815e426bbf271a1bee6f03e1/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.625944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb544378edc4844a8bfe4c58c6b8b04, user_id=e967e5d9815e426bbf271a1bee6f03e1, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:44.627425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:44.673585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:44.674675 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 121/259] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:44 2022] PUT /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/e967e5d9815e426bbf271a1bee6f03e1/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:33:44.737198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44345062-c4ad-4f23-a3ef-be55b9ceeea6 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:44.773763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:44.774515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:44.774802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:44.775069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] PATH_INFO: `/v3/users/3e75b0a0e7f1488a91b60f65ab670d64` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:44.776249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=3e75b0a0e7f1488a91b60f65ab670d64)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:44.777757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:44.779410 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 139/260] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:44 2022] GET /identity/v3/users/3e75b0a0e7f1488a91b60f65ab670d64 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:44.786970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:44.855957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:44.940487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:44.970450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535367c2-2371-4793-a29d-ecd9c1e00615 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:45.025857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa25489-0d38-40b9-affb-c6d20669e2a0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:45.035558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:45.036210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:45.036497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:45.036762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:45.067718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:45.106354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:45.107116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:45.107404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:45.107678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:45.134939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:45.140463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:45.141345 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:45.142200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:45.144471 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 140/261] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:33:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:45.151327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:45.168476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:45.170058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:45.292281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:45.319909 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 122/262] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:45.323257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b1fb8-944b-44f4-997b-a7783c81d2ea None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:45.325885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:45.385411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:45.386573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:45.387116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:45.387576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:45.388536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:45.389030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:45.390888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:45.411813 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 141/263] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:33:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:45.417838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:45.494799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:45.513928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cd1ce-4072-4e9b-95dd-340ef5ab6980 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:45.561734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da958ec4-f241-403d-bd44-a854d7120c5a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:45.595217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:45.595874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:45.596154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:45.596425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:45.597862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:45.599346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:45.602360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:45.603007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:45.603285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:45.603556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] PATH_INFO: `/v3/users/3e75b0a0e7f1488a91b60f65ab670d64` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:45.604900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] RBAC: Authorizing `identity:get_user(user_id=3e75b0a0e7f1488a91b60f65ab670d64)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:45.606493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:45.608239 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 123/264] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:45 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:45.609130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:45.609741 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 142/265] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:45 2022] GET /identity/v3/users/3e75b0a0e7f1488a91b60f65ab670d64 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:45.613771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:45.715546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:45.799880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fadcc-9201-49ad-a49d-2161728aee14 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:45.809670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00633dba-c504-46e4-87aa-625a49c62657 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:45.828849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:45.863247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:45.863867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:45.864147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:45.864454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:45.868793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:45.869409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:45.869709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:45.869983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:45.870725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=3e75b0a0e7f1488a91b60f65ab670d64) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:45.871031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:45.872545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:45.889262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 124/266] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:45 2022] GET /identity/v3/role_assignments?user.id=3e75b0a0e7f1488a91b60f65ab670d64&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:45.985506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:45.987033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:46.126521 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 143/267] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:33:46.131888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:46.251072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af83f-6bbf-4008-b23f-22f1a944a0cd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:46.267083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:46.294895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23f185-04d8-4bb1-ab3d-d3505552aec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:46.295550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23f185-04d8-4bb1-ab3d-d3505552aec8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:46.295832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23f185-04d8-4bb1-ab3d-d3505552aec8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:46.296106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23f185-04d8-4bb1-ab3d-d3505552aec8 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:46.297256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23f185-04d8-4bb1-ab3d-d3505552aec8 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:46.298771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23f185-04d8-4bb1-ab3d-d3505552aec8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:46.300748 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 125/268] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:46 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:46.306542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:46.322094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:46.407529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c35d1f-a073-46b6-bf83-f9261912e3e8 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:46.407702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c35d1f-a073-46b6-bf83-f9261912e3e8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:46.407984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c35d1f-a073-46b6-bf83-f9261912e3e8 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:46.409355 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 126/269] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:46.414786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070600-3b6b-4707-a535-d917dced8097 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:46.415148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070600-3b6b-4707-a535-d917dced8097 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:46.415513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070600-3b6b-4707-a535-d917dced8097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:46.445554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a70d7-62d4-4e93-8618-70e021d7413d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:46.462541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-08070600-3b6b-4707-a535-d917dced8097 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:46.463738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-08070600-3b6b-4707-a535-d917dced8097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:46.499337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:46.499956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:46.500232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:46.500510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:46.501237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] RBAC: Adding query filter params (role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=3e75b0a0e7f1488a91b60f65ab670d64) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:46.501564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:46.503019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:46.517508 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 144/270] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:46 2022] GET /identity/v3/role_assignments?user.id=3e75b0a0e7f1488a91b60f65ab670d64&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=1b7585de1b6442509e3788682caf8774 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:46.566544 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 127/271] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:47.048190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.143463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:47.161627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472e6b5-6e5e-4778-860b-b07ee0b694c0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.161975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472e6b5-6e5e-4778-860b-b07ee0b694c0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.162331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472e6b5-6e5e-4778-860b-b07ee0b694c0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.163792 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 128/272] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:33:47.165869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28374729-241b-4ac9-b770-80dad6e94f21 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:47.168636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a372d9-1373-4394-a32d-1328939403e0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.168990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a372d9-1373-4394-a32d-1328939403e0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.169399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a372d9-1373-4394-a32d-1328939403e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.190686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c4a372d9-1373-4394-a32d-1328939403e0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:47.192433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c4a372d9-1373-4394-a32d-1328939403e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:47.202131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:47.202775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.203051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.203320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.215417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:47.223506 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:47.224423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:47.225233 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:47.229550 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 145/273] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:47.231906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.281887 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 129/274] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:47.342672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f417-7c49-449a-9ac6-16b5d892d3e2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:47.380781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:47.381394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.381703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.381974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.382645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:47.382946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:47.384344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:47.386063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3065453-190e-4636-be8d-3f0a6c04857d None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.386347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3065453-190e-4636-be8d-3f0a6c04857d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.386617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3065453-190e-4636-be8d-3f0a6c04857d None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.388309 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 130/275] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:47.393054 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 146/276] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:47.393501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:47.398137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b3d09-35cc-4f48-9b0b-4d6ef081399d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.398521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b3d09-35cc-4f48-9b0b-4d6ef081399d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.398812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b3d09-35cc-4f48-9b0b-4d6ef081399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.399118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.421107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8b7b3d09-35cc-4f48-9b0b-4d6ef081399d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:47.422374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8b7b3d09-35cc-4f48-9b0b-4d6ef081399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:47.458952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:47.492646 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 131/277] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:47.510711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e9e8a-da43-454b-ba91-4b98c9723e71 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:47.549775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:47.550684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.551132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.551615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] PATH_INFO: `/v3/users/4996e526602e44c58c2bab9a329950cf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.552940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] RBAC: Authorizing `identity:get_user(user_id=4996e526602e44c58c2bab9a329950cf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:47.554639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:47.556505 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 147/278] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:47 2022] GET /identity/v3/users/4996e526602e44c58c2bab9a329950cf => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:47.562343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b3d09-35cc-4f48-9b0b-4d6ef081399d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.633778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.666117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b3d09-35cc-4f48-9b0b-4d6ef081399d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:47.713299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:47.713995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.714273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.714549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.715729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:47.717331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:47.719035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:47.719186 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 132/279] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:47 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:47.725873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.761762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf4038-2692-4825-889a-75cfba9496c7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:47.769841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:47.803496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:47.804381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.804799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.805830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.818621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:47.824578 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:47.825635 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 148/280] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:47.827561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.836611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c535d8-9393-4c77-998a-f692a7660f11 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:47.874174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:47.874868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.875120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.875393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/4996e526602e44c58c2bab9a329950cf/roles/e6b825f310f149b38baa30e8b96142fb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.877282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb544378edc4844a8bfe4c58c6b8b04, user_id=4996e526602e44c58c2bab9a329950cf, role_id=e6b825f310f149b38baa30e8b96142fb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:47.878794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:47.892425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 133/281] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:33:47 2022] PUT /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/users/4996e526602e44c58c2bab9a329950cf/roles/e6b825f310f149b38baa30e8b96142fb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:33:47.898452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:47.946468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524d8f6-8bef-41c9-96f7-c16f1721f261 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:47.979107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:47.989701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:47.990338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:47.990618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:47.990891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:47.996784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:47.998563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:48.000867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 149/282] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:48.006481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:48.026885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:48.034732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce4ab5-ec8b-4359-a4e0-3ab3e43093b9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:48.085571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:48.087106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:48.087880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:48.088393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:48.089565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:48.089905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:48.091476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:48.104814 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 134/283] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:48.110350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:48.133447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413df44-7c13-45a1-8eef-a02705d92e86 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:48.173570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18810ca9-14b3-46ff-969c-017a3cb4c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:48.173570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18810ca9-14b3-46ff-969c-017a3cb4c9c9 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:48.173570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18810ca9-14b3-46ff-969c-017a3cb4c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:48.173570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18810ca9-14b3-46ff-969c-017a3cb4c9c9 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:48.173963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18810ca9-14b3-46ff-969c-017a3cb4c9c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:48.173963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18810ca9-14b3-46ff-969c-017a3cb4c9c9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:48.196994 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 150/284] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:33:48 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:48.198954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84099d6c-bfed-4d23-b41d-05174e1d0466 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:48.199240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84099d6c-bfed-4d23-b41d-05174e1d0466 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:48.199514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84099d6c-bfed-4d23-b41d-05174e1d0466 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:48.201136 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 151/285] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:48.211810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041c46-9bd5-44e5-b91a-6cec458ed530 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:48.212685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041c46-9bd5-44e5-b91a-6cec458ed530 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:48.213128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041c46-9bd5-44e5-b91a-6cec458ed530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:48.224431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b94ac-6f3b-48d9-b537-7dab24f96c11 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:48.244032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8d041c46-9bd5-44e5-b91a-6cec458ed530 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:48.246783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8d041c46-9bd5-44e5-b91a-6cec458ed530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:48.275047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:48.275719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:48.275991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:48.276263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] PATH_INFO: `/v3/users/13b098f5f3cf46aebe25a5fa5ffe8225` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:48.298117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] RBAC: Authorizing `identity:get_user(user_id=13b098f5f3cf46aebe25a5fa5ffe8225)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:48.300010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:48.302528 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 135/286] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:48 2022] GET /identity/v3/users/13b098f5f3cf46aebe25a5fa5ffe8225 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:48.308599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:48.322617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:48.344855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:48.357237 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 152/287] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:48.440132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f51d8-be80-4b2f-b8e2-41d34138581d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:48.476445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:48.477325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:48.477780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:48.478200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:48.479085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, user.id=13b098f5f3cf46aebe25a5fa5ffe8225, scope.system=all) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:48.479552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:48.481171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:48.487145 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 136/288] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 17 20:33:48 2022] GET /identity/v3/role_assignments?user.id=13b098f5f3cf46aebe25a5fa5ffe8225&scope.system=all&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:48.688657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d041c46-9bd5-44e5-b91a-6cec458ed530 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:48.796240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d041c46-9bd5-44e5-b91a-6cec458ed530 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:48.837790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:48.838445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:48.838726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:48.838998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:48.850343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:48.855229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:48.856093 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:48.857033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 153/289] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:48.866600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:48.995783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:49.017634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9b79b-95ea-4a7d-b2fe-45a81a756141 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:49.061125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:49.061839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:49.062124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:49.062396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:49.063078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:49.063377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:49.064769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:49.077651 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 137/290] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:49.083706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:49.135494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:49.200810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa308975-fc88-4ca0-964d-79e95067946b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:49.240855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:49.241520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:49.241926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:49.242199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] PATH_INFO: `/v3/users/e967e5d9815e426bbf271a1bee6f03e1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:49.243353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] RBAC: Authorizing `identity:get_user(user_id=e967e5d9815e426bbf271a1bee6f03e1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:49.244840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:49.246544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 154/291] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:49 2022] GET /identity/v3/users/e967e5d9815e426bbf271a1bee6f03e1 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:49.252697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:49.373292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0495a9-f025-48c2-abd1-2ecae67b594b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:49.421986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:49.422671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:49.422953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:49.423221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:49.445651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:49.512486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:49.518044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:49.657609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 138/292] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:49.663307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:49.797490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e2ea8-02b1-4d89-a674-b93fd2bc0081 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:49.811492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:49.847193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f46d7-7fc7-4a50-a940-f846c9e725cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:49.847857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f46d7-7fc7-4a50-a940-f846c9e725cd None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:49.848139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f46d7-7fc7-4a50-a940-f846c9e725cd None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:49.848408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f46d7-7fc7-4a50-a940-f846c9e725cd None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:49.849591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f46d7-7fc7-4a50-a940-f846c9e725cd None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:49.851033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f46d7-7fc7-4a50-a940-f846c9e725cd None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:49.852981 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 155/293] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:49 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:49.858995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:49.936656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:50.010460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643246f-72f8-4590-9a55-77b42a8da7d6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:50.058861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c098463-cd4a-49fc-958a-8a56bb746a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:50.059402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c098463-cd4a-49fc-958a-8a56bb746a8d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:50.059728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c098463-cd4a-49fc-958a-8a56bb746a8d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:50.059999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c098463-cd4a-49fc-958a-8a56bb746a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:50.060847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c098463-cd4a-49fc-958a-8a56bb746a8d None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=e967e5d9815e426bbf271a1bee6f03e1) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:50.061159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c098463-cd4a-49fc-958a-8a56bb746a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:50.062672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c098463-cd4a-49fc-958a-8a56bb746a8d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:50.072865 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 139/294] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:49 2022] GET /identity/v3/role_assignments?user.id=e967e5d9815e426bbf271a1bee6f03e1&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:50.652787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9208-664d-45b0-be21-677213c0b2da None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:50.653342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9208-664d-45b0-be21-677213c0b2da None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:50.653342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e9208-664d-45b0-be21-677213c0b2da None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:50.655021 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 156/295] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:50.660786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07162f1-eab1-4c3f-984f-7b2017ebc246 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:50.661205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07162f1-eab1-4c3f-984f-7b2017ebc246 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:50.661394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07162f1-eab1-4c3f-984f-7b2017ebc246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:50.683434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a07162f1-eab1-4c3f-984f-7b2017ebc246 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:50.684634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a07162f1-eab1-4c3f-984f-7b2017ebc246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:50.767984 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 140/296] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:51.113151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e9208-664d-45b0-be21-677213c0b2da None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:51.220432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e9208-664d-45b0-be21-677213c0b2da None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:51.256472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f98503-c973-4a38-8df2-ba1a8c47fb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:51.257359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f98503-c973-4a38-8df2-ba1a8c47fb40 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.257935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f98503-c973-4a38-8df2-ba1a8c47fb40 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.258377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f98503-c973-4a38-8df2-ba1a8c47fb40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.269907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f98503-c973-4a38-8df2-ba1a8c47fb40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d6f98503-c973-4a38-8df2-ba1a8c47fb40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:51.275040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:51.276152 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:51.277165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:51.279538 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 157/297] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:51.285939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07162f1-eab1-4c3f-984f-7b2017ebc246 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:51.315131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6458d-9912-4535-b33a-bbd022f64cac None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.315650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6458d-9912-4535-b33a-bbd022f64cac None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.316064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6458d-9912-4535-b33a-bbd022f64cac None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.317954 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 158/298] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:51.323449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473651-5bf5-49a4-8f09-996b1e26ed31 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.323769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473651-5bf5-49a4-8f09-996b1e26ed31 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.324122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473651-5bf5-49a4-8f09-996b1e26ed31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.338231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:51.343322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fa473651-5bf5-49a4-8f09-996b1e26ed31 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:51.345050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fa473651-5bf5-49a4-8f09-996b1e26ed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:51.377903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:51.409310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07162f1-eab1-4c3f-984f-7b2017ebc246 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:51.424257 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 159/299] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:51.427132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9a0f6-bbf4-4cdf-b2a2-e81ab81bceaf None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.427792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9a0f6-bbf4-4cdf-b2a2-e81ab81bceaf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.428152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9a0f6-bbf4-4cdf-b2a2-e81ab81bceaf None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.433824 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 160/300] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:51.433824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca144-1afc-4c63-a701-1e79ce5098d4 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.433824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca144-1afc-4c63-a701-1e79ce5098d4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.433824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ca144-1afc-4c63-a701-1e79ce5098d4 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.433824 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 161/301] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:51.435875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40dc91-0757-4234-9fb3-c7a723b6069e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.436274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40dc91-0757-4234-9fb3-c7a723b6069e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.436843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40dc91-0757-4234-9fb3-c7a723b6069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.453129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66a56-edba-43d3-b790-8978e9f519be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:51.453817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66a56-edba-43d3-b790-8978e9f519be None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.454094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66a56-edba-43d3-b790-8978e9f519be None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.454362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66a56-edba-43d3-b790-8978e9f519be None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.455028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66a56-edba-43d3-b790-8978e9f519be None admin] RBAC: Adding query filter params (name=reader) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:51.455325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66a56-edba-43d3-b790-8978e9f519be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:51.456711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66a56-edba-43d3-b790-8978e9f519be None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:51.457124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cc40dc91-0757-4234-9fb3-c7a723b6069e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:51.458431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cc40dc91-0757-4234-9fb3-c7a723b6069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:51.469687 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 141/302] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:51.469687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e1fb-0da1-4d98-ba00-1192b497f56b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.469687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e1fb-0da1-4d98-ba00-1192b497f56b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.469687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78e1fb-0da1-4d98-ba00-1192b497f56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.484607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ad78e1fb-0da1-4d98-ba00-1192b497f56b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:51.485806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ad78e1fb-0da1-4d98-ba00-1192b497f56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:51.541491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 162/303] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:51.541491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40dc91-0757-4234-9fb3-c7a723b6069e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:51.568034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 142/304] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:51.635733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:51.640152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40dc91-0757-4234-9fb3-c7a723b6069e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:51.680451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:51.681175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.681416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.681728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] PATH_INFO: `/v3/users/4996e526602e44c58c2bab9a329950cf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.682851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] RBAC: Authorizing `identity:get_user(user_id=4996e526602e44c58c2bab9a329950cf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:51.684321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:51.686348 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 163/305] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:51 2022] GET /identity/v3/users/4996e526602e44c58c2bab9a329950cf => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:51.692988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78e1fb-0da1-4d98-ba00-1192b497f56b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:51.725557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:51.775129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:51.797581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78e1fb-0da1-4d98-ba00-1192b497f56b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:51.832340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:51.833340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.833864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.834299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.883105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842b3f4-500c-4a72-8b50-6a097b704616 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:51.910518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:51.911955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:51.918773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:51.919457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:51.919717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:51.919979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:51.921072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:51.922545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:51.925623 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 164/306] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:33:51.925623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:51.944711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:51.960351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:52.019023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 143/307] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:52.019965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.031898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd453b14-491b-4e9d-b6b9-18ea8a524ebb None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.066173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.066800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.067077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.067349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.077251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:52.082548 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:52.083557 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:52.084361 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 165/308] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:52.086293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.121714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbf7d5-a922-4244-b2fb-d20744168bf6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.151305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:52.159249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.160306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.160845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.161212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.176603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:52.183374 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:52.184373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:52.185211 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 144/309] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:52.185922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.194758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:52.203308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168508c8-4189-47eb-9731-4fea49e74b5b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.237080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.237720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.237997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.238257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] PATH_INFO: `/v3/groups` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.238899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.240318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.256283 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 166/310] 10.222.0.26 () {60 vars in 1134 bytes} [Sat Sep 17 20:33:52 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:52.257109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.303641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403df9e-780c-4fc1-8af2-17c3c027f938 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.342010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.342665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.342943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.343217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.344397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.345917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.347655 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 145/311] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:52.349699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.384964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522e48c-e9e9-4445-bdcc-d27eb24be541 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.418245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:52.424601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.425235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.425790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.426062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.427289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:52.427289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.428538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.438280 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 167/312] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:52.439368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.459471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:52.463974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d083d0-c6a6-487b-8de0-b38d999e4291 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.502310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.503015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.503282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.503560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.504285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:52.504583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.506333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.517515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 146/313] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:52.517515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.555181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bfb8f-c2ac-4515-8fc9-35d1b7f512f4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.590743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.591394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.591672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.591936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.592668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] RBAC: Adding query filter params (role.id=e6b825f310f149b38baa30e8b96142fb, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04, user.id=4996e526602e44c58c2bab9a329950cf) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:52.592975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.594506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.600536 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 168/314] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/role_assignments?user.id=4996e526602e44c58c2bab9a329950cf&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=e6b825f310f149b38baa30e8b96142fb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:52.601338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.634144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3e91a-11e6-4579-915e-8504778d933e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.661480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:52.679877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.680503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.680783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.681094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] PATH_INFO: `/v3/users/d8cce05165474e25b5eee01d2348d17c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.700537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:52.704807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=d8cce05165474e25b5eee01d2348d17c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.706460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.713498 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 147/315] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/users/d8cce05165474e25b5eee01d2348d17c => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:52.717575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.737591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912af8e3-4ef2-49cd-b9cd-daac03c29d2c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.790716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.791892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.792345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.793383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] PATH_INFO: `/v3/users/13b098f5f3cf46aebe25a5fa5ffe8225` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.795190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] RBAC: Authorizing `identity:get_user(user_id=13b098f5f3cf46aebe25a5fa5ffe8225)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.797806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.801929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 169/316] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/users/13b098f5f3cf46aebe25a5fa5ffe8225 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:52.810034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:52.852366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949397e-809b-4870-9f07-0b6b24c84fa2 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.898211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae700-8484-4d88-80e0-3e21964f303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.899247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae700-8484-4d88-80e0-3e21964f303c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.899615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae700-8484-4d88-80e0-3e21964f303c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.899889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae700-8484-4d88-80e0-3e21964f303c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.900634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae700-8484-4d88-80e0-3e21964f303c None admin] RBAC: Adding query filter params (role.id=e6b825f310f149b38baa30e8b96142fb, user.id=d8cce05165474e25b5eee01d2348d17c, scope.system=all) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:52.900945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae700-8484-4d88-80e0-3e21964f303c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.902477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae700-8484-4d88-80e0-3e21964f303c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:52.909755 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 148/317] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 17 20:33:52 2022] GET /identity/v3/role_assignments?user.id=d8cce05165474e25b5eee01d2348d17c&scope.system=all&role.id=e6b825f310f149b38baa30e8b96142fb => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:52.947892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403bc09-f9fe-474b-9e7e-04d207ce4171 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:52.986504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478a78a-72c8-4142-bfa0-403054d79533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:52.987187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478a78a-72c8-4142-bfa0-403054d79533 None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:52.987461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478a78a-72c8-4142-bfa0-403054d79533 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:52.987727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478a78a-72c8-4142-bfa0-403054d79533 None admin] PATH_INFO: `/v3/system/users/13b098f5f3cf46aebe25a5fa5ffe8225/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:52.989252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478a78a-72c8-4142-bfa0-403054d79533 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=13b098f5f3cf46aebe25a5fa5ffe8225, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:52.990749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478a78a-72c8-4142-bfa0-403054d79533 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:53.003411 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 170/318] 10.222.0.26 () {58 vars in 1334 bytes} [Sat Sep 17 20:33:52 2022] PUT /identity/v3/system/users/13b098f5f3cf46aebe25a5fa5ffe8225/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:33:54.826451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c796f3-d524-4481-8abe-1b11f511e635 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:54.827004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c796f3-d524-4481-8abe-1b11f511e635 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:54.827004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c796f3-d524-4481-8abe-1b11f511e635 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:54.828335 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 149/319] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:54.834206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d1068-567b-4100-abe1-ce4398eb1921 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:54.834701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d1068-567b-4100-abe1-ce4398eb1921 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:54.834784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d1068-567b-4100-abe1-ce4398eb1921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:54.855786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a03d1068-567b-4100-abe1-ce4398eb1921 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:54.857037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a03d1068-567b-4100-abe1-ce4398eb1921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:54.865296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:54.930580 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 171/320] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:55.255874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c796f3-d524-4481-8abe-1b11f511e635 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:55.307800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:55.367757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c796f3-d524-4481-8abe-1b11f511e635 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:55.407252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99f4f7-c1f7-44f2-87de-cbb897859b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:55.407901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f4f7-c1f7-44f2-87de-cbb897859b02 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:55.408175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f4f7-c1f7-44f2-87de-cbb897859b02 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:55.408443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f4f7-c1f7-44f2-87de-cbb897859b02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:55.409634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99f4f7-c1f7-44f2-87de-cbb897859b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:55.411215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99f4f7-c1f7-44f2-87de-cbb897859b02 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:55.413119 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 150/321] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:33:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:55.419052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d1068-567b-4100-abe1-ce4398eb1921 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:55.467904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0052be-e6c3-4c43-ac31-aca50a53d991 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:55.468261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0052be-e6c3-4c43-ac31-aca50a53d991 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:55.468445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0052be-e6c3-4c43-ac31-aca50a53d991 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:55.470011 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 151/322] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:55.477084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25b2bb-e41d-4d8c-9f63-48aa7fea75f6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:55.477357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25b2bb-e41d-4d8c-9f63-48aa7fea75f6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:55.477654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25b2bb-e41d-4d8c-9f63-48aa7fea75f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:55.497414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fe25b2bb-e41d-4d8c-9f63-48aa7fea75f6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:55.498713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe25b2bb-e41d-4d8c-9f63-48aa7fea75f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:55.539788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d1068-567b-4100-abe1-ce4398eb1921 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:55.571786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 152/323] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:55.580537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb932f55-d890-40c2-b9f4-2633e9fa327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:55.581219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb932f55-d890-40c2-b9f4-2633e9fa327c None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:55.581582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb932f55-d890-40c2-b9f4-2633e9fa327c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:55.581912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb932f55-d890-40c2-b9f4-2633e9fa327c None admin] PATH_INFO: `/v3/groups` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:55.582612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb932f55-d890-40c2-b9f4-2633e9fa327c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:55.584329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb932f55-d890-40c2-b9f4-2633e9fa327c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:55.609782 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 172/324] 10.222.0.26 () {60 vars in 1134 bytes} [Sat Sep 17 20:33:55 2022] POST /identity/v3/groups => generated 226 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:33:55.892627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25b2bb-e41d-4d8c-9f63-48aa7fea75f6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:55.940379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:55.987245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903439c-9827-42c3-a453-992e57a19f70 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:55.987775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903439c-9827-42c3-a453-992e57a19f70 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:55.987983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903439c-9827-42c3-a453-992e57a19f70 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:55.989899 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 173/325] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:55.995585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b518c1-757f-4626-896e-649cca01c698 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:55.995886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b518c1-757f-4626-896e-649cca01c698 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:55.996473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b518c1-757f-4626-896e-649cca01c698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:56.008848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25b2bb-e41d-4d8c-9f63-48aa7fea75f6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:56.020015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:56.030169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-92b518c1-757f-4626-896e-649cca01c698 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:56.032139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-92b518c1-757f-4626-896e-649cca01c698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:56.050442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:56.051396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:56.051688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:56.051960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:56.063961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:56.069059 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:56.070479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:56.071695 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 153/326] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:56.080315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:56.127288 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 174/327] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:33:56.192484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964ceee-1597-4885-89e3-4799cb448a64 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:56.194223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:56.230671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:56.231292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:56.231699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:56.231974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:56.232647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:56.232945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:56.234391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:56.243719 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 154/328] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:56.249634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b518c1-757f-4626-896e-649cca01c698 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:56.355245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b518c1-757f-4626-896e-649cca01c698 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:56.388575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:56.396185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:56.397209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:56.397726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:56.398406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] PATH_INFO: `/v3/users/13b098f5f3cf46aebe25a5fa5ffe8225` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:56.399762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] RBAC: Authorizing `identity:get_user(user_id=13b098f5f3cf46aebe25a5fa5ffe8225)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:56.401431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:56.403630 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 175/329] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:56 2022] GET /identity/v3/users/13b098f5f3cf46aebe25a5fa5ffe8225 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:56.409684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:56.504813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:56.547057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2ef3-d4d0-4e29-b7d8-b9641513849c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:56.590492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:56.590902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:56.591138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:56.591408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:56.592146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] RBAC: Adding query filter params (role.id=78bf7fb4f8e947ba99f38ca1691e6393, user.id=13b098f5f3cf46aebe25a5fa5ffe8225, scope.system=all) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:56.592448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:56.594054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:56.608023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 155/330] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 17 20:33:56 2022] GET /identity/v3/role_assignments?user.id=13b098f5f3cf46aebe25a5fa5ffe8225&scope.system=all&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:56.625207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066b902-e4fb-480d-9bc6-d77f64173c28 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:56.677817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:56.678488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:56.678763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:56.679034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:56.706726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:56.712691 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:56.714006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:33:56.714822 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:56.721589 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 176/331] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:56.724201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:56.762747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:56.851873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7f07b-af88-4a8d-abe9-ef39eed97526 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:56.890447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:56.891083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:56.891359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:56.891627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:56.892302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:56.892597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:56.894037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:56.905317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 156/332] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:56.913920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:57.019551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:57.074913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2b1db-7ee4-4dab-9256-7b198ab1277d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:57.112294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a52001-e8ab-4b04-9676-9217d3b1ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:57.112931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a52001-e8ab-4b04-9676-9217d3b1ab48 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:57.113202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a52001-e8ab-4b04-9676-9217d3b1ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:57.113508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a52001-e8ab-4b04-9676-9217d3b1ab48 None admin] PATH_INFO: `/v3/users/d8cce05165474e25b5eee01d2348d17c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:57.114605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a52001-e8ab-4b04-9676-9217d3b1ab48 None admin] RBAC: Authorizing `identity:get_user(user_id=d8cce05165474e25b5eee01d2348d17c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:57.116060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a52001-e8ab-4b04-9676-9217d3b1ab48 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:57.118291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 177/333] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:33:56 2022] GET /identity/v3/users/d8cce05165474e25b5eee01d2348d17c => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:57.126614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:57.224980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:57.242494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c01d34-9d2c-4513-acf8-78423c8db2ab None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:57.279280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8687dd-363a-4cf2-a7d8-15bc2212bb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:57.279861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8687dd-363a-4cf2-a7d8-15bc2212bb54 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:57.280132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8687dd-363a-4cf2-a7d8-15bc2212bb54 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:57.280399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8687dd-363a-4cf2-a7d8-15bc2212bb54 None admin] PATH_INFO: `/v3/system/users/d8cce05165474e25b5eee01d2348d17c/roles/e6b825f310f149b38baa30e8b96142fb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:57.282209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8687dd-363a-4cf2-a7d8-15bc2212bb54 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d8cce05165474e25b5eee01d2348d17c, role_id=e6b825f310f149b38baa30e8b96142fb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:57.283603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8687dd-363a-4cf2-a7d8-15bc2212bb54 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:57.298970 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 157/334] 10.222.0.26 () {58 vars in 1334 bytes} [Sat Sep 17 20:33:57 2022] PUT /identity/v3/system/users/d8cce05165474e25b5eee01d2348d17c/roles/e6b825f310f149b38baa30e8b96142fb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:33:58.425216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1543a-531a-46eb-aa9f-fa1b9d1562d3 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:58.425811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1543a-531a-46eb-aa9f-fa1b9d1562d3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:58.425811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1543a-531a-46eb-aa9f-fa1b9d1562d3 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:58.427282 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 178/335] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:58.432381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0c3e-ab0e-4eaf-af3d-a65057f7e976 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:58.432815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0c3e-ab0e-4eaf-af3d-a65057f7e976 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:58.432908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0c3e-ab0e-4eaf-af3d-a65057f7e976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:58.459647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b10f0c3e-ab0e-4eaf-af3d-a65057f7e976 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:58.464001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b10f0c3e-ab0e-4eaf-af3d-a65057f7e976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:58.545170 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 158/336] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:58.868657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a04d1e-1663-40a3-9176-742252c4903c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:58.869043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a04d1e-1663-40a3-9176-742252c4903c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:58.869216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a04d1e-1663-40a3-9176-742252c4903c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:58.870941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 179/337] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:58.876601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ebb1b-c75d-42c8-8be5-e198065f3008 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:58.876876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ebb1b-c75d-42c8-8be5-e198065f3008 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:58.877149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ebb1b-c75d-42c8-8be5-e198065f3008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:58.902022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-483ebb1b-c75d-42c8-8be5-e198065f3008 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:58.903213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-483ebb1b-c75d-42c8-8be5-e198065f3008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:33:58.911597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a04d1e-1663-40a3-9176-742252c4903c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:59.017212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 159/338] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:33:59.046424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a04d1e-1663-40a3-9176-742252c4903c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:59.084039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:59.096916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:59.098025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:59.098496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:59.098992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:59.111478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:59.116793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:59.122250 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:33:59.123046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:33:59.125785 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 180/339] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:33:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:33:59.131452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ebb1b-c75d-42c8-8be5-e198065f3008 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:59.153839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:59.256414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ebb1b-c75d-42c8-8be5-e198065f3008 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:59.299449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:59.300006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:59.300279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:59.300586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:59.301229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:33:59.301546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:59.302972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:59.317845 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 160/340] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:33:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:59.318542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:59.463160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cae7c3-e610-4528-bf40-b0ed09177ab9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:59.502983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:59.504018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:59.504512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:59.504971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:59.563544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:59.578029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:59.580820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:59.694910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:59.699145 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 181/341] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:33:59.704731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:59.817130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:59.848937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:33:59.867274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb1b1d-a99b-4e91-8954-c42b69bf2e90 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:59.869426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c697f40-f317-49ce-b091-9185d29a8271 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:33:59.916933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a0262-d69c-408c-9940-b1bbe8b02c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:59.916933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a0262-d69c-408c-9940-b1bbe8b02c40 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:59.916933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a0262-d69c-408c-9940-b1bbe8b02c40 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:59.917299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a0262-d69c-408c-9940-b1bbe8b02c40 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:59.918229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a0262-d69c-408c-9940-b1bbe8b02c40 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:33:59.919781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a0262-d69c-408c-9940-b1bbe8b02c40 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:33:59.921358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119d5b-1d34-49a2-9829-5b090e5f63e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:33:59.922249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119d5b-1d34-49a2-9829-5b090e5f63e0 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:59.922591 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 182/342] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:33:59 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:33:59.923004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119d5b-1d34-49a2-9829-5b090e5f63e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:59.923465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119d5b-1d34-49a2-9829-5b090e5f63e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:59.923757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d30b38-92a2-4628-93d8-4fc47f36e14c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:59.924135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d30b38-92a2-4628-93d8-4fc47f36e14c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:59.924409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d30b38-92a2-4628-93d8-4fc47f36e14c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:59.929980 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 183/343] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:33:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:33:59.931587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d30b38-92a2-4628-93d8-4fc47f36e14c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:33:59.941592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119d5b-1d34-49a2-9829-5b090e5f63e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4e119d5b-1d34-49a2-9829-5b090e5f63e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:33:59.950400 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:33:59.951344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:33:59.952137 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:33:59.954391 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 161/344] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:33:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:33:59.955953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dbcbd-e0a3-449e-b639-e9eb399aa5e4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:33:59.956452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dbcbd-e0a3-449e-b639-e9eb399aa5e4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:33:59.956846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dbcbd-e0a3-449e-b639-e9eb399aa5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:33:59.979902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-881dbcbd-e0a3-449e-b639-e9eb399aa5e4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:33:59.981645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-881dbcbd-e0a3-449e-b639-e9eb399aa5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:00.067621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d30b38-92a2-4628-93d8-4fc47f36e14c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:00.082727 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 162/345] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:00.083437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881dbcbd-e0a3-449e-b639-e9eb399aa5e4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:00.130502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:00.131145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:00.131429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:00.131698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:00.143374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:00.144839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:00.149229 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 184/346] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:33:59 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:00.167813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:00.201487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:00.201690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:00.283389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881dbcbd-e0a3-449e-b639-e9eb399aa5e4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:00.352378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:00.353723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:00.354058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:00.354338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:00.355023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:00.355330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:00.356775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:00.360703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfc41-6be5-4c88-86f7-1dcd8a332656 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:00.377185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 163/347] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:00.397254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:00.426194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:00.426813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:00.430803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:00.431118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:00.431920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:00.432236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:00.437255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:00.448575 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 185/348] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:00 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:00.541945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eb98a-8dcd-4c3d-b405-880338cd285d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:00.573687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:00.589009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:00.589672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:00.589957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:00.590230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:00.674657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:00.676098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:00.743583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:00.842545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 164/349] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:34:00.848065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:00.861880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:00.882334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c19d48-432a-4efe-999a-59e3da7379f5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:00.904390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:00.922311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:00.923335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:00.923810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:00.924273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:00.936763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:00.943104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:00.944066 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:00.944903 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:00.949124 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 186/350] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:00.952635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:01.030898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5155d4-6cdb-4a8f-a01d-36fb134a75a6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:01.098551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:01.099587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.104380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.104929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.106473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:01.108276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:01.112347 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 165/351] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:00 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:01.118857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:01.154396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b09a0-73b6-4547-ad8e-88063fdc333a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:01.203669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:01.204644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.205147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.205658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.206543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:01.207052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:01.208561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:01.219643 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 187/352] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:34:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:01.229944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:01.246701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:01.264179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55892e-b077-43a1-8965-8b7247d3bad4 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:01.287234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:01.336083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:01.337021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.337641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.338162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.342457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:01.344443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:01.346396 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 166/353] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:01 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:01.352114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:01.444183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ab5b-5b1d-4c56-919d-1d4c164d64af None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:01.503803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:01.504362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.504646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.504918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.600251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:01.603313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:01.610175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:01.616687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e2e90-59c1-4c45-b5f3-23e101604aaf None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:01.680164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169508bb-75c8-4f56-9e43-5dc4528b3949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:01.681230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169508bb-75c8-4f56-9e43-5dc4528b3949 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.681761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169508bb-75c8-4f56-9e43-5dc4528b3949 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.682303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169508bb-75c8-4f56-9e43-5dc4528b3949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.683290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169508bb-75c8-4f56-9e43-5dc4528b3949 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:01.683808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169508bb-75c8-4f56-9e43-5dc4528b3949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:01.685650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169508bb-75c8-4f56-9e43-5dc4528b3949 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:01.692276 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 167/354] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:01 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=1b7585de1b6442509e3788682caf8774 => generated 260 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:01.699354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497261a1-5617-466e-9221-b25824f902e0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.699976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497261a1-5617-466e-9221-b25824f902e0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.700544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497261a1-5617-466e-9221-b25824f902e0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.702830 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 168/355] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:01.723444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2ea6-68f6-4314-9cc2-74bf745f4682 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.724068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2ea6-68f6-4314-9cc2-74bf745f4682 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.724173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 188/356] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:01.724660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2ea6-68f6-4314-9cc2-74bf745f4682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.733913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:01.758742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-464a2ea6-68f6-4314-9cc2-74bf745f4682 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:01.759912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-464a2ea6-68f6-4314-9cc2-74bf745f4682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:01.784027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:01.845728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a60d78-c62c-4052-acb7-5928cdbb7018 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:01.850809 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 169/357] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:01.852217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851cde2-8d2c-401a-bd30-ee8a137ac9b5 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.852508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851cde2-8d2c-401a-bd30-ee8a137ac9b5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.852786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851cde2-8d2c-401a-bd30-ee8a137ac9b5 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.854489 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 170/358] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:01.855653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab27e2-16e1-4259-94ae-8f34a3df9980 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.856016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab27e2-16e1-4259-94ae-8f34a3df9980 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.856343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab27e2-16e1-4259-94ae-8f34a3df9980 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.857896 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 171/359] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:01.858794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:01.860367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4763ad-4958-4135-8b1a-90269b72007d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.860709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4763ad-4958-4135-8b1a-90269b72007d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.860995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4763ad-4958-4135-8b1a-90269b72007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.884685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7b4763ad-4958-4135-8b1a-90269b72007d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:01.886200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7b4763ad-4958-4135-8b1a-90269b72007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:01.890828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121f778-0190-4205-9fed-8147ffb8228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:01.891485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f778-0190-4205-9fed-8147ffb8228f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.891760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f778-0190-4205-9fed-8147ffb8228f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.892030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f778-0190-4205-9fed-8147ffb8228f None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.893203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121f778-0190-4205-9fed-8147ffb8228f None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:01.894741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121f778-0190-4205-9fed-8147ffb8228f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:01.896873 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 189/360] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:01 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:01.898164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca7e5-50c3-4016-89c3-afb27ab570d4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:01.898510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca7e5-50c3-4016-89c3-afb27ab570d4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:01.898787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca7e5-50c3-4016-89c3-afb27ab570d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:01.919310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-965ca7e5-50c3-4016-89c3-afb27ab570d4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:01.920562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-965ca7e5-50c3-4016-89c3-afb27ab570d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:01.978314 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 172/361] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:01.979629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4763ad-4958-4135-8b1a-90269b72007d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:01.997570 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 190/362] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:02.092908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:02.116454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4763ad-4958-4135-8b1a-90269b72007d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:02.160762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:02.161432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:02.161717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:02.161990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:02.163120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:02.164588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:02.166519 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 173/363] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:01 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:02.172571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca7e5-50c3-4016-89c3-afb27ab570d4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:02.313136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca7e5-50c3-4016-89c3-afb27ab570d4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:02.315525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:02.347846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:02.348380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:02.348658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:02.348930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:02.349689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:02.350001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:02.351447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:02.360396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 191/364] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:02 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:02.429768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:02.446138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:02.599426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da562f-4d81-451a-aedf-2fe825c26261 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:02.630897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac8945-42c9-4c10-82a2-c1c8adde87a0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:02.658190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:02.659296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:02.659788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:02.660302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:02.683800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:02.684567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:02.684882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:02.685152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:02.685430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:02.691595 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:02.692454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:02.693271 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:02.695978 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 174/365] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:02.697106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:02.703443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:02.708687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:02.709683 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:34:02.710606 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:02.713573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 192/366] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:02.714553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:02.728704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:02.749068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:02.833806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674be46-d819-46da-ac60-aa76a6f56f02 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:02.838814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936789f-daa3-4e06-9bcb-66dec01fe02b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:02.879621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:02.880246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:02.880503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:02.880781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:02.892644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:02.897733 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:02.898609 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:02.899476 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 175/367] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:34:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:02.900722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:02.901976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:02.903154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:02.903658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:02.904105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:02.904972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:02.905534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:02.907369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:02.927848 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 193/368] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:02.928860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:02.986223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:03.014689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb5ef7-c715-4501-96c2-5a7c109a517f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:03.055229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:03.056213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:03.056631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:03.057037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:03.058129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:03.058597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:03.060577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:03.069705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 176/369] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:34:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:03.071091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:03.180379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52668e1a-60a5-423f-aed6-0b5ac0448fc5 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:03.196102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:03.218899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:03.219781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:03.220555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:03.221149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:03.225671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:03.227962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5f11a-1b18-426c-9177-6a9a702d8470 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:03.296568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:03.298613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:03.354193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:03.354996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:03.355340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:03.355695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:03.356472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:03.356837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:03.358329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:03.383386 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 194/370] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:03.384373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:03.410779 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 177/371] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:03.412714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:03.448039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:03.527329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d9fb4-cb98-42f2-ad7b-42e82b5b3ff8 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:03.564667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:03.565396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:03.565755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:03.566101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:03.585269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d11f8-b4a0-43a7-a4be-df43320edf8e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:03.642710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:03.643469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:03.643809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:03.644137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] PATH_INFO: `/v3/users/d8cce05165474e25b5eee01d2348d17c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:03.645360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] RBAC: Authorizing `identity:get_user(user_id=d8cce05165474e25b5eee01d2348d17c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:03.646907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:03.647293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:03.648851 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 195/372] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:34:03 2022] GET /identity/v3/users/d8cce05165474e25b5eee01d2348d17c => generated 293 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:03.648966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:03.653640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:03.684541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:03.697869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:03.773877 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 178/373] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:03.775098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:03.810709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a519-ba4a-4f22-a23c-92964f0ec687 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:03.864982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:03.865662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:03.865938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:03.866201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] PATH_INFO: `/v3/projects/c8633d9055db49bfa42074c66a0581c2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:03.867333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] RBAC: Authorizing `identity:get_project(project_id=c8633d9055db49bfa42074c66a0581c2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:03.868777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:03.870447 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 196/374] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:03 2022] GET /identity/v3/projects/c8633d9055db49bfa42074c66a0581c2 => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:03.871226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:03.889201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f2beb-2170-45d8-8fe6-13a7a59108e9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:03.925906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:03.933075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:03.934841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:03.935228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:03.935595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:03.936434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] RBAC: Adding query filter params (role.id=e6b825f310f149b38baa30e8b96142fb, user.id=d8cce05165474e25b5eee01d2348d17c, scope.system=all) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:03.936840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:03.938475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:03.954704 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 179/375] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 17 20:34:03 2022] GET /identity/v3/role_assignments?user.id=d8cce05165474e25b5eee01d2348d17c&scope.system=all&role.id=e6b825f310f149b38baa30e8b96142fb => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:03.955825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:04.061100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:04.074145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505b933-c750-4661-88c1-b1e7a9639b62 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:04.083477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74c4f9-ac4b-4618-991e-f119c783e828 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:04.117226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:04.118010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:04.118293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:04.118596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] PATH_INFO: `/v3/groups/1f00f125b9dc46198b77c39e0e6ad3fe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:04.126013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] RBAC: Authorizing `identity:get_group(group_id=1f00f125b9dc46198b77c39e0e6ad3fe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:04.127660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:04.129818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 180/376] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:03 2022] GET /identity/v3/groups/1f00f125b9dc46198b77c39e0e6ad3fe => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:04.139018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:04.152014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:04.152676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:04.152945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:04.153215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:04.154653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:04.156147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:04.166857 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 197/377] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:03 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:04.172832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:04.213912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:04.262024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd7639-56b0-4b71-a1ce-b3f4df10bbfb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:04.301242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e249b1-fa95-4830-91dc-48540b7ddc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:04.301930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e249b1-fa95-4830-91dc-48540b7ddc54 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:04.302205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e249b1-fa95-4830-91dc-48540b7ddc54 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:04.302474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e249b1-fa95-4830-91dc-48540b7ddc54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:04.303325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e249b1-fa95-4830-91dc-48540b7ddc54 None admin] RBAC: Adding query filter params (group.id=1f00f125b9dc46198b77c39e0e6ad3fe, role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=c8633d9055db49bfa42074c66a0581c2) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:04.303632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e249b1-fa95-4830-91dc-48540b7ddc54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:04.305241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e249b1-fa95-4830-91dc-48540b7ddc54 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:04.312013 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 181/378] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:04 2022] GET /identity/v3/role_assignments?group.id=1f00f125b9dc46198b77c39e0e6ad3fe&scope.project.id=c8633d9055db49bfa42074c66a0581c2&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:04.313283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d924b-d3db-4a0a-a931-39444b01edc9 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:04.313604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d924b-d3db-4a0a-a931-39444b01edc9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:04.313891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d924b-d3db-4a0a-a931-39444b01edc9 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:04.315387 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 182/379] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:04.321853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de927993-783d-4495-b9f4-30907ed659fe None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:04.322184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de927993-783d-4495-b9f4-30907ed659fe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:04.322392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de927993-783d-4495-b9f4-30907ed659fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:04.354792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-de927993-783d-4495-b9f4-30907ed659fe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:04.355696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-de927993-783d-4495-b9f4-30907ed659fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:04.424711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed600094-0b1b-449d-a938-938ed1d60462 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:04.433168 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 183/380] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:04.486631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:04.487272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:04.487543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:04.487807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:04.494311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:04.495819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:04.497940 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 198/381] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:04 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:04.504039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de927993-783d-4495-b9f4-30907ed659fe None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:04.601087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:04.661276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de927993-783d-4495-b9f4-30907ed659fe None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:04.707271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:04.707942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:04.708216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:04.708529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:04.709324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:04.709673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:04.711124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:04.717118 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 184/382] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:04 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=1b7585de1b6442509e3788682caf8774 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:04.951485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:04.989961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:05.092957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834c8d6-7ca5-4b13-88ba-40100fbce66d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:05.145708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:05.146348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.146621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.146890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.158229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:05.163429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:05.164326 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:05.165373 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 199/383] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:05.171525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:05.290374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e09b60-a6bd-4f41-803e-a46ffa7043e6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:05.303766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:05.330489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:05.331101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.331374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.331641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.332301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:05.332599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:05.334069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:05.342999 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 185/384] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:34:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:05.348625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:05.441586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:05.459401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8bda9-dff8-471d-892d-c9d5e6e383a3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:05.496016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8b183-8cbc-4b96-a604-d4b342e23e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:05.496648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8b183-8cbc-4b96-a604-d4b342e23e87 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.496925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8b183-8cbc-4b96-a604-d4b342e23e87 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.497264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8b183-8cbc-4b96-a604-d4b342e23e87 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.498415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8b183-8cbc-4b96-a604-d4b342e23e87 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:05.499918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8b183-8cbc-4b96-a604-d4b342e23e87 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:05.501918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 200/385] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:05 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:05.507656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:05.560158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaac524-92ba-483f-aba3-fb14a6ddafec None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.560331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaac524-92ba-483f-aba3-fb14a6ddafec None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.560606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaac524-92ba-483f-aba3-fb14a6ddafec None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.563465 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 201/386] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:05.568585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7de83-6e37-485a-b609-85bddbfc60e6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.568859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7de83-6e37-485a-b609-85bddbfc60e6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.569126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7de83-6e37-485a-b609-85bddbfc60e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.595991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0be7de83-6e37-485a-b609-85bddbfc60e6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:05.597178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0be7de83-6e37-485a-b609-85bddbfc60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:05.658043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1f4c6-d1f7-494a-9252-0483d9cfd7ef None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:05.681178 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 202/387] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:05.692294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfe28f-becf-472d-92c7-72f5c6ef86a3 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.692294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfe28f-becf-472d-92c7-72f5c6ef86a3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.692294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfe28f-becf-472d-92c7-72f5c6ef86a3 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.692294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 203/388] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:05.700416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb9bcd-6111-40f8-b217-534d3949c12d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.700694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb9bcd-6111-40f8-b217-534d3949c12d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.700962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb9bcd-6111-40f8-b217-534d3949c12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.704354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:05.705084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.705368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.705725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.707690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:05.709277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:05.711422 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 186/389] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:05 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:05.716778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:05.738559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-98eb9bcd-6111-40f8-b217-534d3949c12d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:05.739735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98eb9bcd-6111-40f8-b217-534d3949c12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:05.769871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:05.825653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07abab4-1ea7-4be8-976f-69a92ecb1377 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:05.851975 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 204/390] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:05.875065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:05.875748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:05.876030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:05.876305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/groups/e3c5c712634544b1a2553113377ce014/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:05.882805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] RBAC: Authorizing `identity:create_grant(project_id=54c0489c5be04e868df9a06af5b5ed5b, group_id=e3c5c712634544b1a2553113377ce014, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:05.884265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:05.918514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 187/391] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 17 20:34:05 2022] PUT /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/groups/e3c5c712634544b1a2553113377ce014/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:34:06.122562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb9bcd-6111-40f8-b217-534d3949c12d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.138522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:06.227672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb9bcd-6111-40f8-b217-534d3949c12d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:06.244760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.272273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:06.272927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:06.273203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:06.273491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:06.289507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:06.294279 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:06.295212 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:06.296018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 205/392] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:06.301552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.381593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac9251-0ef7-45b3-b660-8b55fe847de0 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:06.384095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:06.393853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:06.431145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56fe92-8afb-4a03-b7bf-56d01238d391 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:06.434021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:06.434752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:06.435030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:06.435300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:06.449301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:06.457304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:06.458270 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:06.459064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 188/393] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:06.468558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.482376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:06.483289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:06.483800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:06.484232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:06.485169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:06.485679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:06.487439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:06.498966 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 206/394] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:06.504093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.621112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc722d-0bf8-4d00-a72f-1b3b5b99d6fb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:06.653071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95056aab-54ab-42af-99ed-cedbd0dc0f31 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:06.663554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:06.664663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:06.665131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:06.665700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:06.671700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] RBAC: Adding query filter params (name=member) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:06.672375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:06.674456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:06.697982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 189/395] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:06.699376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:06.700381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:06.700920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:06.701376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:06.706194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:06.707913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:06.710063 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 207/396] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:06.712556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.717561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.727421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:06.751755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:06.841095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfdb55-db3c-47da-b164-a1dc1218cb90 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:06.883558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:06.884575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:06.885043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:06.885563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:06.886892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:06.889157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:06.891761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 190/397] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:06.898151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:06.948305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17411298-f67a-4b85-b5c1-c2aca33f3896 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:07.022097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:07.023206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:07.023674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:07.024131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:07.025499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:07.027133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:07.029707 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 208/398] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:07.037041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:07.050211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:07.064237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abceb6b-d3e6-48a9-aa3b-cc46c66c9ef0 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:07.144660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:07.155209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:07.156730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:07.157215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:07.157705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:07.162200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:07.164092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:07.166349 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 191/399] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:06 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:07.172223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:07.185696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9166a47-45a8-456c-a621-fa43e0a032cc None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:07.228934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c005280-b544-4fb9-a78e-e6273be02003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:07.229970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c005280-b544-4fb9-a78e-e6273be02003 None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:07.230225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c005280-b544-4fb9-a78e-e6273be02003 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:07.230494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c005280-b544-4fb9-a78e-e6273be02003 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/groups/e3c5c712634544b1a2553113377ce014/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:07.233232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c005280-b544-4fb9-a78e-e6273be02003 None admin] RBAC: Authorizing `identity:create_grant(project_id=54c0489c5be04e868df9a06af5b5ed5b, group_id=e3c5c712634544b1a2553113377ce014, role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:07.235313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c005280-b544-4fb9-a78e-e6273be02003 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:07.263319 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 209/400] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 17 20:34:07 2022] PUT /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b/groups/e3c5c712634544b1a2553113377ce014/roles/1b7585de1b6442509e3788682caf8774 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:07.368702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda910a-4a7d-433f-824c-ee8bb72f344d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:07.415258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c1116-6368-4881-a246-4ed7ad253295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:07.415943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c1116-6368-4881-a246-4ed7ad253295 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:07.416224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c1116-6368-4881-a246-4ed7ad253295 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:07.416499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c1116-6368-4881-a246-4ed7ad253295 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/groups/e3c5c712634544b1a2553113377ce014/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:07.418654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c1116-6368-4881-a246-4ed7ad253295 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb544378edc4844a8bfe4c58c6b8b04, group_id=e3c5c712634544b1a2553113377ce014, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:07.420308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c1116-6368-4881-a246-4ed7ad253295 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:07.438344 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 192/401] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 17 20:34:07 2022] PUT /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/groups/e3c5c712634544b1a2553113377ce014/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:34:07.787112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f686aea-fe4c-48d5-9dd5-fe25cbd539d0 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:07.787767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f686aea-fe4c-48d5-9dd5-fe25cbd539d0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:07.788233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f686aea-fe4c-48d5-9dd5-fe25cbd539d0 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:07.790029 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 210/402] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:07.795785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e339cf8-3692-478b-961e-02570e97fb7c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:07.796116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e339cf8-3692-478b-961e-02570e97fb7c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:07.796334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e339cf8-3692-478b-961e-02570e97fb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:07.818142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6e339cf8-3692-478b-961e-02570e97fb7c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:07.819353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6e339cf8-3692-478b-961e-02570e97fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:07.835800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:07.916434 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 193/403] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:08.069402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59e2fc-9901-438f-8dee-42328b5d25d3 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:08.069756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59e2fc-9901-438f-8dee-42328b5d25d3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:08.069964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59e2fc-9901-438f-8dee-42328b5d25d3 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:08.071568 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 211/404] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:08.077132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6557904-ef3b-4b44-af7c-74dfd888de62 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:08.077423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6557904-ef3b-4b44-af7c-74dfd888de62 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:08.077723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6557904-ef3b-4b44-af7c-74dfd888de62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:08.102489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b6557904-ef3b-4b44-af7c-74dfd888de62 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:08.103703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b6557904-ef3b-4b44-af7c-74dfd888de62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:08.184423 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 194/405] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:08.396349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59e2fc-9901-438f-8dee-42328b5d25d3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:08.513786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6557904-ef3b-4b44-af7c-74dfd888de62 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:08.524528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59e2fc-9901-438f-8dee-42328b5d25d3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:08.560742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:08.561764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:08.562352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:08.562942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:08.573880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:08.579414 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:08.580299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:08.581084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:08.583691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 212/406] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:08.583888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:08.589970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:08.592066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:08.613180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6557904-ef3b-4b44-af7c-74dfd888de62 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:08.666271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:08.666843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:08.667126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:08.667396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:08.701615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:08.716363 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:08.717330 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:08.718164 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:08.721699 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 195/407] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:34:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:08.727500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:08.734471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741288e-11ff-4569-ae0c-2f94b6656ff2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:08.788973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:08.790052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:08.790641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:08.791150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:08.792085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:08.792588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:08.794377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:08.804247 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 213/408] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:08.809342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:08.909972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:08.963487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd37f6-8a10-4fd1-9870-eb590b35fe12 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:08.968103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538abc24-4a9d-4731-920b-99ed4c02e72d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:09.000146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:09.001144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.001651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.002116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] PATH_INFO: `/v3/groups/1f00f125b9dc46198b77c39e0e6ad3fe` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.003537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] RBAC: Authorizing `identity:get_group(group_id=1f00f125b9dc46198b77c39e0e6ad3fe)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:09.005198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:09.007398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 214/409] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:08 2022] GET /identity/v3/groups/1f00f125b9dc46198b77c39e0e6ad3fe => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:09.012590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:09.083170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:09.083411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.083634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.084205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.084886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:09.085211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:09.086632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:09.118756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe60dc-0d55-4430-980d-13d261888e50 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:09.119716 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 196/410] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:09.130419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:09.139713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:09.153237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:09.162283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:09.163172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.164462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.165016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] PATH_INFO: `/v3/projects/c8633d9055db49bfa42074c66a0581c2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.167768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] RBAC: Authorizing `identity:get_project(project_id=c8633d9055db49bfa42074c66a0581c2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:09.171881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:09.176789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 215/411] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:09 2022] GET /identity/v3/projects/c8633d9055db49bfa42074c66a0581c2 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:09.182417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:09.342777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbe9fc-b5de-48c6-ac20-5d96a944b691 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:09.379528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3854563-ce27-4ed5-a389-39d2f39eaed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:09.380281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c29b9-aa6e-43e6-83d6-48901eff4f8a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:09.380786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3854563-ce27-4ed5-a389-39d2f39eaed3 None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.381237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3854563-ce27-4ed5-a389-39d2f39eaed3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.381835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3854563-ce27-4ed5-a389-39d2f39eaed3 None admin] PATH_INFO: `/v3/projects/c8633d9055db49bfa42074c66a0581c2/groups/1f00f125b9dc46198b77c39e0e6ad3fe/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.384025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3854563-ce27-4ed5-a389-39d2f39eaed3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8633d9055db49bfa42074c66a0581c2, group_id=1f00f125b9dc46198b77c39e0e6ad3fe, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:09.385777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3854563-ce27-4ed5-a389-39d2f39eaed3 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:09.406939 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 216/412] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 17 20:34:09 2022] PUT /identity/v3/projects/c8633d9055db49bfa42074c66a0581c2/groups/1f00f125b9dc46198b77c39e0e6ad3fe/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:34:09.409071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93063-ae65-4a35-baca-9c9269d41ffd None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.409576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93063-ae65-4a35-baca-9c9269d41ffd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.410336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93063-ae65-4a35-baca-9c9269d41ffd None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.413257 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 217/413] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:09.427038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68cfd0-a44f-4ae6-b343-f4bf6f914ea5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.427672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68cfd0-a44f-4ae6-b343-f4bf6f914ea5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.428139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68cfd0-a44f-4ae6-b343-f4bf6f914ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.437494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:09.437494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.437494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.437494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.438411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:09.440472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:09.442236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 197/414] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:09 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:09.451114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:09.473250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1e68cfd0-a44f-4ae6-b343-f4bf6f914ea5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:09.474702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1e68cfd0-a44f-4ae6-b343-f4bf6f914ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:09.560914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:09.599239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:09.616950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 218/415] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:09.646980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6aab-0053-40ec-a8f1-ec00f0595ed8 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:09.697986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:09.698899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.699456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.699922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.701328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:09.703153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:09.709848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 198/416] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:09 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:09.712053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68cfd0-a44f-4ae6-b343-f4bf6f914ea5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:09.867927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68cfd0-a44f-4ae6-b343-f4bf6f914ea5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:09.920012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacf28e-a7ac-43df-9ce5-ce6d27b7a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:09.920689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacf28e-a7ac-43df-9ce5-ce6d27b7a0ea None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:09.920991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacf28e-a7ac-43df-9ce5-ce6d27b7a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:09.921225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacf28e-a7ac-43df-9ce5-ce6d27b7a0ea None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/groups/e3c5c712634544b1a2553113377ce014/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:09.923566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacf28e-a7ac-43df-9ce5-ce6d27b7a0ea None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb544378edc4844a8bfe4c58c6b8b04, group_id=e3c5c712634544b1a2553113377ce014, role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:09.925020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacf28e-a7ac-43df-9ce5-ce6d27b7a0ea None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:09.944886 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 219/417] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 17 20:34:09 2022] PUT /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04/groups/e3c5c712634544b1a2553113377ce014/roles/1b7585de1b6442509e3788682caf8774 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:10.040188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:10.100637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3b565-0d16-45b3-b8c5-b41cfb701853 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.101033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3b565-0d16-45b3-b8c5-b41cfb701853 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.101221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3b565-0d16-45b3-b8c5-b41cfb701853 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.103177 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 220/418] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:10.109056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d24cf-e6e3-4d3c-a5d2-32fb44ca608f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.109437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d24cf-e6e3-4d3c-a5d2-32fb44ca608f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.109750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d24cf-e6e3-4d3c-a5d2-32fb44ca608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.130010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-757d24cf-e6e3-4d3c-a5d2-32fb44ca608f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:10.131278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-757d24cf-e6e3-4d3c-a5d2-32fb44ca608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:10.176799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:10.206835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bb377-f117-4dae-94b2-3a469214f388 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:10.247997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:10.248667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.249012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.249300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.251262 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 221/419] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:10.268546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:10.278822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:10.285189 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:10.286203 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:10.286999 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:10.289669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 199/420] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:10.296598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d24cf-e6e3-4d3c-a5d2-32fb44ca608f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:10.403816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d24cf-e6e3-4d3c-a5d2-32fb44ca608f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:10.438666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:10.440082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.440581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.441085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.442010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:10.442511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:10.444235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:10.453246 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 222/421] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:34:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:10.459402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:10.565487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7c4bc-fdbe-4520-89c6-b47b9f8f5f84 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:10.600946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f09142-a5ac-4829-807f-1f277b066a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:10.602444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f09142-a5ac-4829-807f-1f277b066a2b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.602943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f09142-a5ac-4829-807f-1f277b066a2b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.603408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f09142-a5ac-4829-807f-1f277b066a2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.671577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:10.681743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f09142-a5ac-4829-807f-1f277b066a2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:10.683582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f09142-a5ac-4829-807f-1f277b066a2b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:10.736714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:10.794271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:10.802394 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 200/422] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:10.808664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0f51f-2d82-4674-9ecd-27f73a549254 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.808950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0f51f-2d82-4674-9ecd-27f73a549254 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.809225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0f51f-2d82-4674-9ecd-27f73a549254 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.814772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 201/423] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:10.814772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0f51f-2d82-4674-9ecd-27f73a549254 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:10.886283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeebc49-0288-4e40-bf00-9c8b55755a63 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:10.922507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36558fb-97d4-4ca0-a6c9-91876d27f0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:10.923161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36558fb-97d4-4ca0-a6c9-91876d27f0c1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.923614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36558fb-97d4-4ca0-a6c9-91876d27f0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.923951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36558fb-97d4-4ca0-a6c9-91876d27f0c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.935007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36558fb-97d4-4ca0-a6c9-91876d27f0c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:10.936552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0f51f-2d82-4674-9ecd-27f73a549254 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-c36558fb-97d4-4ca0-a6c9-91876d27f0c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:10.940383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:10.941527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:10.942582 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:10.943515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:10.943713 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 223/424] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:34:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:10.945000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a7d13-bc5a-40a1-9282-30c0ca0d2e3f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.945369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a7d13-bc5a-40a1-9282-30c0ca0d2e3f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.945731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a7d13-bc5a-40a1-9282-30c0ca0d2e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.968260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-eb7a7d13-bc5a-40a1-9282-30c0ca0d2e3f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:10.969713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eb7a7d13-bc5a-40a1-9282-30c0ca0d2e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:10.982692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:10.983834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:10.984314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:10.984991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:10.986789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:10.988925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:10.991156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 202/425] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:10 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:10.992253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:11.042860 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 224/426] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:11.049583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a7d13-bc5a-40a1-9282-30c0ca0d2e3f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:11.084643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:11.172952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb2942-934e-4096-8b85-fe45dea3f6a2 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:11.232399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:11.233021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:11.233298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:11.233604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:11.234527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a7d13-bc5a-40a1-9282-30c0ca0d2e3f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:11.236246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:11.236555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:11.237989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:11.255959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 203/427] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:11.259756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:11.299711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:11.301446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:11.302116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:11.302400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:11.302956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:11.305860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:11.307807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:11.310168 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 225/428] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:11 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:11.322380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:11.440756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:11.449328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e100934-e22b-4c56-8952-bcad52fc8279 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:11.491294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:11.492157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:11.492638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:11.493038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:11.494537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:11.494840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:11.496273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:11.502545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 226/429] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:11 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:11.510553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:11.533528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6705594-23f2-47a4-a2f9-a7435434f365 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:11.629651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:11.631382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:11.634249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:11.634772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:11.758244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b2bc8-f56a-445a-8299-f793d7f47540 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:11.774500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:11.776665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:11.810667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:11.811323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:11.811598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:11.811864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:11.819281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:11.830688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:11.836951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:11.837938 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:34:11.838784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:11.841589 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 227/430] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:11.848089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:11.874746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:11.927672 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 204/431] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:11.933568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.006505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a410a89-6873-4171-a827-908061e0aa37 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:12.059984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:12.062181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.062800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.063203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.064023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:12.064453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:12.066384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:12.085818 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 228/432] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:34:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:12.090262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.103559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32d398-674e-4157-bfc4-cf4a176c4059 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:12.133174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:12.145792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:12.146938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.147404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.147964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] PATH_INFO: `/v3/projects/54c0489c5be04e868df9a06af5b5ed5b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.150041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] RBAC: Authorizing `identity:get_project(project_id=54c0489c5be04e868df9a06af5b5ed5b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:12.151903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:12.154368 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 205/433] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:11 2022] GET /identity/v3/projects/54c0489c5be04e868df9a06af5b5ed5b => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:12.160600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.186576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:12.219162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745267-b4c6-4392-9255-fa329d8e47a5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:12.264259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf911724-8ccd-4680-a69f-dfbdae0527ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:12.264802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911724-8ccd-4680-a69f-dfbdae0527ff None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.265076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911724-8ccd-4680-a69f-dfbdae0527ff None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.265344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911724-8ccd-4680-a69f-dfbdae0527ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.303599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cc507-36eb-42b7-a2c3-10cc392bf57a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:12.342341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64acf5d-d6f2-4516-b651-c032b04fe874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:12.342896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64acf5d-d6f2-4516-b651-c032b04fe874 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.343173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64acf5d-d6f2-4516-b651-c032b04fe874 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.343450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64acf5d-d6f2-4516-b651-c032b04fe874 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.344569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64acf5d-d6f2-4516-b651-c032b04fe874 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:12.346069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64acf5d-d6f2-4516-b651-c032b04fe874 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:12.347206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf911724-8ccd-4680-a69f-dfbdae0527ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:12.352488 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 206/434] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:12 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:12.352638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf911724-8ccd-4680-a69f-dfbdae0527ff None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:12.352718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd13600-5e2a-4937-9d8a-853ee4ef7b8e None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.352718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd13600-5e2a-4937-9d8a-853ee4ef7b8e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.352718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd13600-5e2a-4937-9d8a-853ee4ef7b8e None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.354691 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 207/435] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:12.355497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd13600-5e2a-4937-9d8a-853ee4ef7b8e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.420469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:12.430738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:12.458073 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 229/436] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:12.459332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b31e-7cce-4c88-a7b3-7e1f8f4b017c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.459611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b31e-7cce-4c88-a7b3-7e1f8f4b017c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.459879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20b31e-7cce-4c88-a7b3-7e1f8f4b017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.467232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd13600-5e2a-4937-9d8a-853ee4ef7b8e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:12.478199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5b20b31e-7cce-4c88-a7b3-7e1f8f4b017c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:12.479361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5b20b31e-7cce-4c88-a7b3-7e1f8f4b017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:12.503614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:12.504253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.504532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.504803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.505591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=54c0489c5be04e868df9a06af5b5ed5b) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:12.505899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:12.507405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:12.521620 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 208/437] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:12 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=54c0489c5be04e868df9a06af5b5ed5b&role.id=1b7585de1b6442509e3788682caf8774 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:12.521620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.565661 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 230/438] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:12.642447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380f0d-15d9-4332-b708-3e4851a21382 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:12.672124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:12.681685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:12.682342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.682618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.682888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.684068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:12.685637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:12.687531 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 209/439] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:12 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:12.693269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20b31e-7cce-4c88-a7b3-7e1f8f4b017c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.799511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20b31e-7cce-4c88-a7b3-7e1f8f4b017c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:12.802694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:12.835572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:12.836496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:12.836939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:12.837370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:12.838699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:12.840322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:12.842527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 231/440] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:12 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:12.848685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.905443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:12.967958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9815b9-6e50-4015-8a32-ab2caa7f21eb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.003190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7e2cb-7aef-4950-a911-aeac8d381590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.003801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e2cb-7aef-4950-a911-aeac8d381590 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.004075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e2cb-7aef-4950-a911-aeac8d381590 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.004342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e2cb-7aef-4950-a911-aeac8d381590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.005071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e2cb-7aef-4950-a911-aeac8d381590 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=78bf7fb4f8e947ba99f38ca1691e6393, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:13.005374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e2cb-7aef-4950-a911-aeac8d381590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:13.006869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e2cb-7aef-4950-a911-aeac8d381590 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:13.012939 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 210/441] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:12 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=78bf7fb4f8e947ba99f38ca1691e6393 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:13.014287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba550707-2333-42aa-8a79-65fbcf7b9f5f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.014673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba550707-2333-42aa-8a79-65fbcf7b9f5f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.014841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba550707-2333-42aa-8a79-65fbcf7b9f5f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.016055 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 211/442] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:13.022141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297af23-a532-490f-b031-3ea980ded32d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.022329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5aeae9-f9b8-431b-8632-331b6e89085b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.022444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297af23-a532-490f-b031-3ea980ded32d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.022686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297af23-a532-490f-b031-3ea980ded32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.043331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3297af23-a532-490f-b031-3ea980ded32d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:13.044650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3297af23-a532-490f-b031-3ea980ded32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:13.065254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.065834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.066110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.066373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.078280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:13.083788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:13.084840 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:13.086481 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 232/443] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:13.091700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.108711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:13.122243 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 212/444] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:13.142160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:13.222969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dee9-b8bf-4b28-8578-bc99ee7e6fee None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.259458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.260080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.260348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.260691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.261346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:13.261634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:13.263109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:13.271443 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 233/445] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:13.277191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297af23-a532-490f-b031-3ea980ded32d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.383805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297af23-a532-490f-b031-3ea980ded32d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.440530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.441420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.441871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.442273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.454942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.525131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:13.536964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:13.538522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:13.580595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:13.588475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb7284-5845-49ae-a7ec-e3ebd4fea466 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.625065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.626919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.627204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.627469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.638914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:13.648669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 213/446] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:13.649752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:13.649950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:13.650838 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:13.651654 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 234/447] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:13.657025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.756172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:13.765782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccb9f3-bb73-430b-8780-80258c20a7b9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.766111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9ce97-13e9-453f-bed7-d43d18158dfd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.799579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.800193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.800473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.800764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.801422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:13.801765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:13.803181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:13.809826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.810461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.810733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.811002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] PATH_INFO: `/v3/projects/c8633d9055db49bfa42074c66a0581c2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.811771 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 235/448] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:13.812133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] RBAC: Authorizing `identity:get_project(project_id=c8633d9055db49bfa42074c66a0581c2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:13.813682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:13.817571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 214/449] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/projects/c8633d9055db49bfa42074c66a0581c2 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:13.819041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.821062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.839814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:13.926103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f3e7-742e-4f63-bc63-e5490d93ce70 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.930002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171f1b-4ba0-4180-852e-5e98bb3b16ae None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:13.959791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.960552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.960836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.961100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] PATH_INFO: `/v3/groups/1f00f125b9dc46198b77c39e0e6ad3fe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.962244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] RBAC: Authorizing `identity:get_group(group_id=1f00f125b9dc46198b77c39e0e6ad3fe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:13.963694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:13.964012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:13.964616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:13.964892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:13.965415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:13.965578 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 215/450] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/groups/1f00f125b9dc46198b77c39e0e6ad3fe => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:13.971212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:13.998399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:14.032064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:14.033541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:14.049827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:14.069662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00b387-abe0-4461-8f59-da7119eeb1c4 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:14.104223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:14.104827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:14.105088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:14.105373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:14.106146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] RBAC: Adding query filter params (group.id=1f00f125b9dc46198b77c39e0e6ad3fe, role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=c8633d9055db49bfa42074c66a0581c2) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:14.106469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:14.107930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:14.114970 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 216/451] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/role_assignments?group.id=1f00f125b9dc46198b77c39e0e6ad3fe&scope.project.id=c8633d9055db49bfa42074c66a0581c2&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:14.131776 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 236/452] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:14.137921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:14.240662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:14.251203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b24b72-e710-4d14-af9f-721f01889af3 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:14.287805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:14.288444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:14.288718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:14.288996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] PATH_INFO: `/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:14.290181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb544378edc4844a8bfe4c58c6b8b04)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:14.291639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:14.293294 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 217/453] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:34:14 2022] GET /identity/v3/projects/3fb544378edc4844a8bfe4c58c6b8b04 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:14.299477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:14.401880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589d6d6-5961-4f8e-9bc5-7f22bd8f64c9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:14.418053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:14.437260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84de1c-a953-488c-9cb7-9bc2f586d5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:14.437909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84de1c-a953-488c-9cb7-9bc2f586d5c7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:14.438183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84de1c-a953-488c-9cb7-9bc2f586d5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:14.438451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84de1c-a953-488c-9cb7-9bc2f586d5c7 None admin] PATH_INFO: `/v3/groups/e3c5c712634544b1a2553113377ce014` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:14.439617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84de1c-a953-488c-9cb7-9bc2f586d5c7 None admin] RBAC: Authorizing `identity:get_group(group_id=e3c5c712634544b1a2553113377ce014)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:14.441061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84de1c-a953-488c-9cb7-9bc2f586d5c7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:14.442662 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 237/454] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 17 20:34:14 2022] GET /identity/v3/groups/e3c5c712634544b1a2553113377ce014 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:14.449550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:14.576856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfc359-5870-4970-869d-17a3203386a6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:14.612908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efde5c-26a9-4520-81af-95c04b024519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:14.613502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efde5c-26a9-4520-81af-95c04b024519 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:14.613813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efde5c-26a9-4520-81af-95c04b024519 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:14.614083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efde5c-26a9-4520-81af-95c04b024519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:14.614806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efde5c-26a9-4520-81af-95c04b024519 None admin] RBAC: Adding query filter params (group.id=e3c5c712634544b1a2553113377ce014, role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:14.615088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efde5c-26a9-4520-81af-95c04b024519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:14.616529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efde5c-26a9-4520-81af-95c04b024519 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:14.622891 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 218/455] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 17 20:34:14 2022] GET /identity/v3/role_assignments?group.id=e3c5c712634544b1a2553113377ce014&scope.project.id=3fb544378edc4844a8bfe4c58c6b8b04&role.id=1b7585de1b6442509e3788682caf8774 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:17.583341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cace31-1a8f-4b28-b31f-69cb4bdc42af None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:17.583886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cace31-1a8f-4b28-b31f-69cb4bdc42af None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:17.584189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cace31-1a8f-4b28-b31f-69cb4bdc42af None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:17.586040 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 238/456] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:17.591159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f3166-1b8e-422b-bd24-fa7470c93611 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:17.591944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f3166-1b8e-422b-bd24-fa7470c93611 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:17.592414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f3166-1b8e-422b-bd24-fa7470c93611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:17.617862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-381f3166-1b8e-422b-bd24-fa7470c93611 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:17.619514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-381f3166-1b8e-422b-bd24-fa7470c93611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:17.644630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:17.710842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 219/457] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:17.792534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6161c-c268-4dd0-b0f2-cc10d7d30031 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:17.792855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6161c-c268-4dd0-b0f2-cc10d7d30031 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:17.793087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6161c-c268-4dd0-b0f2-cc10d7d30031 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:17.794666 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 239/458] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:17.799695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c8b14-91ff-4657-bd23-55edc00c0c1d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:17.799985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c8b14-91ff-4657-bd23-55edc00c0c1d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:17.800265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c8b14-91ff-4657-bd23-55edc00c0c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:17.821727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3c6c8b14-91ff-4657-bd23-55edc00c0c1d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:17.822908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3c6c8b14-91ff-4657-bd23-55edc00c0c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:17.908203 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 220/459] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:18.035180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae264fb-1bc2-45de-9682-21688da1bca5 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.035463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae264fb-1bc2-45de-9682-21688da1bca5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.035735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae264fb-1bc2-45de-9682-21688da1bca5 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.037422 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 240/460] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:18.042311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dc4d9-f103-4c0a-bb44-a19e1e90017d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.042600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dc4d9-f103-4c0a-bb44-a19e1e90017d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.042879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dc4d9-f103-4c0a-bb44-a19e1e90017d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.065659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2f8dc4d9-f103-4c0a-bb44-a19e1e90017d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:18.066853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f8dc4d9-f103-4c0a-bb44-a19e1e90017d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:18.085692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae264fb-1bc2-45de-9682-21688da1bca5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.173285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 221/461] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:18.265420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae264fb-1bc2-45de-9682-21688da1bca5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:18.310105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:18.310741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.311013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.311281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.319468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dc4d9-f103-4c0a-bb44-a19e1e90017d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.333499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:18.333499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:18.349398 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:18.350392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:18.351587 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:18.351587 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:18.351587 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:18.351587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 241/462] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:18.352571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:18.361097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.437910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:18.519816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dc4d9-f103-4c0a-bb44-a19e1e90017d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:18.541854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3b05c-261b-4c62-ac0f-0b52a540b7ec None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:18.561866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:18.562693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.562980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.563250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.574364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:18.576269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:18.582799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:18.583823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.584364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.584364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.585379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:18.585725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:18.587375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:18.591877 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:18.593316 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:18.594385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:18.594385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:18.594385 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:18.600982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 222/463] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:18.602234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.605573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 242/464] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:18.607249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.672100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:18.710123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:18.717005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fad424-479b-4cd8-a947-2431ff34cd56 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:18.721317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf773-1b9e-4ba1-8bb9-4fa30981f371 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:18.756637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:18.757250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.757551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.757824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.758475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:18.758776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:18.760169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:18.761551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:18.762491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.763020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.763506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.776632 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 243/465] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:18.776632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.780105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:18.782204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:18.798470 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:18.799454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:18.800320 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:18.800320 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:18.800320 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:18.807316 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 223/466] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:18.808740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.875429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7a7af-ccf3-4671-8cfa-3d7d1a43d8f8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:18.911992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:18.912601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.912881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.913152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.913845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:18.915285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:18.915551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bde23-dcc1-4f20-af42-6344fc3d5d42 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:18.935220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:18.949578 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 244/467] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:34:18 2022] POST /identity/v3/users => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:18.949578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:18.956824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:18.973449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038100e-831f-4941-a3be-f0b6533e774e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:18.974113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038100e-831f-4941-a3be-f0b6533e774e None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:18.974391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038100e-831f-4941-a3be-f0b6533e774e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:18.974663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038100e-831f-4941-a3be-f0b6533e774e None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:18.975333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038100e-831f-4941-a3be-f0b6533e774e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:18.977035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038100e-831f-4941-a3be-f0b6533e774e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:19.005705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 224/468] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:34:18 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:19.006992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159f10-141b-45ef-af55-b804f8300423 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.007289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159f10-141b-45ef-af55-b804f8300423 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.007607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159f10-141b-45ef-af55-b804f8300423 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.009641 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 225/469] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:19.017500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf4097-aa29-457f-96c1-09aa97ba53c7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.017844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf4097-aa29-457f-96c1-09aa97ba53c7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.018068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf4097-aa29-457f-96c1-09aa97ba53c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.062691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-51bf4097-aa29-457f-96c1-09aa97ba53c7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:19.064023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-51bf4097-aa29-457f-96c1-09aa97ba53c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:19.146682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efe503-f47b-40f3-a465-cc90b9fb897a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:19.161194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 226/470] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:19.194755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bf918-e62d-4542-9279-06d32073a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:19.195392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bf918-e62d-4542-9279-06d32073a41d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.195664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bf918-e62d-4542-9279-06d32073a41d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.195933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bf918-e62d-4542-9279-06d32073a41d None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.196607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bf918-e62d-4542-9279-06d32073a41d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:19.196970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bf918-e62d-4542-9279-06d32073a41d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:19.198548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bf918-e62d-4542-9279-06d32073a41d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:19.213423 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 245/471] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:19.218816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf4097-aa29-457f-96c1-09aa97ba53c7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:19.343891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf4097-aa29-457f-96c1-09aa97ba53c7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:19.345589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:19.382896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059845a3-4bf6-447c-9ab0-ac311a527e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:19.383518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059845a3-4bf6-447c-9ab0-ac311a527e22 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.384117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059845a3-4bf6-447c-9ab0-ac311a527e22 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.384393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059845a3-4bf6-447c-9ab0-ac311a527e22 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.385052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059845a3-4bf6-447c-9ab0-ac311a527e22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:19.386555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059845a3-4bf6-447c-9ab0-ac311a527e22 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:19.411987 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 227/472] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:34:19 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:19.491227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c88d5-158d-4317-9b05-76d8cad0182e None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.491423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c88d5-158d-4317-9b05-76d8cad0182e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.491683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c88d5-158d-4317-9b05-76d8cad0182e None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.493484 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 246/473] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:19.499549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7693d-dc3d-4c74-aba3-4998d2d34425 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.499850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7693d-dc3d-4c74-aba3-4998d2d34425 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.500096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7693d-dc3d-4c74-aba3-4998d2d34425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.521383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-40a7693d-dc3d-4c74-aba3-4998d2d34425 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:19.522691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-40a7693d-dc3d-4c74-aba3-4998d2d34425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:19.547787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c88d5-158d-4317-9b05-76d8cad0182e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:19.632895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 228/474] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:19.651702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:19.698485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c88d5-158d-4317-9b05-76d8cad0182e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:19.735899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:19.736784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.737227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.737684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.750662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:19.752698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:19.768842 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:19.770021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:19.771194 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:19.771194 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:19.771194 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:19.772619 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 247/475] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:19.779104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7693d-dc3d-4c74-aba3-4998d2d34425 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:19.869148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:19.884805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7693d-dc3d-4c74-aba3-4998d2d34425 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:19.920676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:19.921299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:19.921615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:19.921890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:19.922537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:19.922835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:19.924236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:19.936359 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 229/476] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:19.942155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:20.043037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:20.066902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89599cbe-da0b-4359-a352-ae26a3d92095 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:20.098505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:20.111933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:20.112782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:20.113060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:20.113331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:20.114033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:20.115745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:20.144385 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 248/477] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:34:19 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:20.181064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3763b4-861a-46cc-828b-4b4695f1b67f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:20.221235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:20.222315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:20.222796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:20.223289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:20.224261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:20.225956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:20.240922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.sql.core [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 20:34:20.240922 ncdv-master-1 devstack@keystone.service[99166]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 20:34:20.240922 ncdv-master-1 devstack@keystone.service[99166]: [parameters: {'id': 'a110b59ec96545f0a7afb6d56a08dffc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 20:34:20.240922 ncdv-master-1 devstack@keystone.service[99166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 20:34:20.261662 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return ref.to_dict() Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:34:20.262589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application yield resource Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise self.value Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application session.commit() Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 20:34:20.263661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.flush() Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application compat.raise_( Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 20:34:20.264530 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:34:20.265417 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:34:20.270117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [parameters: {'id': 'a110b59ec96545f0a7afb6d56a08dffc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:20.270921 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 20:34:20.271713 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:20.272543 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 230/478] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 17 20:34:20 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:20.274924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:20.414227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74332ed4-b8aa-4f4e-b6d8-6c72ece3968f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:20.449786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab095d-a5f2-40d5-bd13-3410884bb356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:20.450458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab095d-a5f2-40d5-bd13-3410884bb356 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:20.450733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab095d-a5f2-40d5-bd13-3410884bb356 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:20.451003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab095d-a5f2-40d5-bd13-3410884bb356 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:20.461313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab095d-a5f2-40d5-bd13-3410884bb356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-37ab095d-a5f2-40d5-bd13-3410884bb356 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:20.466644 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:20.468755 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:20.470770 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 249/479] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:34:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:20.478960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:20.498487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:20.605287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14be6b-ad2f-4a82-923a-16331a171910 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:20.648269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e2e77-0d73-4ce7-a837-58b4855d2418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:20.648892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e2e77-0d73-4ce7-a837-58b4855d2418 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:20.649151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e2e77-0d73-4ce7-a837-58b4855d2418 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:20.649421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e2e77-0d73-4ce7-a837-58b4855d2418 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:20.650126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e2e77-0d73-4ce7-a837-58b4855d2418 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:20.650431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e2e77-0d73-4ce7-a837-58b4855d2418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:20.651824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e2e77-0d73-4ce7-a837-58b4855d2418 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:20.665013 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 231/480] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:22.072249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921c779-b393-4689-aa02-fcb0202139a2 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:22.072790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921c779-b393-4689-aa02-fcb0202139a2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:22.072790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921c779-b393-4689-aa02-fcb0202139a2 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:22.074333 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 250/481] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:22.079487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e497a04-ded8-4850-a5ee-0abe2e726bb9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:22.079882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e497a04-ded8-4850-a5ee-0abe2e726bb9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:22.080068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e497a04-ded8-4850-a5ee-0abe2e726bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:22.106511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3e497a04-ded8-4850-a5ee-0abe2e726bb9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:22.107808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3e497a04-ded8-4850-a5ee-0abe2e726bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:22.201607 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 232/482] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:22.335802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1256cb9-03f0-49a8-aec7-48a331304a0f None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:22.335998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1256cb9-03f0-49a8-aec7-48a331304a0f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:22.336266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1256cb9-03f0-49a8-aec7-48a331304a0f None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:22.337655 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 251/483] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:22.343414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6be9c-2452-45e8-8a95-90351aaa47c1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:22.343703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6be9c-2452-45e8-8a95-90351aaa47c1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:22.343979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6be9c-2452-45e8-8a95-90351aaa47c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:22.356218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:22.370432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-05d6be9c-2452-45e8-8a95-90351aaa47c1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:22.371635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-05d6be9c-2452-45e8-8a95-90351aaa47c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:22.456957 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 233/484] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:22.636065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b044-95d4-4df9-8f85-78fdb9bc3553 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:22.636444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b044-95d4-4df9-8f85-78fdb9bc3553 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:22.636622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b044-95d4-4df9-8f85-78fdb9bc3553 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:22.638029 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 252/485] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:22.651261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405b6b5-2f80-466c-8fc6-fff76aac35b0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:22.651553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405b6b5-2f80-466c-8fc6-fff76aac35b0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:22.651823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405b6b5-2f80-466c-8fc6-fff76aac35b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:22.682484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a405b6b5-2f80-466c-8fc6-fff76aac35b0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:22.683680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a405b6b5-2f80-466c-8fc6-fff76aac35b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:22.708841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b044-95d4-4df9-8f85-78fdb9bc3553 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:22.783036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:22.783981 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 234/486] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:22.864500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b044-95d4-4df9-8f85-78fdb9bc3553 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:22.892696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405b6b5-2f80-466c-8fc6-fff76aac35b0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:22.912909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:22.913566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:22.913844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:22.914114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:22.936152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:22.943040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:22.943952 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:22.944764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:22.947031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 253/487] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:22.953709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.026772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.057298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.060482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d78f54-ef16-4e18-8909-017b9c90fff4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.095009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.095672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.095950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.096382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.097187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:23.097674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.099274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.108512 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 254/488] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:23.114297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.116983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405b6b5-2f80-466c-8fc6-fff76aac35b0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.202271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.203026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.203404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.203772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.230869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:23.235948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc647efb-ca57-4a8f-846e-95852cffb9e1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:23.237019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:23.238019 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:23.238855 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:23.243560 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 235/489] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:23.244441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.277182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.278282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.278755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.279215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.290879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.292626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:23.307424 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:23.308387 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:23.309263 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:23.309263 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:23.309263 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:23.311081 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 255/490] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:23.312158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.331548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.374765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5202e2-0813-4717-b4c9-f81618a9dd03 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.398265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.409745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.410669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.411165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.411645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.421985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:23.427306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:23.428243 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:23.429133 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:23.430972 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 236/491] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:23.431599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.433833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e709fff-ae0c-4bb6-ac08-5c24c6dc07e6 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.470255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cd18d-a606-48d7-a3d6-67e100632493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.470773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cd18d-a606-48d7-a3d6-67e100632493 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.471049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cd18d-a606-48d7-a3d6-67e100632493 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.471614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cd18d-a606-48d7-a3d6-67e100632493 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.471976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cd18d-a606-48d7-a3d6-67e100632493 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:23.472280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cd18d-a606-48d7-a3d6-67e100632493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.473805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cd18d-a606-48d7-a3d6-67e100632493 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.482524 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 256/492] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:23.483671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cef5cd-960d-40cc-b86a-b95b1dd45ffa None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.483957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cef5cd-960d-40cc-b86a-b95b1dd45ffa None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.484228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cef5cd-960d-40cc-b86a-b95b1dd45ffa None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.485536 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 257/493] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:23.486160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cef5cd-960d-40cc-b86a-b95b1dd45ffa None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.553689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebd3db-04ed-409e-b693-9b147b9de26a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.569055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.588257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.588867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.589139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.589405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.590075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:23.590372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.591834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.594373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cef5cd-960d-40cc-b86a-b95b1dd45ffa None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.608840 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 237/494] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:23.608840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.636804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3d4e4-6070-499b-8df8-083db4f35843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.637485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3d4e4-6070-499b-8df8-083db4f35843 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.637796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3d4e4-6070-499b-8df8-083db4f35843 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.638067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3d4e4-6070-499b-8df8-083db4f35843 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.638726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3d4e4-6070-499b-8df8-083db4f35843 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:23.639040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3d4e4-6070-499b-8df8-083db4f35843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.640418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3d4e4-6070-499b-8df8-083db4f35843 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.648850 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 258/495] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:23.651336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504038-e22d-47ec-8a80-c8e900fe0cf6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.652219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504038-e22d-47ec-8a80-c8e900fe0cf6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.652960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c504038-e22d-47ec-8a80-c8e900fe0cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.654606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.675811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9c504038-e22d-47ec-8a80-c8e900fe0cf6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:23.677004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c504038-e22d-47ec-8a80-c8e900fe0cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:23.713251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc6be-e436-4509-b046-0cec6ca9d9d7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.747262 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 259/496] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:23.748406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504038-e22d-47ec-8a80-c8e900fe0cf6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.758430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.759094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.759371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.759711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.774137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.776105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:23.790724 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:23.791680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:23.792612 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:23.792612 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:23.792612 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:23.796700 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 238/497] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:23.797508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.820728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.855994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c504038-e22d-47ec-8a80-c8e900fe0cf6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.892262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a94207-ea0c-4c0e-a932-c41a19ba0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.892891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a94207-ea0c-4c0e-a932-c41a19ba0335 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.893167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a94207-ea0c-4c0e-a932-c41a19ba0335 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.893441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a94207-ea0c-4c0e-a932-c41a19ba0335 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.896511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:23.902085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a94207-ea0c-4c0e-a932-c41a19ba0335 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.903612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a94207-ea0c-4c0e-a932-c41a19ba0335 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.905378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b26d3b-4b54-4f45-a833-086fb4a2b49b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-17a94207-ea0c-4c0e-a932-c41a19ba0335 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:23.921042 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:23.922054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:23.923016 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 260/498] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:23.923637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9164a0-11ca-431a-9464-a76a367922c8 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.923708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9164a0-11ca-431a-9464-a76a367922c8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.924699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9164a0-11ca-431a-9464-a76a367922c8 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.925123 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 261/499] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:23.929684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9164a0-11ca-431a-9464-a76a367922c8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:23.950553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:23.951220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:23.951496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:23.951765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:23.963482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:23.965547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:23.983446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:23.984354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:23.992109 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:23.992109 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:23.992109 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:23.992109 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 239/500] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:23.992109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.038725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9164a0-11ca-431a-9464-a76a367922c8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.066563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.075214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba39c0-8f02-40d1-8ffe-dd9eb4cfaac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.075851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba39c0-8f02-40d1-8ffe-dd9eb4cfaac2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.076128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba39c0-8f02-40d1-8ffe-dd9eb4cfaac2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.076400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba39c0-8f02-40d1-8ffe-dd9eb4cfaac2 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.077041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba39c0-8f02-40d1-8ffe-dd9eb4cfaac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:24.077345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba39c0-8f02-40d1-8ffe-dd9eb4cfaac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.078778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba39c0-8f02-40d1-8ffe-dd9eb4cfaac2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.090055 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 262/501] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:24.091294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c18b-0396-451d-bc5f-91ea5c5281c3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.091476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58f342-58cf-4c6d-9877-be555ead2343 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.091586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c18b-0396-451d-bc5f-91ea5c5281c3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.091835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c18b-0396-451d-bc5f-91ea5c5281c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.111014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5d03c18b-0396-451d-bc5f-91ea5c5281c3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:24.112227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5d03c18b-0396-451d-bc5f-91ea5c5281c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:24.126867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.127477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.127757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.128026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.128765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:24.129072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.130565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.166900 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 240/502] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:34:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:24.168004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.176748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.183230 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 263/503] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:24.184081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c18b-0396-451d-bc5f-91ea5c5281c3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.285158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c18b-0396-451d-bc5f-91ea5c5281c3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.290203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b24698-d28a-4b39-81db-5e56e5d66e78 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.305954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.325693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.326258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.326533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.326808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.332451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.333058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.333339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.333643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.334297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:24.334599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.335995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.337822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:24.343711 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:24.344605 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:24.345417 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:24.346092 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 264/504] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:24.346639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.352904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 241/505] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:24.354082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.421867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.448564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af823ad9-ebcb-44ac-831c-01fdc8844b2d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.459436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4b0e0-3777-45fe-97f1-1c51744e2d8f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.482321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.483120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.483714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.484002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] PATH_INFO: `/v3/users/glance` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.491088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.492544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.493624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.494404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.494681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.495063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:24.503995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:24.504934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 265/506] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:24.505880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.545143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.566100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.567621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.611043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e245d7-22a4-4c9d-85ba-03c27d2b641d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.628925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.645231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.645865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.646134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.646397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.647051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:24.647344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.648743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.657234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 266/507] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:24.657968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.668108 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 242/508] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:24.669225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.761020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.764074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5d4f6-191d-4fde-bc4f-0792cdfeba69 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.767361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb0526-89ba-4793-91ec-b805cd36c7d8 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.800477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.801028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.801298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.801597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.802674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.803308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.803582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.803853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.811212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.811468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.812696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.813046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:24.823575 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:24.824493 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 267/509] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:24.825592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:24.827463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:24.828327 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:24.829221 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:24.829221 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:24.829221 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:24.829221 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 243/510] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:24.829970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.848487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:24.926339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ca395-2a82-40a9-b3f3-b0642f909ddb None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.926993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da3871-96d1-43e0-b2a0-b61d5ba05840 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:24.959423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.960021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.960294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.960396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:24.960562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.960936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:24.961212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:24.961308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:24.961489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:24.961590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.963033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.972473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:24.974019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:24.988694 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:24.989656 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:24.990505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:24.990505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:24.990505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:24.990505 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 244/511] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:24.990734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:24.998968 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 268/512] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:24.999651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.018165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.062164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.094004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a359c75-1ed0-499f-bf23-04d1ee97f92d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.099974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4727f-a8e7-4953-ad3a-bdbb4c8ea193 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.126486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.127126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.127700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.127700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.133526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.134494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.134999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.135361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.136097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:25.136406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.137950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.139192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.140655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.155171 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 269/513] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:25.156957 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:25.157901 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:25.158793 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:25.158793 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:25.158793 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:25.158957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.159034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 245/514] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:25.159034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.242161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.260962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43402f49-5936-46b0-b720-5e6595059afe None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.266139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ef51d-0fa6-40bd-b4de-6acb7dd3a5c1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.290860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.302418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.303027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.303301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.303566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.304204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:25.304502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.305986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.306260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.306871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.307151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.307424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.308066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:25.308376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.309807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.317779 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 246/515] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:25.318355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.321056 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 270/516] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:25.321905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.419515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf148a-2dd1-4cde-8f2a-e1cc3be920bc None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.423333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66ee78-572b-434f-b3cf-d8539c868dff None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.451583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.452184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.452457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.452725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.456030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.456140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.456682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.456961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.457226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.504332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.520138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.521636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.522081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.523524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.621601 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 247/517] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:25.622510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.644783 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 271/518] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:25.645577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.690085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.713654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.732344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62484c4-1e0a-48d0-ae60-16335fc6921f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.755170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6687b-b5e0-406f-9a3a-e7650866e1bc None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.765503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.766110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.766382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.766649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.767278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:25.767574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.768960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.779928 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 248/519] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:25.780691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.789717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.790355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.790630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.790899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.797995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.799454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:25.811052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:25.812007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 272/520] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:25.812919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.883248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916f9cb-6d79-41f8-8de1-19a3a41ba521 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.905140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.911666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9976cc3-3a5e-4116-9b08-d724296ea1b7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:25.916399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.917010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.917289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.917593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.918268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.919733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.935675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:25.940654 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 249/521] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:34:25 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:25.941644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:25.958884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:25.959546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:25.959819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:25.960080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:25.972911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:25.974291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:25.991984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:25.992984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:25.999244 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:25.999244 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:25.999244 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:25.999244 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 273/522] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:25.999244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.060596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9563012-04e5-452c-a50a-90052ddbefd0 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.096883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.097566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.097849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.098122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.110107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.111662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.112653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da032338-39c6-47bd-8937-a7e960a92b29 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:26.127860 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:26.129410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:26.130562 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:26.130562 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:26.130562 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:26.133775 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 250/523] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:26.134670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.156608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.157256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.157564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.157838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.166374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.167835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:26.181167 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:26.182431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 274/524] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:26.183585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.202112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.202304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.258208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61b88a-e4ce-4750-83bd-d4a1bd54d791 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.292230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.292848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.293127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.293398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.294264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:26.294571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.296030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.300358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dae97-8641-44d3-a27d-d59124e4056c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.313696 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 251/525] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:26.314523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.344508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.345118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.345390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.345689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.346330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:26.346629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.347998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.360815 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 275/526] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:26.362502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.416852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d679402-e015-46ec-b547-aebb76ed0215 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.426605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.438271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.454490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.455118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.455384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.455680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.456374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:26.456696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.458130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.470075 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 252/527] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:26.470891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.479411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4497-45e2-438b-83c8-36f21fa3aedc None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.516424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.517301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.517842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.518271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.519178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:26.519615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.521342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.542703 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 276/528] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:26.543834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.578594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c30dbc-61c9-44e2-82af-0b60d650f1dd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.614963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.615567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.615851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.616132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.651199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd500e-d59d-453b-b685-6e37dd1ebd4f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.651727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.689833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.691374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.693229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.699677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.700314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.700584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.700847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.708346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.709840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:26.721037 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:26.721951 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:26.722802 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 277/529] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:26.723271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.805648 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 253/530] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:26.806861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.831646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e07bd2-cc91-4b69-9337-dbecf2d8277d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.866807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.867433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.867716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.867980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.873439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.874798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.876300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:26.887466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:26.888411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 278/531] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:26.894031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:26.916307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027f828-a642-4391-b7ae-4417ba8c085f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:26.943513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:26.950918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:26.951532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:26.951811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:26.952088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:26.952817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=724ce8c58c1b473e9e3af18009532c9f) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:26.953125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:26.954606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:26.969514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 254/532] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/role_assignments?user.id=724ce8c58c1b473e9e3af18009532c9f&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:26.969514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:27.015553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55acfcf-3420-4f85-bbd7-8df22cefdc69 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:27.065933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:27.066586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:27.067099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:27.067369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:27.068528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:27.069084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:27.070609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:27.071117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50ce9-84ab-4944-bc18-8b91d6a92046 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:27.093853 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 279/533] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:27.094890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:27.108925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:27.116125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:27.116775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:27.117069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:27.117371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:27.130773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:27.132432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:27.149732 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:27.150714 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:27.152848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:27.152848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:27.152848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:27.152848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 255/534] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:27.152848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:27.225977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab254c7-1ac8-4318-af55-1ebce49bfd9e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:27.240215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:27.268355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:27.269305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:27.269975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:27.270353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:27.271164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:27.271567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:27.273610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:27.285242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff01a89-a7ab-423b-aa5a-3067c2ea4b33 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:27.297056 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 280/535] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:27.298048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:27.329699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:27.330570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:27.330847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:27.331116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:27.331995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=94ce9c64a0bb478485c47ebedd5a29f3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:27.332304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:27.334401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:27.342060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 256/536] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:27 2022] GET /identity/v3/role_assignments?user.id=94ce9c64a0bb478485c47ebedd5a29f3&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:27.342753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:27.430634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:27.483192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec7bee-9835-41bf-bdd9-d15fad0f1719 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:27.514648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c1e98-f43f-44fd-b77f-2587f405f8c1 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:27.545192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:27.545987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:27.546420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:27.546692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:27.547357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:27.547652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:27.549069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:27.552640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:27.553781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:27.554202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:27.554613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:27.555516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=d45ab41902864586811c81948540fdac) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:27.555966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:27.557602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:27.564216 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 257/537] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:27 2022] GET /identity/v3/role_assignments?user.id=d45ab41902864586811c81948540fdac&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:27.577863 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 281/538] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:27.584168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:27.736723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42141a94-ca40-42ef-b760-3ff5f708373e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:27.756354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:27.789386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:27.790066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:27.790339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:27.790728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:27.807545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:27.809430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:27.824276 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:27.825284 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:27.826232 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:27.828640 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 258/539] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:27.834361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:27.864096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:27.963227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c46c9-5da1-4bd9-92d3-b6a879723d97 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:28.015139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51659603-f7cc-46aa-be2b-be229b1ac853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:28.016142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51659603-f7cc-46aa-be2b-be229b1ac853 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:28.016702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51659603-f7cc-46aa-be2b-be229b1ac853 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:28.017166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51659603-f7cc-46aa-be2b-be229b1ac853 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:28.018100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51659603-f7cc-46aa-be2b-be229b1ac853 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:28.018602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51659603-f7cc-46aa-be2b-be229b1ac853 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:28.020234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51659603-f7cc-46aa-be2b-be229b1ac853 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:28.037383 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 282/540] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:28.043701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:28.184885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7570665-2eda-4749-8041-d065d890dc54 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:28.232987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb84f4-2e71-4743-a41b-6e30ea7ba403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:28.233628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb84f4-2e71-4743-a41b-6e30ea7ba403 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:28.233941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb84f4-2e71-4743-a41b-6e30ea7ba403 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:28.234215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb84f4-2e71-4743-a41b-6e30ea7ba403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:28.234946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb84f4-2e71-4743-a41b-6e30ea7ba403 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=a824d5df8638409bbaf4d6e8d22d63ca) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:28.235247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb84f4-2e71-4743-a41b-6e30ea7ba403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:28.236693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb84f4-2e71-4743-a41b-6e30ea7ba403 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:28.243945 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 259/541] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:28 2022] GET /identity/v3/role_assignments?user.id=a824d5df8638409bbaf4d6e8d22d63ca&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:28.502613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03d14-d901-479f-a677-b0ab79980801 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:28.503708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03d14-d901-479f-a677-b0ab79980801 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:28.504221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03d14-d901-479f-a677-b0ab79980801 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:28.506294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 283/542] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:28.517410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70919ec6-3c4f-48c4-88b4-755741c40440 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:28.517767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70919ec6-3c4f-48c4-88b4-755741c40440 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:28.517988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70919ec6-3c4f-48c4-88b4-755741c40440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:28.542462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-70919ec6-3c4f-48c4-88b4-755741c40440 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:28.543657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-70919ec6-3c4f-48c4-88b4-755741c40440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:28.582388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:28.622672 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 260/543] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:29.022483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad03d14-d901-479f-a677-b0ab79980801 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:29.133946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:29.149698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad03d14-d901-479f-a677-b0ab79980801 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:29.187097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:29.188208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:29.188687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:29.189167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:29.200717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:29.205841 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:29.206783 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:29.207609 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:29.210275 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 284/544] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:29.218474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70919ec6-3c4f-48c4-88b4-755741c40440 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:29.364369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70919ec6-3c4f-48c4-88b4-755741c40440 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:29.412873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:29.413516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:29.413799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:29.414067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:29.414744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:29.415040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:29.416431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:29.429217 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 261/545] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:29.435314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:29.573030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3a76-4353-4aaf-bc78-ce60d735fcef None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:29.615888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:29.616835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:29.617328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:29.617822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:29.632366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:29.634299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:29.647471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:29.655460 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:29.656725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:29.657716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:29.657716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:29.657716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:29.660691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 285/546] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:29.666557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:29.713699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:29.825220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361880d3-e460-4975-a5f6-b5d1facb807a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:29.874265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fcb29-6011-4c6e-836e-cdd63cb77b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:29.874803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fcb29-6011-4c6e-836e-cdd63cb77b2d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:29.875081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fcb29-6011-4c6e-836e-cdd63cb77b2d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:29.875350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fcb29-6011-4c6e-836e-cdd63cb77b2d None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:29.876006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fcb29-6011-4c6e-836e-cdd63cb77b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:29.876309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fcb29-6011-4c6e-836e-cdd63cb77b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:29.877721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fcb29-6011-4c6e-836e-cdd63cb77b2d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:29.892723 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 262/547] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:29.898335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:30.050156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d5c9-1d91-4d98-a403-8583a92887a8 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.050493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d5c9-1d91-4d98-a403-8583a92887a8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.051010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d5c9-1d91-4d98-a403-8583a92887a8 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.053282 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 263/548] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:30.057190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9381-70e7-458a-8baf-1f61e2cbc736 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:30.057321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e82a90-e83d-4f36-a3cd-11bb3c29fd72 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.057635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e82a90-e83d-4f36-a3cd-11bb3c29fd72 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.057972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e82a90-e83d-4f36-a3cd-11bb3c29fd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.087151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-25e82a90-e83d-4f36-a3cd-11bb3c29fd72 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:30.088352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25e82a90-e83d-4f36-a3cd-11bb3c29fd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:30.121137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:30.122427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.122907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.123616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] PATH_INFO: `/v3/users/swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.141876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:30.144669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:30.165137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:30.166370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:30.167299 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:30.169921 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 286/549] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:30.176860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:30.245877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:30.255265 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 264/550] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:30.294908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b916f-8f61-4d43-bead-81ab2291128c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:30.331224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10593d-483e-4caf-902f-e8af1d2f5deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:30.332203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10593d-483e-4caf-902f-e8af1d2f5deb None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.332733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10593d-483e-4caf-902f-e8af1d2f5deb None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.333202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10593d-483e-4caf-902f-e8af1d2f5deb None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.334233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10593d-483e-4caf-902f-e8af1d2f5deb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:30.334766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10593d-483e-4caf-902f-e8af1d2f5deb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:30.336402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10593d-483e-4caf-902f-e8af1d2f5deb None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:30.354749 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 287/551] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 17 20:34:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:30.366575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e82a90-e83d-4f36-a3cd-11bb3c29fd72 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:30.392061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:30.512063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e82a90-e83d-4f36-a3cd-11bb3c29fd72 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:30.555110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:30.555722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.555991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.556255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.638736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f37e9-9bf9-4656-b2d8-6ff6cb3adebe None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.639709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f37e9-9bf9-4656-b2d8-6ff6cb3adebe None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.639882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:30.640379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f37e9-9bf9-4656-b2d8-6ff6cb3adebe None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.641347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:30.642581 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 288/552] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:30.648519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9168008-d600-4b54-80d2-f45317f65e67 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.649147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9168008-d600-4b54-80d2-f45317f65e67 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.649654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9168008-d600-4b54-80d2-f45317f65e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.671352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e9168008-d600-4b54-80d2-f45317f65e67 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:30.672937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e9168008-d600-4b54-80d2-f45317f65e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:30.676424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:30.751450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 289/553] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:30.752874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991898-aea3-4466-9114-122e3445624c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.761949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991898-aea3-4466-9114-122e3445624c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.762333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991898-aea3-4466-9114-122e3445624c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.767918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 290/554] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:30.771473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062a7d-1447-4c8a-bc27-221a4d2a3e4e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.771918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062a7d-1447-4c8a-bc27-221a4d2a3e4e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.772320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e062a7d-1447-4c8a-bc27-221a4d2a3e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:30.798134 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 265/555] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:30.805496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:30.816727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0e062a7d-1447-4c8a-bc27-221a4d2a3e4e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:30.817963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0e062a7d-1447-4c8a-bc27-221a4d2a3e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:30.869810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:30.905596 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 291/556] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:30.905596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e062a7d-1447-4c8a-bc27-221a4d2a3e4e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:30.943797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95445b-6bf1-4035-83b0-7afe1dbc9a6f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:30.967210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:30.998646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:30.999287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:30.999567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:30.999842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.014778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.016252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:31.046466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:31.047479 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:31.048446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:31.048446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:31.048446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:31.049137 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 266/557] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:31.053635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.085495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e062a7d-1447-4c8a-bc27-221a4d2a3e4e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.136701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.137344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.137684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.137957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.149101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:31.154038 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:31.154955 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:31.155865 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 292/558] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:31.156947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.186003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c690ec6-e263-4b8e-be50-b0443daaf482 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.203222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.220883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404164e9-edd6-46e4-93d3-e764d4a42dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.221751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404164e9-edd6-46e4-93d3-e764d4a42dda None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.222035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404164e9-edd6-46e4-93d3-e764d4a42dda None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.222305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404164e9-edd6-46e4-93d3-e764d4a42dda None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.222958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404164e9-edd6-46e4-93d3-e764d4a42dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:31.223274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404164e9-edd6-46e4-93d3-e764d4a42dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.224653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404164e9-edd6-46e4-93d3-e764d4a42dda None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.236649 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 267/559] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:31.237865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa32e6-18cd-4af4-9782-3bd8004d944e None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.238153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa32e6-18cd-4af4-9782-3bd8004d944e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.238631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa32e6-18cd-4af4-9782-3bd8004d944e None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.243586 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 268/560] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:31.243586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa32e6-18cd-4af4-9782-3bd8004d944e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.271604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.277518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563b175-3e0e-4161-8e6f-12411ab2cdb6 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.312610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.313229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.313532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.313799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.327133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:31.331975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:31.332937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 293/561] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:31.333830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.347417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa32e6-18cd-4af4-9782-3bd8004d944e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.380410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4794f-d8fb-4dd2-aa51-4c79b123f3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.381107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4794f-d8fb-4dd2-aa51-4c79b123f3fd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.381391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4794f-d8fb-4dd2-aa51-4c79b123f3fd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.381709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4794f-d8fb-4dd2-aa51-4c79b123f3fd None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.382381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4794f-d8fb-4dd2-aa51-4c79b123f3fd None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:31.382682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4794f-d8fb-4dd2-aa51-4c79b123f3fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.384064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4794f-d8fb-4dd2-aa51-4c79b123f3fd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.391801 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 269/562] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:31.392933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dda9e-c921-448a-8e84-ac1562f19830 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.393210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dda9e-c921-448a-8e84-ac1562f19830 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.393507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dda9e-c921-448a-8e84-ac1562f19830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.411549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-566dda9e-c921-448a-8e84-ac1562f19830 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:31.412658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-566dda9e-c921-448a-8e84-ac1562f19830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:31.434181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.438402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc5d44-f504-42ed-9220-4ebb43bf5644 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.477632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.478231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.478499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.478795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.479124 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 270/563] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:31.479964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dda9e-c921-448a-8e84-ac1562f19830 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.492818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.495206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:31.507429 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:31.508361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 294/564] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:31.514718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.517573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.594468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dda9e-c921-448a-8e84-ac1562f19830 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.621538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaff19-447f-44f9-b599-b7ef0ee4410d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.629078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.629758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.630033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.630301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.640969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:31.646036 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:31.646981 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:31.647817 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 271/565] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:31.648503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.661325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.662026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.662246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.662523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.663192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:31.663492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.664862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.673340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 295/566] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:31.674279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.698424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.748552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.768676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74243c-55b2-42d4-845d-f90a2fea5f80 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.779063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce681de1-694e-4be9-96ea-1981cc127a3c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.813004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.813680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.813956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.814154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.814289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.814763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.815040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.815309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.816004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:31.816303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.817770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.825785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.827464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.830306 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 296/567] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:31.830876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:31.843147 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:31.844341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:31.845548 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:31.845548 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:31.845548 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:31.845548 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 272/568] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:31.845548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.921888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.940631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b6255-f876-4e2e-b8f6-77a5fc2e6188 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.948439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a689a4-8c2f-4ad0-8c9b-153d338dc07b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:31.972894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.973527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.973803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.974078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.974737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:31.975033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.976413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:31.980903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:31.981591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:31.981867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:31.982132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:31.984015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:31.984147 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 297/569] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:31.984927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:31.994502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:31.995504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:32.009774 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:32.010784 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:32.011710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:32.011710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:32.011710 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:32.011710 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 273/570] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:32.011710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.089126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccda627-3310-431d-885b-aa5b21124b99 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.108361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f5708-4353-49aa-a603-1f3b53ae9958 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.123239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.123870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.124144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.124411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.134810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:32.140040 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:32.141200 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 298/571] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:32.142475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.144578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.145173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.145308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.145443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.146216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.146944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=8eea4641bfd940fe8a94ec79484ef4d3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:32.147269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.148860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.155830 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 274/572] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/role_assignments?user.id=8eea4641bfd940fe8a94ec79484ef4d3&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:32.157219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.230500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.269485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402346fa-9763-48ce-87a8-1ec5828e7b52 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.292578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19553d73-5d10-4143-8950-7144a5abab70 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.309300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.310094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.310371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.310642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.311301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:32.311605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.313060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.330811 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 299/573] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:32.331601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.335528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.336259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.336550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.336814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.348857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.350455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:32.365924 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:32.366969 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:32.367872 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:32.367872 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:32.367872 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:32.367872 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 275/574] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:32.367872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.405170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.450599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd7037-ccc1-4426-b82f-9ce5f9a0b2a4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.478021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78042eaa-3c0a-4386-978b-13a132ec6a0f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.479833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.488522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.489064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.489344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.489655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.490465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:32.490776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.492160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.509065 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 300/575] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:32.509065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.518840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.519458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.519801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.520131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.520816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:32.521121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.522554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.533515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 276/576] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:32.533515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.629573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a54a86-8f37-44ea-bb40-bc9ca10e7f90 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.633908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5dc9c-db46-49e5-a641-a5ec62af1656 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.654562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.671247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.671869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.672150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.672426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.672804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.673239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:32.673419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.673539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.673750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.674033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.674791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.684041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.685673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.687639 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 277/577] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:32.688688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:32.706732 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:32.707766 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 301/578] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:32.713764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.717354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.797788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356fdc9-6d72-4233-8554-8b0bd745d959 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.822593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f73f2-49e5-4957-80a6-9807a2a7b324 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.832565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570a306-4384-4136-9762-87416988a83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.833188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570a306-4384-4136-9762-87416988a83b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.833510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570a306-4384-4136-9762-87416988a83b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.833745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570a306-4384-4136-9762-87416988a83b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.840951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570a306-4384-4136-9762-87416988a83b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.842472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570a306-4384-4136-9762-87416988a83b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-2570a306-4384-4136-9762-87416988a83b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:32.855275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:32.856231 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 278/579] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:32.857275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570a306-4384-4136-9762-87416988a83b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.864382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:32.865213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:32.865536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:32.865809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:32.877499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:32.878982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:32.894972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:32.895954 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:32.896870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:32.896870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:32.896870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:32.896870 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 302/580] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:32.897187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:32.935582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.950392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:32.966425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570a306-4384-4136-9762-87416988a83b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:32.999616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.000243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.000522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.000793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] PATH_INFO: `/v3/users/glance` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.008074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.009567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.009680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87edbe-7f62-47f9-ba25-0eda8f8442bf None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:33.020521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:33.021496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:33.022468 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 279/581] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:33.023054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.053594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.054491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.054912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.055343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.056378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:33.056889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.058515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.078340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 303/582] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:34:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:33.079346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.137283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac59b7-146c-40c1-b2e4-8f65ac88cebb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.176291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.176501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.177266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.177594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.177969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.178952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:33.179264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.180718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.186799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fd9c4-9b38-47c1-9e4a-4ef6a764e3ed None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.199408 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 280/583] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:33.200523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.203128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.226093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.227135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.227135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.227549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.228277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:33.228721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.230213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.243154 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 304/584] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:33.248345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.310434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a70e4-c6f9-43c1-a186-3bccedb24260 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.347132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.347861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.348182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.348513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.349320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:33.349735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.351260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.359585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c197fb4-9ce9-4cf6-95d7-e64f557bb5ae None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.369332 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 281/585] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:33.370210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.399386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.400021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.400301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.400576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.412045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.413574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:33.427991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:33.429215 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:33.430368 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:33.430368 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:33.430368 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:33.431087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.433853 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 305/586] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:33.434849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.440118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.473972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb473b-dea8-4e22-bf07-dab7438f9cce None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.509844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.510478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.510755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.511028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.521491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.522973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:33.539373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:33.540328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:33.541236 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:33.541236 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:33.541236 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:33.541236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 282/587] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:33.541236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.548363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78acf88-09d1-45b0-a26f-5ab08e9ec2b1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.584407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.584978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.585252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.585554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.593407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.594878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:33.606342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:33.607495 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 306/588] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:33.608449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.648231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e2874-77d5-4960-aae7-7cbedf016f1f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.662128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.682241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.689451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.690170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.690453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.690726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.701888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.703354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:33.717780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:33.718736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:33.719666 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:33.719666 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:33.719666 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:33.719666 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 283/589] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:33.719666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.734089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc043155-7e4f-49c7-b5a2-0dc497840004 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.768054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.768672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.768948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.769224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.769895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:33.770198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.771665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.784000 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 307/590] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:33.784738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.836741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a8ec-dcc1-4404-8969-d6161287b225 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.885786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.886404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.886689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.886950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.887628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:33.887931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.889322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.894074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f3ad-8db7-46dd-a7be-c8f37ccf2828 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:33.903260 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 284/591] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:33.904090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:33.934912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.935854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:33.946142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:33.946881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:33.947256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:33.947634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:33.948480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:33.948875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:33.950459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:33.968892 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 308/592] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:33.969753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.019603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5bcb2-f405-4f42-9ae9-04fe41c4c88d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.055360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.055969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.056249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.056525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.057291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:34.057628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.059146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.071168 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 285/593] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:34.073240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2c615-9b4e-44a5-b078-d80bf4e014bb None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.074441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.110099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.110961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.111383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.111831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.119920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.121433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:34.134159 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:34.135180 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:34.136113 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 309/594] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:34.136635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.172196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.186301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.188439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aa2bc-67da-4bea-bbb1-d8d016b48961 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.222340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.223067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.223413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.223696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.230484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.232005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:34.244914 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:34.245913 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 286/595] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:34.246830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.250986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9155b-5e44-43e6-b3e0-fd14cddeeefd None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.290642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.291389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.291736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.292071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.304146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.305768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:34.321084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:34.322332 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:34.323474 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:34.323474 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:34.323474 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:34.328854 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 310/596] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:34.328854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.355984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624538b-4449-4b6e-935c-7692e2e60e16 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.388983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.389622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.389903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.390190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.396773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.398263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:34.409760 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:34.410856 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:34.411974 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:34.415444 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 287/597] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:34.415444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.418336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.435683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.445232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c2469-c24f-4e80-9eb2-508c79218625 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.480307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.481172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.481929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.482276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.483237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:34.483545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.484993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.498302 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 311/598] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:34.499119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.529390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81265d21-68b0-4541-827f-3b875dc84409 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.565236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.565879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.566159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.566430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.567122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:34.567423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.568871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.586306 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 288/599] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:34.587125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.606096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3a8bb-61d7-4c91-9b75-35ff12013c9d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.639653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.640227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.640495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.640737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.641402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:34.641731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.643106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.646844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.655572 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 312/600] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:34.656791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.673502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.693530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafd974-7024-427a-81f4-a6743f9b644a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.725986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d781d-5306-4bd6-b8b7-aa3457880f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.726598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d781d-5306-4bd6-b8b7-aa3457880f1b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.726876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d781d-5306-4bd6-b8b7-aa3457880f1b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.727147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d781d-5306-4bd6-b8b7-aa3457880f1b None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.727830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d781d-5306-4bd6-b8b7-aa3457880f1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:34.728139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d781d-5306-4bd6-b8b7-aa3457880f1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.729595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d781d-5306-4bd6-b8b7-aa3457880f1b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.741071 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 289/601] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:34.742259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd75671-cfb5-4b45-9f2d-da15030fca37 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.742540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd75671-cfb5-4b45-9f2d-da15030fca37 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.742816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd75671-cfb5-4b45-9f2d-da15030fca37 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.748465 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 290/602] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:34.748465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd75671-cfb5-4b45-9f2d-da15030fca37 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.759060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5afd-8377-4b52-8bb7-bcf7cde21c50 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.794049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.794698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.794978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.795253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/724ce8c58c1b473e9e3af18009532c9f/roles/b888b6f28fcb407795b56d37fdf7c6f6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.809024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=724ce8c58c1b473e9e3af18009532c9f, role_id=b888b6f28fcb407795b56d37fdf7c6f6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.810586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.824563 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 313/603] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:34:34 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/724ce8c58c1b473e9e3af18009532c9f/roles/b888b6f28fcb407795b56d37fdf7c6f6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:34.825414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.858362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd75671-cfb5-4b45-9f2d-da15030fca37 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:34.884597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.898695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:34.899379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:34.899718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:34.899997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/d45ab41902864586811c81948540fdac/roles/b888b6f28fcb407795b56d37fdf7c6f6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:34.907114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:34.919427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=d45ab41902864586811c81948540fdac, role_id=b888b6f28fcb407795b56d37fdf7c6f6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:34.920922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:34.936138 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 291/604] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:34:34 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/d45ab41902864586811c81948540fdac/roles/b888b6f28fcb407795b56d37fdf7c6f6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:34.946538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:34.955208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e510e33-141d-4c7d-85cd-0b7cfcaa4a86 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:35.012545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747f9f9-837d-4ccd-ac61-c3c450fe1f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:35.013576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f9f9-837d-4ccd-ac61-c3c450fe1f55 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:35.014040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f9f9-837d-4ccd-ac61-c3c450fe1f55 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:35.014468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f9f9-837d-4ccd-ac61-c3c450fe1f55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:35.025675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747f9f9-837d-4ccd-ac61-c3c450fe1f55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:35.028514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747f9f9-837d-4ccd-ac61-c3c450fe1f55 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-2747f9f9-837d-4ccd-ac61-c3c450fe1f55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:35.046972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:35.048177 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:35.049319 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:35.052555 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 314/605] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:35.054163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1da417-ef60-4fea-a7eb-0d24817a3615 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:35.054580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1da417-ef60-4fea-a7eb-0d24817a3615 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:35.054980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1da417-ef60-4fea-a7eb-0d24817a3615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:35.079551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fb1da417-ef60-4fea-a7eb-0d24817a3615 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:35.080795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fb1da417-ef60-4fea-a7eb-0d24817a3615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:35.108039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af804883-d098-4939-8e01-fd2e54fc0989 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:35.159078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:35.159725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:35.160010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:35.160291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/94ce9c64a0bb478485c47ebedd5a29f3/roles/b888b6f28fcb407795b56d37fdf7c6f6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:35.173611 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 315/606] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:35.174944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1da417-ef60-4fea-a7eb-0d24817a3615 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:35.179282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=94ce9c64a0bb478485c47ebedd5a29f3, role_id=b888b6f28fcb407795b56d37fdf7c6f6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:35.180383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:35.198154 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 292/607] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:34:34 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/94ce9c64a0bb478485c47ebedd5a29f3/roles/b888b6f28fcb407795b56d37fdf7c6f6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:35.239047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:35.293609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1da417-ef60-4fea-a7eb-0d24817a3615 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:35.332740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:35.333380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:35.333729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:35.333997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:35.334703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:35.335002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:35.337057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:35.351244 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 316/608] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:35.357550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:35.418043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:35.511824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb67ed1-7983-4931-84d5-3526b605ee9b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:35.549168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:35.550141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:35.550567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:35.550985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/a824d5df8638409bbaf4d6e8d22d63ca/roles/b888b6f28fcb407795b56d37fdf7c6f6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:35.566656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=a824d5df8638409bbaf4d6e8d22d63ca, role_id=b888b6f28fcb407795b56d37fdf7c6f6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:35.568485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:35.583810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:35.585810 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 293/609] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:34:35 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/a824d5df8638409bbaf4d6e8d22d63ca/roles/b888b6f28fcb407795b56d37fdf7c6f6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:35.717640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33796e19-f891-4c01-a075-6d38f8dd6e1f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:35.734331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:35.757444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:35.758150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:35.758632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:35.759200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:35.772285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:35.778063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:35.779335 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:35.780425 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:35.785493 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 317/610] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:35.788791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:35.895335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:35.925274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ec95-2828-4fe7-a7b7-6579faf512f7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:35.961811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:35.962658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:35.963147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:35.963586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:35.964398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:35.964849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:35.966440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:35.977431 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 294/611] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:35.982760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:36.090062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005456e7-a64e-4732-af39-bb38c145d69a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:36.139729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:36.140634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:36.141088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:36.141539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:36.159935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:36.161674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:36.178961 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:36.180237 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:36.181361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:36.181361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:36.181361 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:36.182879 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 318/612] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:36.189130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:36.294579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e07913-b1de-422b-acae-b471f3966f9e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:36.339971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:36.340942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:36.341368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:36.341808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:36.342608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:36.343081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:36.344628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:36.359615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:36.365319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 295/613] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:36.371484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:36.406875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:36.502397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebe20b-485b-4a92-8d53-32ae60cc4874 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:36.558390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:36.558941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:36.559214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:36.559513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:36.572848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:36.574343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:36.585388 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:36.586540 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:36.587722 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 319/614] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:36.592536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:36.730212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ac247-3777-477c-a892-9d948f7b67ea None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:36.773296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:36.774296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:36.774779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:36.775280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:36.776273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:36.776778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:36.778501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:36.801096 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 296/615] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 17 20:34:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:36.806447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:36.938403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:36.959942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ce0a4-c34a-466c-b315-9f3f5ed558b8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:37.011062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:37.011702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:37.011993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:37.012251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:37.023310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:37.024771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:37.043547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:37.044593 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:37.045550 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:37.045550 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:37.045550 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:37.046144 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 320/616] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:37.050539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:37.135264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:37.183376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d893d-8f99-48e5-b5c9-7abfaf7618db None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:37.225404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:37.226422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:37.226891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:37.227345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:37.228364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:37.228874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:37.230520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:37.243219 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 297/617] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:37.249018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:37.384915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8adc96-594e-4b6a-bc8f-b268560f1c6f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:37.429509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:37.430219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:37.430502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:37.430775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:37.441791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:37.443288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:37.445260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:37.458847 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:37.459802 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:37.460692 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:37.461606 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 321/618] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:37.466084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:37.570080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68835596-f76a-484d-a0a8-6a19f588565a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:37.584780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:37.609564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed713d-b942-48cb-b58e-72e9076cf593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:37.610182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed713d-b942-48cb-b58e-72e9076cf593 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:37.610457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed713d-b942-48cb-b58e-72e9076cf593 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:37.610728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed713d-b942-48cb-b58e-72e9076cf593 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:37.611423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed713d-b942-48cb-b58e-72e9076cf593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:37.611724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed713d-b942-48cb-b58e-72e9076cf593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:37.613168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed713d-b942-48cb-b58e-72e9076cf593 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:37.626549 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 298/619] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:37.634729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:37.705582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d241f90-006e-4997-96bb-2334a5074d2d None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:37.705869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d241f90-006e-4997-96bb-2334a5074d2d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:37.706141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d241f90-006e-4997-96bb-2334a5074d2d None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:37.707747 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 299/620] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:37.714199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b13ebf-b53f-4c28-97b1-c561fb7d41a8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:37.714480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b13ebf-b53f-4c28-97b1-c561fb7d41a8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:37.714748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b13ebf-b53f-4c28-97b1-c561fb7d41a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:37.745878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-64b13ebf-b53f-4c28-97b1-c561fb7d41a8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:37.747167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-64b13ebf-b53f-4c28-97b1-c561fb7d41a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:37.824260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6337c-85fa-403c-ab9c-eeed1caf5f54 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:37.870282 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 300/621] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:37.880887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993269e-e3e2-45c3-b304-770e9a56a7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:37.882656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993269e-e3e2-45c3-b304-770e9a56a7ae None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:37.882656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993269e-e3e2-45c3-b304-770e9a56a7ae None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:37.882656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993269e-e3e2-45c3-b304-770e9a56a7ae None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/8eea4641bfd940fe8a94ec79484ef4d3/roles/b888b6f28fcb407795b56d37fdf7c6f6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:37.903409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993269e-e3e2-45c3-b304-770e9a56a7ae None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=8eea4641bfd940fe8a94ec79484ef4d3, role_id=b888b6f28fcb407795b56d37fdf7c6f6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:37.904929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993269e-e3e2-45c3-b304-770e9a56a7ae None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:37.920269 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 322/622] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:34:37 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/8eea4641bfd940fe8a94ec79484ef4d3/roles/b888b6f28fcb407795b56d37fdf7c6f6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:38.311302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7674-42f2-40fc-ba9c-0709dbd488cc None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.311709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7674-42f2-40fc-ba9c-0709dbd488cc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.311897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c7674-42f2-40fc-ba9c-0709dbd488cc None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.313563 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 301/623] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:38.318689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2996a-6d3a-4e8e-9515-2ffa138c68fc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.318970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2996a-6d3a-4e8e-9515-2ffa138c68fc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.319238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2996a-6d3a-4e8e-9515-2ffa138c68fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.342346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-eff2996a-6d3a-4e8e-9515-2ffa138c68fc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:38.343542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eff2996a-6d3a-4e8e-9515-2ffa138c68fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:38.383535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c7674-42f2-40fc-ba9c-0709dbd488cc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:38.394396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:38.423671 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 323/624] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:38.516687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c7674-42f2-40fc-ba9c-0709dbd488cc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:38.519898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:38.557006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:38.558026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.558503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.558985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.566604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b94631-880a-470c-93a0-5471a37be582 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.567109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b94631-880a-470c-93a0-5471a37be582 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.567575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b94631-880a-470c-93a0-5471a37be582 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.569426 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 324/625] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:38.574506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843503a2-5d56-4381-88aa-0180366eb2d6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.574753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843503a2-5d56-4381-88aa-0180366eb2d6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.575060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843503a2-5d56-4381-88aa-0180366eb2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.575696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:38.580858 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:38.581862 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:38.582705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:38.585308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 302/626] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:38.595585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:38.621568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-843503a2-5d56-4381-88aa-0180366eb2d6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:38.622861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-843503a2-5d56-4381-88aa-0180366eb2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:38.731231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18d2d5-b521-466d-9a96-b7134d02e08d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:38.747910 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 325/627] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:38.776676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe08f3-d216-4557-b2c3-519318d3d66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:38.777788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe08f3-d216-4557-b2c3-519318d3d66b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.778307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe08f3-d216-4557-b2c3-519318d3d66b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.778771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe08f3-d216-4557-b2c3-519318d3d66b None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.779677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe08f3-d216-4557-b2c3-519318d3d66b None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:38.780280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe08f3-d216-4557-b2c3-519318d3d66b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:38.782309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe08f3-d216-4557-b2c3-519318d3d66b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:38.791120 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 303/628] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:38.797396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843503a2-5d56-4381-88aa-0180366eb2d6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:38.824327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1346d-da1f-4820-98ac-73a354c143a4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.824832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1346d-da1f-4820-98ac-73a354c143a4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.825340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1346d-da1f-4820-98ac-73a354c143a4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.827707 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 304/629] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:38.833751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1e1c7-3034-4ae7-90a3-93a50e90aae9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.834245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1e1c7-3034-4ae7-90a3-93a50e90aae9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.834691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1e1c7-3034-4ae7-90a3-93a50e90aae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:38.854105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a9b1e1c7-3034-4ae7-90a3-93a50e90aae9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:38.855536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9b1e1c7-3034-4ae7-90a3-93a50e90aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:38.877431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:38.920043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843503a2-5d56-4381-88aa-0180366eb2d6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:38.930554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:38.947173 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 305/630] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:38.947173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1e1c7-3034-4ae7-90a3-93a50e90aae9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:38.983396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:38.984207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:38.984580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:38.984991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.012608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.014879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:39.038443 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:39.039656 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:39.040788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:39.040788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:39.040788 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:39.045560 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 326/631] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:39.050128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.110953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1e1c7-3034-4ae7-90a3-93a50e90aae9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.157370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.158199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.158572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.158960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.171935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:39.178339 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:39.179468 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:39.180534 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:39.184641 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 306/632] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:39.186000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.191867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc889ac-e011-4495-921f-604f25576741 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.209022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:39.231006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.231795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.232176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.232578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.233334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:39.233742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.235233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.253501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 327/633] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:39.254607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.279961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:39.311723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7746f56-910a-4a0d-b6a8-ce8fc89b4727 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.352464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.353108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.353434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.353978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.367006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:39.372166 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:39.373123 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:39.378654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:39.379448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d80ce-a46a-4a39-9625-96116d2065f5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.379523 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 307/634] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:39.379523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.417392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.418167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.418896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.419175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.419850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:39.420150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.421947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.431392 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 328/635] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:39.436819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.461884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:39.487459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313a8f0-de1a-4b74-92e9-119556b75974 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.523748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.524390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.524639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.525091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.528875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:39.532524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.534024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:39.547004 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:39.547996 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 308/636] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:39.548959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.554403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2895e25-b37f-4c10-9836-874fae6981df None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.589811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.590459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.590897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.591176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.602734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:39.608150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:39.609112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:39.610024 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 329/637] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:39.611484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.657339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d771189-84aa-4fe0-a076-2b32da04c844 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.695605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.696222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.696501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.696735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.697435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:39.697770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.699606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.702979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:39.710036 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 309/638] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:39.710823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.730641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a2fff-0d45-42ed-ac1d-5a57f9c06dec None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.766088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.766828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.767218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.767526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.779289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.781342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.783087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:39.798486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:39.799523 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:39.801034 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:39.801034 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:39.801034 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:39.801034 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 330/639] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:39.801034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.821966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b4fd4-daea-4e59-b046-032f2a7eca1c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.858849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.859547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.859829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.860101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.860860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:39.861166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.862922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.882252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 310/640] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:39.883460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.920399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c3525-757b-4bd6-baa1-aad74b489165 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:39.947976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:39.957197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:39.957852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:39.958120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:39.958386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:39.959045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:39.959343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:39.960702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:39.973299 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 331/641] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:39.973299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:39.997098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed4d07-1bd6-453d-b728-d6a2d0129786 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.029678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.030234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.030369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.030576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.031203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.041378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.042890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:40.057432 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:40.058580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:40.059568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:40.059568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:40.059568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:40.059568 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 311/642] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:40.060303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.080968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390220b-2674-44f4-899b-e9a7de91a4a0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.116292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.117052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.117330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.117929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.118832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:40.119136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.120515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.138485 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 332/643] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:40.139755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.161364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d71647-879b-48fb-bd53-81b84364d813 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.174686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.206014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.206624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.207034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.207204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.272354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75758c-66d7-4216-af13-fdc93c41cc5c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.281283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.282806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.290615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.309923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.310551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.310823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.311085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.321286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.322755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:40.337395 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:40.338390 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:40.339454 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:40.339454 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:40.339454 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:40.339454 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 333/644] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:40.339454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.388994 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 312/645] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:40.390208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.401507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.452459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb4159-2b80-47e0-b209-cbc3d46d4d20 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.498329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.499041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.499324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.499591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.500228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:40.500528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.501937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.503774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7837bbf-323b-43dd-b89e-ad6e43b64b8a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.518118 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 334/646] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:40.518118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.542040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.559097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.560530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.561181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.561668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.571449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.573289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:40.587782 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:40.588955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:34:40.590013 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:40.593382 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 313/647] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:40.594224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.622568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b507f68-ceec-4f41-8d41-1833a84965b3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.657390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.658031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.658305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.658576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.659211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:40.659506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.660105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.660851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.675158 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 335/648] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:40.676003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.696655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff5076-a950-4ff3-b56a-9f5e18e5eafb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.729878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75f33b-65e2-4a88-8808-6b44dea545b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.730455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f33b-65e2-4a88-8808-6b44dea545b8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.730733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f33b-65e2-4a88-8808-6b44dea545b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.731004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75f33b-65e2-4a88-8808-6b44dea545b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.741225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75f33b-65e2-4a88-8808-6b44dea545b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.742735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75f33b-65e2-4a88-8808-6b44dea545b8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-cd75f33b-65e2-4a88-8808-6b44dea545b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:40.756658 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:40.757853 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:40.758757 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:40.758757 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:40.758757 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:40.758757 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 314/649] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:40.758994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8576c-8383-4fea-a89c-e4bb22eb2203 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.759358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8576c-8383-4fea-a89c-e4bb22eb2203 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.759453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8576c-8383-4fea-a89c-e4bb22eb2203 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.760821 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 315/650] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:40.761347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8576c-8383-4fea-a89c-e4bb22eb2203 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.770468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.783629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b95d2-df4d-4e35-bf68-d699ce8d8c33 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.817021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.817688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.817962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.818226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] PATH_INFO: `/v3/users/glance` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.825659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.827112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:40.838142 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:40.839125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 336/651] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:40.840072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.866693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8576c-8383-4fea-a89c-e4bb22eb2203 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.885832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:40.900207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.900816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.901093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.901416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.902182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:40.902535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.904107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.921107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 316/652] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:40.922005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:40.939399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57c8c1-6ba2-4ed8-a30b-722910d47a66 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:40.971459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff6b67-5750-47e3-ab06-d916d25f77ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:40.972089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff6b67-5750-47e3-ab06-d916d25f77ec None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.972376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff6b67-5750-47e3-ab06-d916d25f77ec None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.972641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff6b67-5750-47e3-ab06-d916d25f77ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:40.980560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff6b67-5750-47e3-ab06-d916d25f77ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:40.982025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff6b67-5750-47e3-ab06-d916d25f77ec None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-95ff6b67-5750-47e3-ab06-d916d25f77ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:40.992815 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:40.994025 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 337/653] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:40.995213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e735-6c90-4b8b-b88f-98e617293974 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:40.996030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e735-6c90-4b8b-b88f-98e617293974 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:40.996030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e735-6c90-4b8b-b88f-98e617293974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.012929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-06a1e735-6c90-4b8b-b88f-98e617293974 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:41.014255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-06a1e735-6c90-4b8b-b88f-98e617293974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:41.016178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.022768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab3c13-c8b6-43eb-88bc-26d373ac03a6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.057104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.057734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.058015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.058274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.058918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:41.059217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.060656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.071762 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 317/654] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:41.072629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.083583 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 338/655] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:41.084576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e735-6c90-4b8b-b88f-98e617293974 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.112058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.175871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c03cf-9749-4857-9ad7-b7f64a0a4812 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.189002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e735-6c90-4b8b-b88f-98e617293974 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.210623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.211379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.211734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.212076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.212869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:41.213240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.214806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.225715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.226355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.226641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.226904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.232884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 318/656] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:41.236068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.237937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.305452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.307207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.338424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.346652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60cdf0-ee99-46e9-94d2-f11b8e9dfb52 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.384941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.385585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.385866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.386135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.386861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:41.387291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.388629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.410718 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 319/657] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:41.410718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.410959 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 339/658] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:41.411525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.449136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.503617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4cd0d-8d8e-4965-9cb5-4f16bb08195b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.511131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492dc764-ad7e-4aad-8fa3-e9365794454d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.538129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.538763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.539037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.539299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.543985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.544595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.544870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.545137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.545744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.547187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:41.558138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:41.559875 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 320/659] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:41.561095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.577423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.615356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.616814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.654998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.660417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512887d-0812-4ac5-a9d6-c01c2319c407 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.695687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.696322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.696603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.696875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.707321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:41.713373 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:41.714545 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:41.715412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 321/660] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:41.716158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.716226 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 340/661] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:41.716786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.790331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.812156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269d6f1-3a9e-4996-b61e-f6d8223d0093 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.816274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8185c6-0447-445d-8af2-dcf47c8947eb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.845285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.845958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.846234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.846503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.850643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:41.851348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:41.851633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:41.851904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:41.853641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:41.858412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.859867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:41.874138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:41.875354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:41.876267 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:41.876267 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:41.876267 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:41.876267 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 341/662] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:41.876267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:41.920785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:41.922286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:41.974428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79331482-c865-444f-b9dc-bb00253f20d4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:41.991803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.007955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.008556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.008828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.009080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.009794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:42.010097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.011529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.017448 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 322/663] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:42.018356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.028518 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 342/664] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:42.028518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.072703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.118091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1afff-17a8-4ecf-a920-29eb59e2c3bd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.123162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01d10a-34b8-4540-82b4-7b18d47ba315 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.150896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.151506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.151784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.152055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.152720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:42.153057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.154504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.156418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.157046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.157319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.157625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.162571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 323/665] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:42.163354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.171220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.173080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:42.187824 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:42.188902 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:42.190031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:42.190031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:42.190031 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:42.191193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 343/666] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:42.191998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.199304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.263123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bdd1a-b465-47e9-b8da-753e6714cf4d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.291265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b9a6f-b727-4619-978e-07007336b20b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.292769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.298128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.298763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.299038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.299305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.299942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:42.300240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.301656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.313616 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 324/667] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:42.314401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.330338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.330911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.331182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.331445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.341652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.343139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:42.357927 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:42.359058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:42.360083 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:42.360083 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:42.360083 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:42.360083 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 344/668] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:42.360358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.408981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.416021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6a721-4280-4704-a19c-1240703b60c1 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.450014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.450614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.450887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.451141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.451857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=724ce8c58c1b473e9e3af18009532c9f) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:42.452159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.453632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.456521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25f81e-999a-400f-b94c-8f39837c1394 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.459167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 325/669] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/role_assignments?user.id=724ce8c58c1b473e9e3af18009532c9f&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:42.459931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.508565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.509585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.510026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.510305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.522617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.524094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:42.540563 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:42.541667 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:42.542771 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:42.542771 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:42.542771 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:42.547033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 345/670] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:42.547995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.557854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.577578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999ee4-f0aa-4ffd-939d-f38ade4e3051 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.616537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.617525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.618074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.618521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.619414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:42.619905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.621500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.639429 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 326/671] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:42.640657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.657260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6e269-b314-4065-bd72-69e974a06c5d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.664643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.703313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.703980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.704260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.704951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.712543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.714676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:42.729613 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:42.730547 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:42.731539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 346/672] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:42.732039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.767704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555696c9-272c-464c-849e-4f915bd8f917 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.802509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.803111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.803380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.803642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.804422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.805340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:42.806089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.807768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.820971 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 327/673] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:42.821758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.846015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b21350-6da6-4f4b-abd5-bd3bd94cc351 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.880477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.881000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.881270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.881569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.882224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:42.882520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.883916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.896575 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 347/674] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:42.897239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:42.908089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:42.929944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea7b5b-bd21-4447-8c58-204be811264c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:42.968817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:42.970045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:42.970326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:42.970859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:42.978706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:42.980156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:42.991201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:42.992100 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 328/675] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:42.993029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.008741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ece3-8bbc-4b9b-99d9-2e734fd38aff None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.036348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:43.045652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.046265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.046536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.046799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.047480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:43.047780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.049241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.061511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 348/676] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:43.062321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.099991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381c79e-a8ae-4a35-b27f-5e020cbed93d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.136642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.137200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.137542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.137818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.144404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.145923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.148807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:43.159705 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:43.160606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:43.161603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:43.165672 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 329/677] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:43.166486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.178492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae173c-c110-4d19-b0f9-bff226c3c98a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.217369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.218177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.218463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.218714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.227936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.230024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:43.242343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:43.243327 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:43.244289 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:43.248410 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 349/678] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:43.249286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.273152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e21f2f-6e06-4930-ba05-141dcee853be None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.301402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:43.309743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.310478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.310849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.311216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.312020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:43.312326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.313826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.330918 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 330/679] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:43.330918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.364552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a424-0cd4-4df3-8885-020f87c37d9e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.388740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:43.399833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.400517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.400848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.401215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.401959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=d45ab41902864586811c81948540fdac) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:43.402260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.403714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.410005 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 350/680] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/role_assignments?user.id=d45ab41902864586811c81948540fdac&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:43.423362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.432967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da567b-bbe4-4cf9-9a6a-8193cc21fbd1 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.473511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.474193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.474525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.474793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.475508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:43.475827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.477394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.493615 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 331/681] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:43.494721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.572366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f529c-5d84-424d-a587-633e1f5d02d1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.585884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:43.615304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.615979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.616298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.616572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.617331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:43.617677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.619157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.626701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e30be-8924-4f53-9bf3-5b9362f38dfc None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.639942 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 351/682] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:43.640507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.656762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:43.676118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.676786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.677072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.677346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.678127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=94ce9c64a0bb478485c47ebedd5a29f3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:43.678437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.680703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.687450 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 332/683] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/role_assignments?user.id=94ce9c64a0bb478485c47ebedd5a29f3&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:43.693277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:43.763647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224efdf-5780-468e-9d78-d20dd619ad04 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.802832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.803464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.803740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.804009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.804849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=a824d5df8638409bbaf4d6e8d22d63ca) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:43.805161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.806737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:43.817793 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 352/684] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/role_assignments?user.id=a824d5df8638409bbaf4d6e8d22d63ca&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:43.830032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7953e-6ab6-4c36-ae4b-507b262b8ecf None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:43.873811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:43.874824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:43.875297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:43.875763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:43.950666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:43.972242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:43.973790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:44.094814 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 333/685] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:44.100699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:44.135146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:44.211731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e65c-b1d8-44fc-8ea1-3798f7fe0388 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:44.252534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:44.253177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:44.253452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:44.253752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:44.389546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:44.391026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:44.406229 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:44.407228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:44.408343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:44.408343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:44.408343 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:44.408343 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 353/686] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:44.414743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:44.538023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:44.556257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172121f-365a-484b-ba1d-6adfa4de27c3 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:44.606262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:44.606880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:44.607154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:44.607421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:44.608069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:44.608372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:44.609785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:44.624937 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 334/687] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:44.630476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:44.770970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:44.790369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380554a-b2e0-4320-9563-2bd35c350bc7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:44.835010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:44.835647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:44.835921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:44.836189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:44.843046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:44.844497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:44.855599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:44.856600 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:44.857895 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 354/688] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:44.864430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:45.009235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f6b38-6d8d-44ac-9d3c-c26e0cb51fb6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:45.055591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa55432-a21c-44ec-ba7c-679367fd4e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:45.056131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55432-a21c-44ec-ba7c-679367fd4e75 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:45.056423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55432-a21c-44ec-ba7c-679367fd4e75 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:45.056698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55432-a21c-44ec-ba7c-679367fd4e75 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:45.057396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa55432-a21c-44ec-ba7c-679367fd4e75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:45.057728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa55432-a21c-44ec-ba7c-679367fd4e75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:45.059177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa55432-a21c-44ec-ba7c-679367fd4e75 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:45.076530 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 335/689] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:45.082586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:45.183868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef45cff-fda6-4a28-a269-143ecb5d77aa None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:45.184203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef45cff-fda6-4a28-a269-143ecb5d77aa None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:45.184489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef45cff-fda6-4a28-a269-143ecb5d77aa None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:45.186128 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 336/690] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:45.192828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600b50a-534c-4056-aa10-c793ebd30cf2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:45.193107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600b50a-534c-4056-aa10-c793ebd30cf2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:45.193374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600b50a-534c-4056-aa10-c793ebd30cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:45.209806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:45.227222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8600b50a-534c-4056-aa10-c793ebd30cf2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:45.229505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8600b50a-534c-4056-aa10-c793ebd30cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:45.251202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8301-c4b2-4185-a610-0c6e3c0f6513 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:45.315145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:45.315682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:45.315952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:45.316220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:45.316946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=8eea4641bfd940fe8a94ec79484ef4d3) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:45.317247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:45.318717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:45.330082 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 355/691] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:45 2022] GET /identity/v3/role_assignments?user.id=8eea4641bfd940fe8a94ec79484ef4d3&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:45.348492 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 337/692] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:45.783629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:45.795575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:45.892376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20a950-d60a-4211-8887-70f0460958c6 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:45.926895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:45.927603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:45.927878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:45.928146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:45.938343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:45.943555 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:45.944447 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:45.945304 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 356/693] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:45.952143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600b50a-534c-4056-aa10-c793ebd30cf2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:46.058189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600b50a-534c-4056-aa10-c793ebd30cf2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:46.092871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:46.093518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:46.093801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:46.094067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:46.094746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:46.095043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:46.096460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:46.098145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:46.109946 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 338/694] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:46.113046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:46.202860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:46.245631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7582d86-a52a-4353-a8e9-20e29526acc8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:46.280644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:46.281653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:46.281933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:46.282261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:46.297373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:46.298928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:46.319995 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:46.321023 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:46.321984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:46.321984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:46.321984 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:46.321984 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 357/695] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:46.328582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:46.481525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096835c6-48b4-4793-958c-624dc2cf9ff4 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:46.529748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9374d-8612-4f5a-9ca8-045cdb76436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:46.531152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9374d-8612-4f5a-9ca8-045cdb76436f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:46.531622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9374d-8612-4f5a-9ca8-045cdb76436f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:46.532080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9374d-8612-4f5a-9ca8-045cdb76436f None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:46.532946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9374d-8612-4f5a-9ca8-045cdb76436f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:46.533446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9374d-8612-4f5a-9ca8-045cdb76436f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:46.535126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9374d-8612-4f5a-9ca8-045cdb76436f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:46.554005 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 339/696] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:46.559632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:46.672171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd7695-02fe-4bf8-8d52-e263159624cc None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:46.672505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd7695-02fe-4bf8-8d52-e263159624cc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:46.672720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd7695-02fe-4bf8-8d52-e263159624cc None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:46.674104 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 340/697] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:46.682055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea4e30-a246-41b3-976d-51e08f2be1b0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:46.682334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea4e30-a246-41b3-976d-51e08f2be1b0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:46.682608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea4e30-a246-41b3-976d-51e08f2be1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:46.716581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-42ea4e30-a246-41b3-976d-51e08f2be1b0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:46.717802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-42ea4e30-a246-41b3-976d-51e08f2be1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:46.734309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71f9ac-2f09-4bdd-bed7-6c9451fd15da None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:46.765662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:46.794839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:46.795479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:46.795756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:46.796027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:46.808849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:46.812230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:46.831199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:46.832445 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:46.833641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:46.834221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:46.837106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 341/699] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:46.837221 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 358/699] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:34:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:46.841529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:46.980386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a726a19-9d26-4e98-b901-efaab5bbab7f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:47.026202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b5815-4617-44f3-bbd0-0384e50c6f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:47.026725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b5815-4617-44f3-bbd0-0384e50c6f2c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.026999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b5815-4617-44f3-bbd0-0384e50c6f2c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.027267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b5815-4617-44f3-bbd0-0384e50c6f2c None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.027983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b5815-4617-44f3-bbd0-0384e50c6f2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:47.028320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b5815-4617-44f3-bbd0-0384e50c6f2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:47.029767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b5815-4617-44f3-bbd0-0384e50c6f2c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:47.039033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e6207-530f-4531-ae7d-a4d4ce15614c None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.039555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e6207-530f-4531-ae7d-a4d4ce15614c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.039555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e6207-530f-4531-ae7d-a4d4ce15614c None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.043785 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 342/700] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:47.044965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f85dbe-57ab-4b01-af27-3e51ad8acebc None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.045175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f85dbe-57ab-4b01-af27-3e51ad8acebc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.045454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f85dbe-57ab-4b01-af27-3e51ad8acebc None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.047098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 343/701] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:47.048112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13477af-84c5-40c5-9e73-70547fee03b9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.048386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13477af-84c5-40c5-9e73-70547fee03b9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.048655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13477af-84c5-40c5-9e73-70547fee03b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.052077 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 359/702] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:34:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:47.053511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55472122-3d9e-4928-8283-65f1ad81409e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.053796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55472122-3d9e-4928-8283-65f1ad81409e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.054065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55472122-3d9e-4928-8283-65f1ad81409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.067256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b13477af-84c5-40c5-9e73-70547fee03b9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:47.068518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b13477af-84c5-40c5-9e73-70547fee03b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:47.072062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-55472122-3d9e-4928-8283-65f1ad81409e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:47.073250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-55472122-3d9e-4928-8283-65f1ad81409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:47.121881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:47.145871 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 360/703] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:47.147035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55472122-3d9e-4928-8283-65f1ad81409e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.151860 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 344/704] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:47.189001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13477af-84c5-40c5-9e73-70547fee03b9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.323611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55472122-3d9e-4928-8283-65f1ad81409e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:47.341221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13477af-84c5-40c5-9e73-70547fee03b9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:47.358283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:47.379179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:47.379819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.380095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.380364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.380960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:47.382375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:47.392938 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:47.394045 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:47.394928 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 345/705] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:47.400078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.410735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:47.411369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.411647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.411913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.461814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:47.502864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:47.504656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:47.514433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a797f6-a580-43e9-aaed-f433c55a88eb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:47.551164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:47.552155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.552433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.553098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.553795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:47.554094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:47.555493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:47.560888 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 346/706] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:47.561719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.614316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:47.631836 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 361/707] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:47.637294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.672544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615aeb4-b951-4fbe-a285-13a2cdff017d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:47.711003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:47.711648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.711979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.712256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.713775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:47.723508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:47.724995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:47.740179 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:47.741321 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:47.742295 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:47.742295 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:47.742295 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:47.742295 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 347/708] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:47.742531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.756185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfaabf-e145-432d-ae64-c9e0afb5a1c9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:47.793213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:47.794198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.794620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.795059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.795937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:47.796448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:47.798159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:47.804362 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 362/709] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:47.805424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.854603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:47.863644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d1fbc-af80-4185-b781-257c298e3adf None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:47.908773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:47.909523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:47.909871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:47.910205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] PATH_INFO: `/v3/services/network` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:47.911013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:47.912605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:47.923153 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:47.924458 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:34:47.925536 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:47.930026 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 348/710] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:47.931117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:47.956152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d771e-63cb-4966-92b2-10a4b6597964 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.002631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:48.010851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.012070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.012625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.013053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.028245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.031157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.049669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3548e2-e73d-403f-aaef-fcf7d5a01780 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:48.053148 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:48.054360 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:48.055397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:48.055397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:48.055397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:48.057203 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 363/711] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:48.058238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:48.091830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.092444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.092725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.092992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.093668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:48.093969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.096362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.108632 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 349/712] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:48.109556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:48.131266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:48.196025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b6b51-d04a-4122-b92e-7b09f09b28e4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.219269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31187d28-374d-4f8a-a426-8e8638c8d10a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.240295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156699-f384-4929-a31d-480a6484d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.241644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156699-f384-4929-a31d-480a6484d704 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.242139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156699-f384-4929-a31d-480a6484d704 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.242591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156699-f384-4929-a31d-480a6484d704 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.243526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24156699-f384-4929-a31d-480a6484d704 None admin] RBAC: Adding query filter params (name=network) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:48.244063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24156699-f384-4929-a31d-480a6484d704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.245692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24156699-f384-4929-a31d-480a6484d704 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.256196 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 364/713] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:34:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:48.257536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156699-f384-4929-a31d-480a6484d704 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:48.271111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.271512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.271844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.272114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.272813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:48.273114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.274627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.292084 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 350/714] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:48.292084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:48.321336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:48.378858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156699-f384-4929-a31d-480a6484d704 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.407142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:48.418065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.419139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.419419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.419690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.420393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.422177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.432977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7d256-6cce-4adf-b035-7d0f9b83442b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.448944 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 365/715] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:34:48 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:48.449848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:48.475344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.476462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.476943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.477409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.478351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] RBAC: Adding query filter params (type=network) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:48.478876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.480777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.488180 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 351/716] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:34:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:48.567002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec31eb-53d0-4770-88ee-11b40a0823ce None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.569733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:48.608234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6b42e-ce98-474f-9afb-7f20ed7c8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.608886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6b42e-ce98-474f-9afb-7f20ed7c8f59 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.609170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6b42e-ce98-474f-9afb-7f20ed7c8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.609425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6b42e-ce98-474f-9afb-7f20ed7c8f59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.617507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6b42e-ce98-474f-9afb-7f20ed7c8f59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.619001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6b42e-ce98-474f-9afb-7f20ed7c8f59 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-2aa6b42e-ce98-474f-9afb-7f20ed7c8f59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:48.631724 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:48.632746 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:48.634049 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 366/717] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:48.641434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:48.776139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd6f8-4a2f-4e2f-b5ea-3efa1a5bea42 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.781048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbf021-2c8f-4099-8cbf-f53ab6aa290b None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.781587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbf021-2c8f-4099-8cbf-f53ab6aa290b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.782007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbf021-2c8f-4099-8cbf-f53ab6aa290b None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.783535 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 367/718] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:48.789366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81489c93-c817-48a2-8bf2-1ee9e37dc28d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.789973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81489c93-c817-48a2-8bf2-1ee9e37dc28d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.790461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81489c93-c817-48a2-8bf2-1ee9e37dc28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.812122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-81489c93-c817-48a2-8bf2-1ee9e37dc28d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:48.814265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-81489c93-c817-48a2-8bf2-1ee9e37dc28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:48.818762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.819399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.819672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.819936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:48.820644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:48.820941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:48.822473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:48.837957 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 352/719] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:48.844865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:48.849125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:48.907516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 368/720] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:48.961256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e570f-c60b-425e-b383-1e0cacedd154 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:48.998304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:48.998908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:48.999183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:48.999447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:49.000357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=724ce8c58c1b473e9e3af18009532c9f) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:49.000658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:49.002147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:49.007941 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 353/721] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:48 2022] GET /identity/v3/role_assignments?user.id=724ce8c58c1b473e9e3af18009532c9f&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:49.252144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81489c93-c817-48a2-8bf2-1ee9e37dc28d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:49.354142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:49.360997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81489c93-c817-48a2-8bf2-1ee9e37dc28d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:49.404923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:49.405584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:49.406195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:49.406459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:49.421912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:49.426685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:49.427687 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:49.428559 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 369/722] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:49.433996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:49.657812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:49.708322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459e8bf-6f85-4439-8eb9-3fecfe8e1a2c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:49.757280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:49.757913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:49.758227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:49.758443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:49.759113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:49.759404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:49.760772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:49.769911 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 354/723] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:49.775551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:49.911607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363997-1700-4374-bcb3-d67da72b97ac None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:49.954812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:49.955796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:49.956291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:49.956749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:49.977045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:49.979425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:49.996058 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:49.997294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:49.998228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:49.998228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:49.998228 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:50.000314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 370/724] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:50.008355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:50.160435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c801d46-c528-43ec-90d1-67fbe7121ffb None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:50.202339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:50.212961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:50.213856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:50.214331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:50.214823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:50.215516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:50.215818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:50.217195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:50.232650 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 355/725] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:50.238053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:50.252586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:50.385359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6c6a6-fe51-4326-8ac6-ee045986a7d7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:50.427817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:50.428465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:50.428726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:50.428996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:50.439253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:50.440698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:50.451964 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:50.452981 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:50.453996 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 371/726] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:50.458780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:50.601932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a5576-5443-49d0-9966-49aeaa5f2402 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:50.645028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15adc4f-2b33-4107-aec6-3e2405cc15e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:50.645662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15adc4f-2b33-4107-aec6-3e2405cc15e4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:50.645938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15adc4f-2b33-4107-aec6-3e2405cc15e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:50.646201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15adc4f-2b33-4107-aec6-3e2405cc15e4 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:50.646937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15adc4f-2b33-4107-aec6-3e2405cc15e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:50.647235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15adc4f-2b33-4107-aec6-3e2405cc15e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:50.648662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15adc4f-2b33-4107-aec6-3e2405cc15e4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:50.674579 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 356/727] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 17 20:34:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:50.682456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:50.784251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:50.832100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfd2c1-931a-434b-8832-dd3573ff5892 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:50.870908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074762-f4e5-4f37-90ea-821c5f3351fe None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:50.871241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074762-f4e5-4f37-90ea-821c5f3351fe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:50.871539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074762-f4e5-4f37-90ea-821c5f3351fe None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:50.873067 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 357/728] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:50.878959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7acc9a-40f9-409c-bd56-5caf9a2110f9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:50.879242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7acc9a-40f9-409c-bd56-5caf9a2110f9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:50.879512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7acc9a-40f9-409c-bd56-5caf9a2110f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:50.882536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:50.883449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:50.883968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:50.884438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:50.908244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-da7acc9a-40f9-409c-bd56-5caf9a2110f9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:50.911328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-da7acc9a-40f9-409c-bd56-5caf9a2110f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:50.983669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:50.985138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:51.007693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:51.053108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:51.087685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 358/729] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:51.089387 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 372/730] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:34:51.098507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7acc9a-40f9-409c-bd56-5caf9a2110f9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:51.236056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7acc9a-40f9-409c-bd56-5caf9a2110f9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:51.275330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:51.276033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.276304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.276570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.293032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:51.294601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:51.313736 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:51.314753 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:51.316273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:51.316273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:51.316273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:51.316273 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 359/731] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:51.320512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:51.435029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ae8be-6a84-43ea-a50b-e690e8f11cf3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:51.455023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:51.471295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:51.471911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.472184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.472449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.473093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:51.473390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:51.474804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:51.500959 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 373/732] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:51.506426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:51.513862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:51.563134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:51.591792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70343708-02cf-4eb6-96f5-8b109019c593 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:51.625842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c6375-d597-45e1-8083-ee540b032fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:51.626447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c6375-d597-45e1-8083-ee540b032fdd None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.626717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c6375-d597-45e1-8083-ee540b032fdd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.626977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c6375-d597-45e1-8083-ee540b032fdd None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.627788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c6375-d597-45e1-8083-ee540b032fdd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:51.629161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c6375-d597-45e1-8083-ee540b032fdd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:51.652552 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 360/733] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:34:51 2022] POST /identity/v3/services => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:51.652552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24537598-affb-4bec-a368-89029ddc06c3 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.652552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24537598-affb-4bec-a368-89029ddc06c3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.652552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24537598-affb-4bec-a368-89029ddc06c3 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.652552 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 361/734] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:51.661347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2ce00-2bfd-4ee1-841c-c0c83de0fc2d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:51.671861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875a90-a834-42c0-9415-730a85c8e275 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.672188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875a90-a834-42c0-9415-730a85c8e275 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.672402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875a90-a834-42c0-9415-730a85c8e275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.709803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-24875a90-a834-42c0-9415-730a85c8e275 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:51.715590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-24875a90-a834-42c0-9415-730a85c8e275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:51.738201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:51.740394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.741091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.742303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.756099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:51.758954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:51.780313 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:51.781713 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:51.782709 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:51.783676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 374/735] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:51.790720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:51.842004 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 362/736] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:34:51.843256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4480d-84b5-40f3-9926-72a553d3073b None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.843538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4480d-84b5-40f3-9926-72a553d3073b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.843806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4480d-84b5-40f3-9926-72a553d3073b None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.849528 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 363/737] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:51.851124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946e4fb-2322-4248-9d17-b5c33a29c083 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:51.851478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946e4fb-2322-4248-9d17-b5c33a29c083 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:51.851814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946e4fb-2322-4248-9d17-b5c33a29c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:51.878972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0946e4fb-2322-4248-9d17-b5c33a29c083 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:51.880214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0946e4fb-2322-4248-9d17-b5c33a29c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:51.925864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:51.951150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:51.982460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28658d64-060d-4343-9c3a-3166f676beed None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:52.001512 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 364/738] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:52.041812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:52.042854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:52.043324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:52.043819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:52.045128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:52.045894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:52.048364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:52.065325 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 375/739] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:52.074075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946e4fb-2322-4248-9d17-b5c33a29c083 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:52.178167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946e4fb-2322-4248-9d17-b5c33a29c083 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:52.212356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474cefa-cb44-4519-8fb8-5c24f65b412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:52.213326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474cefa-cb44-4519-8fb8-5c24f65b412f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:52.213831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474cefa-cb44-4519-8fb8-5c24f65b412f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:52.214290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474cefa-cb44-4519-8fb8-5c24f65b412f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:52.215201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474cefa-cb44-4519-8fb8-5c24f65b412f None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=8eea4641bfd940fe8a94ec79484ef4d3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:52.215790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474cefa-cb44-4519-8fb8-5c24f65b412f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:52.217416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474cefa-cb44-4519-8fb8-5c24f65b412f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:52.223600 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 365/740] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:52 2022] GET /identity/v3/role_assignments?user.id=8eea4641bfd940fe8a94ec79484ef4d3&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:52.235738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:52.275307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c94a8-785b-4dea-b403-39d76f77f22d None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:52.276487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c94a8-785b-4dea-b403-39d76f77f22d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:52.277263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c94a8-785b-4dea-b403-39d76f77f22d None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:52.280156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 366/741] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:34:52.286507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0af63-cc32-4c2a-8151-631914c1a12c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:52.287108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0af63-cc32-4c2a-8151-631914c1a12c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:52.287869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0af63-cc32-4c2a-8151-631914c1a12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:52.314003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c5d0af63-cc32-4c2a-8151-631914c1a12c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:52.315886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c5d0af63-cc32-4c2a-8151-631914c1a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:52.333890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:52.419229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9415e-05b8-4a48-9c8c-2b8ea4845685 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:52.434970 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 367/742] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:34:52.436146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0af63-cc32-4c2a-8151-631914c1a12c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:52.459311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:52.460507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:52.460895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:52.461259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:52.465500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:52.465500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:52.484582 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 376/743] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:34:52 2022] POST /identity/v3/services => generated 240 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:34:52.597047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0af63-cc32-4c2a-8151-631914c1a12c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:52.655388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:52.656657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:52.657106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:52.657552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:52.665818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:52.670013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:52.675414 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:52.676326 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:52.677170 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 368/744] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:52.684551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:52.688239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:52.815103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff5dc4-05f6-416d-a1e1-150f2f095da3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:52.851405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:52.852029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:52.852306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:52.852575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:52.853249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:52.853577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:52.854969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:52.865562 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 377/745] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:52.865832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:52.871082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:52.944487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:52.990309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059e36f-de61-4782-9464-203e91431f63 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.033601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab87a-19b5-48ad-91b8-3af8af2e5ff9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.062038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.063150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.063559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.063823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.070049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.070777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.071042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.071306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.081911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:53.086685 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:53.087622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:53.088464 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:53.089109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:53.089181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 369/746] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:53.089764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:53.094087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:53.117337 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:53.119584 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:53.121857 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:53.121857 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:53.121857 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:53.121857 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 378/747] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:53.127737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:53.221482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:53.297615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce9676-9d36-439c-87d0-3e2fa1ad97af None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.311435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caced89a-7808-45cf-88b9-57b24a58f128 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.315146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:53.335263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.336247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.336715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.337210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.337981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:53.338371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:53.340091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:53.357584 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 379/748] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:53.365483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.365632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:53.365732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.365732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.365732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.365732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:53.367835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:53.369239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:53.386351 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 370/749] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:53.390735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:53.496189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac6344-0cb5-4b4a-9577-85d57e148575 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.549547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.551718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.551992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.552260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.565962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:53.567416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:53.586820 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:53.587818 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:53.589157 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 380/750] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:53.592665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d3ee6-268b-423d-bcf5-00d104ba644c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.597603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:53.627176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:53.650114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.651113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.651388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.651651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.670069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:53.671542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:53.677971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:53.708240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:53.709247 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:53.710226 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:53.710226 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:53.710226 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:53.711347 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 371/751] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:53.715505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:53.803456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d2ad6-db6b-4d64-af8e-831be6055338 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.857545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36db93-6ba3-43b4-94dc-c59416d0b166 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:53.872609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.873563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.874068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.874538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.875474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:53.875979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:53.877652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:53.914730 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 381/752] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:53.920481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:53.920950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:53.921639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:53.921917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:53.922189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:53.931649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:53.931956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:53.933387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:53.960584 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 372/753] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:53.965811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:54.017889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:54.051139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:54.058708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbea726-f8b4-446f-8412-414a1c54a8d0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:54.091233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120050e-0a6d-48f0-a4a1-ed7d0bdfe248 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:54.098750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:54.099612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.100483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.100904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.129109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:54.130178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.130722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.131268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] PATH_INFO: `/v3/users/nova` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.139457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:54.141252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:54.154328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:54.155306 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:34:54.156655 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:54.159337 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 373/754] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:54.166593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:54.173364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:54.175674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:54.357783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:54.366705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:54.374167 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 382/755] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:34:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 20:34:54.379798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:54.384476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc7209-947e-4231-926a-00f5c835673b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:54.444240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:54.446638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.447135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.447637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.449672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:54.454049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:54.455750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:54.488317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 374/756] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:34:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:54.493448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:54.529771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921f659-16e1-48ea-93a5-00a7714c21e9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:54.573678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:54.574969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.575452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.575939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.602806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:54.605022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:54.637124 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:54.638354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:54.639331 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:54.639331 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:54.639331 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:54.642765 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 383/757] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:54.643689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:54.715061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3485a0-7176-40f6-853f-f67c8786a579 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:54.741872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:54.787982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04989a03-feab-493f-9f7f-977ad254f00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:54.788641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989a03-feab-493f-9f7f-977ad254f00e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.788919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989a03-feab-493f-9f7f-977ad254f00e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.793527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989a03-feab-493f-9f7f-977ad254f00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.801342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04989a03-feab-493f-9f7f-977ad254f00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:54.803066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04989a03-feab-493f-9f7f-977ad254f00e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:54.813147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a112c-e47b-4a06-baba-9515a298ac3a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-04989a03-feab-493f-9f7f-977ad254f00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:54.819541 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:54.820528 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:54.821451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:54.821451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:54.821451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:54.825728 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 375/758] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:54.827189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa96a6-1bb1-415b-bcab-7d00bdc48ccf None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.827484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa96a6-1bb1-415b-bcab-7d00bdc48ccf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.828044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa96a6-1bb1-415b-bcab-7d00bdc48ccf None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.830069 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 376/759] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:54.830806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa96a6-1bb1-415b-bcab-7d00bdc48ccf None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:54.839902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:54.852281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce7fa8-23a4-4629-b9fb-9b7456752fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:54.853319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce7fa8-23a4-4629-b9fb-9b7456752fed None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.854154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce7fa8-23a4-4629-b9fb-9b7456752fed None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.854593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce7fa8-23a4-4629-b9fb-9b7456752fed None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.855695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce7fa8-23a4-4629-b9fb-9b7456752fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:54.856360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce7fa8-23a4-4629-b9fb-9b7456752fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:54.858228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce7fa8-23a4-4629-b9fb-9b7456752fed None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:54.873796 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 384/760] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:54.875258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6a9d6-9ffc-43bd-bd2a-70979ee49fe6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.875691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6a9d6-9ffc-43bd-bd2a-70979ee49fe6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.876110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6a9d6-9ffc-43bd-bd2a-70979ee49fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.897941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-80e6a9d6-9ffc-43bd-bd2a-70979ee49fe6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:54.899801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-80e6a9d6-9ffc-43bd-bd2a-70979ee49fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:54.958877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa96a6-1bb1-415b-bcab-7d00bdc48ccf None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:54.997529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:54.998312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:54.998609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:54.998864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:54.999522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:54.999826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.000730 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 385/761] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:34:55.001763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6a9d6-9ffc-43bd-bd2a-70979ee49fe6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.009567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.032676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 377/762] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:55.039038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.042194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:55.115763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6a9d6-9ffc-43bd-bd2a-70979ee49fe6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.154526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec8bc0-dd5f-42ca-ae59-e519fcf1854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.155089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec8bc0-dd5f-42ca-ae59-e519fcf1854d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.155363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec8bc0-dd5f-42ca-ae59-e519fcf1854d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.155632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec8bc0-dd5f-42ca-ae59-e519fcf1854d None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.166572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec8bc0-dd5f-42ca-ae59-e519fcf1854d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.168765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec8bc0-dd5f-42ca-ae59-e519fcf1854d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.171431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-81ec8bc0-dd5f-42ca-ae59-e519fcf1854d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:55.185602 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:55.186569 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:55.187479 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:55.189772 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 386/763] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:55.191838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26057-9434-4579-adf3-2af364f71ad8 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.192326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26057-9434-4579-adf3-2af364f71ad8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.192799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26057-9434-4579-adf3-2af364f71ad8 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.195381 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 387/764] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:55.196403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26057-9434-4579-adf3-2af364f71ad8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.221420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07d1b6-4e8d-4b4c-83a4-49bf8bfa8fc0 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.258918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a6829-a7df-4cd3-ac0e-f8c68810e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.259616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6829-a7df-4cd3-ac0e-f8c68810e256 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.259889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6829-a7df-4cd3-ac0e-f8c68810e256 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.260200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6829-a7df-4cd3-ac0e-f8c68810e256 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.268074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a6829-a7df-4cd3-ac0e-f8c68810e256 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.269793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a6829-a7df-4cd3-ac0e-f8c68810e256 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-b58a6829-a7df-4cd3-ac0e-f8c68810e256 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:55.282670 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:55.283606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:55.284529 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 378/765] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:55.285511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095bfee-6556-43ff-88fb-742be93c24ab None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.285685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095bfee-6556-43ff-88fb-742be93c24ab None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.285954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095bfee-6556-43ff-88fb-742be93c24ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.305252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0095bfee-6556-43ff-88fb-742be93c24ab None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:55.306473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0095bfee-6556-43ff-88fb-742be93c24ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:55.321850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:55.331156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26057-9434-4579-adf3-2af364f71ad8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.379746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2e383-2bae-407b-a610-86912176dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.380686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2e383-2bae-407b-a610-86912176dc55 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.381134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2e383-2bae-407b-a610-86912176dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.381616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2e383-2bae-407b-a610-86912176dc55 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.382468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2e383-2bae-407b-a610-86912176dc55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:55.382930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2e383-2bae-407b-a610-86912176dc55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.384551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2e383-2bae-407b-a610-86912176dc55 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.385782 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 379/766] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:34:55.386790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095bfee-6556-43ff-88fb-742be93c24ab None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.400631 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 388/767] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:55.402434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053012c2-9015-4a4b-a4e9-398875d8d643 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.403437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053012c2-9015-4a4b-a4e9-398875d8d643 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.404023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053012c2-9015-4a4b-a4e9-398875d8d643 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.406419 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 389/768] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:55.407297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053012c2-9015-4a4b-a4e9-398875d8d643 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.448145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:55.495176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095bfee-6556-43ff-88fb-742be93c24ab None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.516330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053012c2-9015-4a4b-a4e9-398875d8d643 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.532165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d949ffb-1c8c-4bdc-9cda-6042cdfbb14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.533040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949ffb-1c8c-4bdc-9cda-6042cdfbb14c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.533523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949ffb-1c8c-4bdc-9cda-6042cdfbb14c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.534091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949ffb-1c8c-4bdc-9cda-6042cdfbb14c None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.535084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d949ffb-1c8c-4bdc-9cda-6042cdfbb14c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:55.535571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d949ffb-1c8c-4bdc-9cda-6042cdfbb14c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.537220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d949ffb-1c8c-4bdc-9cda-6042cdfbb14c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.554900 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 380/769] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:55.555067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f27-e00c-41bc-bb6b-b3ea205147eb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.555525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f27-e00c-41bc-bb6b-b3ea205147eb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.555641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f27-e00c-41bc-bb6b-b3ea205147eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.557038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.557769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.558049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.558345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.559073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=a090c2f1086243a19fd4f5ade6d7560b) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:55.559412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.560875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.577867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 390/770] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/role_assignments?user.id=a090c2f1086243a19fd4f5ade6d7560b&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:55.577867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.578922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-dffe4f27-e00c-41bc-bb6b-b3ea205147eb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:55.580463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dffe4f27-e00c-41bc-bb6b-b3ea205147eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:55.610148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:55.682797 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 381/771] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:34:55.684369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4f27-e00c-41bc-bb6b-b3ea205147eb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.720662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449404-8036-4ad6-9713-3ea792888665 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.738667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:55.771087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.771852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.772177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.772446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.773082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.774632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:55.787205 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:55.788342 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:55.789825 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 391/772] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:55.790675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.799905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4f27-e00c-41bc-bb6b-b3ea205147eb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.836343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.837010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.837284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.837605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/724ce8c58c1b473e9e3af18009532c9f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.839627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=724ce8c58c1b473e9e3af18009532c9f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:55.841156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:55.857012 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 382/773] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:34:55 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/724ce8c58c1b473e9e3af18009532c9f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:55.858232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.899014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:55.907410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e5d9-1486-47cc-b024-47402113d305 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.944527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:55.945438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:55.945745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:55.946015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:55.957479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:55.962796 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:55.963793 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:55.964641 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 392/774] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:55.970165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:55.983217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b82fc-97ea-454e-9335-1e406c4bf5aa None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:55.986413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:56.047631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.048932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.049231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.049754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.050448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:56.050758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.052256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.058956 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 383/775] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:56.060340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.104529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d0f56-9086-4932-9967-018fc401ac9d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.144160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.144858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.145135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.145404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.146156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:56.146460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.148024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.163052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 393/776] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:56.164082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.174561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:56.192224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3aa514-6e8b-4b96-a5d1-91b818ece6b1 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.233995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.234760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.234867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.235077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] PATH_INFO: `/v3/services/network` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.235667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.237055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:56.248451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:56.249811 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:56.250673 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 384/777] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:56.251575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.284644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:56.300910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b1581-962b-4eec-b2f5-bca369fcdf0c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.346234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.347493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.348252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.348718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.350214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:56.350690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.353170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.358206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9ff4-32b1-471f-9709-a368defb83b7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.360286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 394/778] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:56.362240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.399515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.400160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.400435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.400701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.414789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.416541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:56.431463 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:56.432496 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:56.433390 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:56.433390 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:56.433390 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:56.437776 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 385/779] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:56.438737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.463420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:56.507479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53188ce0-c07d-4dc1-bda1-66c3127a2681 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.526976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:56.545727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b780e2-50ba-4232-b4f8-3bb748e3a6ce None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.554764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.556069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.556569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.557385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.558865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] RBAC: Adding query filter params (name=network) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:56.559441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.561768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.568331 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 395/780] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:56.569675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.593211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.594012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.594454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.594820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.595627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] RBAC: Adding query filter params (interface=public, service_id=5e5d8a744ba14c8aab362b231c61bb7d, region_id=RegionOne) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:56.596028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.597544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.605557 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 386/781] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/endpoints?service_id=5e5d8a744ba14c8aab362b231c61bb7d&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:56.606614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.703475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63097a2f-c6fa-4885-912a-74b36f73048c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.721558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a57de-6854-4f96-be37-6d479fb4d170 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.749560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.750591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.750889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.751163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.752082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:56.752394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.754073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.756308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:56.760960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.762229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.762808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.763274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.764737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] RBAC: Adding query filter params (type=network) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:56.765274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.767028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.774560 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 396/782] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:56.774560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.778325 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 387/783] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:56.779461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.803234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:56.885810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bde6f9-b6e7-4eee-a17b-aedcf4376518 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.910892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a17cc6-9424-44d1-b1e3-4c04d3e5b831 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:56.924301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.925329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.925833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.926311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.927247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.928926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.935304 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 397/784] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:56.936481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:56.951790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:56.952734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:56.953022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:56.953295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] PATH_INFO: `/v3/users/swift` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:56.963300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:56.965506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:34:56.981636 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:56.982699 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:56.983696 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 388/785] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:56.996800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:57.032567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:57.088633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:57.122490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8820d2-ded9-4a26-a458-2b0a21d5413c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:57.147825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238703-1546-4be6-b961-65b4564c3ef0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:57.188825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:57.189477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:57.189769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:57.190044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:57.190759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] RBAC: Adding query filter params (interface=public, service_id=e37614906d7747e5b09de78421c24651, region_id=RegionOne) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:57.191067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:57.192802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:57.203139 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 398/786] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/endpoints?service_id=e37614906d7747e5b09de78421c24651&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:57.209553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:57.213925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:57.214559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:57.214837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:57.215108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:57.215925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:57.216232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:57.218616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:57.238191 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 389/787] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 17 20:34:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:57.243636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:57.336002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0cd08-f6c0-48e0-aca9-6a3c148aab7d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:57.356334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88d3bc-ec5b-43f2-b610-69ee64b75a4a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:57.363676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:57.378005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:57.378625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:57.378906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:57.379180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:57.380246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:57.381951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:57.388432 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 399/788] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:34:57 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:57.409843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:57.430893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:57.431558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:57.431846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:57.432125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:57.453953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:57.455445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:57.481874 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:57.482893 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:57.483802 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:57.483802 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:57.483802 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:57.483802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 390/789] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:57.490509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:57.640857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff950ed3-3737-489d-ab62-d40d21b60d87 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:57.681863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:57.682496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:57.682781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:57.683051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:57.683711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:57.684015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:57.685511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:57.699201 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 400/790] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:57.706614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:57.826555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a30ba-7e7e-461c-8d80-5a5ee58e82c7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:57.863449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:57.864120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:57.864400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:57.864672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:57.872782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:57.874317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:34:57.888006 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:57.888989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:57.889955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 391/791] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:34:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:57.896647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:57.947622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:58.014471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaa42c-0dd3-49bb-a6ea-7f6ac8f14840 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:58.058378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b337a1-145f-480c-84df-360c75518caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:58.058996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b337a1-145f-480c-84df-360c75518caf None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:58.059441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b337a1-145f-480c-84df-360c75518caf None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:58.059723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b337a1-145f-480c-84df-360c75518caf None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:58.060472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b337a1-145f-480c-84df-360c75518caf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:58.060780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b337a1-145f-480c-84df-360c75518caf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:58.062247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b337a1-145f-480c-84df-360c75518caf None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:58.080847 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 401/792] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:34:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:58.087135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:58.139936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:58.214998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cba0d7-e4a2-48b5-bbc1-29d28e888863 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:58.261788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fac799-2567-4557-bde3-21a52d275875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:58.262708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fac799-2567-4557-bde3-21a52d275875 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:58.263004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fac799-2567-4557-bde3-21a52d275875 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:58.263343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fac799-2567-4557-bde3-21a52d275875 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/8eea4641bfd940fe8a94ec79484ef4d3/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:58.267679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fac799-2567-4557-bde3-21a52d275875 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=8eea4641bfd940fe8a94ec79484ef4d3, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:58.269302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fac799-2567-4557-bde3-21a52d275875 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:58.304034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 392/793] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:34:58 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/8eea4641bfd940fe8a94ec79484ef4d3/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:34:58.364594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a315615-8b3b-4123-b5d1-402564d85032 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:58.364877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a315615-8b3b-4123-b5d1-402564d85032 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:58.365147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a315615-8b3b-4123-b5d1-402564d85032 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:58.366854 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 402/794] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:58.376212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af06b-f453-4d20-8eb6-8724a663a8cb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:58.376565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af06b-f453-4d20-8eb6-8724a663a8cb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:58.376793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af06b-f453-4d20-8eb6-8724a663a8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:58.401926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-023af06b-f453-4d20-8eb6-8724a663a8cb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:58.403244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-023af06b-f453-4d20-8eb6-8724a663a8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:58.485451 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 393/795] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:34:58.839270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d5cde-f206-442b-b464-1b29a219597e None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:58.839780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d5cde-f206-442b-b464-1b29a219597e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:58.839780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d5cde-f206-442b-b464-1b29a219597e None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:58.841408 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 403/796] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:58.846704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15442699-c30e-4ac3-93da-3b033aa1e4ff None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:58.847131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15442699-c30e-4ac3-93da-3b033aa1e4ff None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:58.847304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15442699-c30e-4ac3-93da-3b033aa1e4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:58.865521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-15442699-c30e-4ac3-93da-3b033aa1e4ff None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:34:58.866694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-15442699-c30e-4ac3-93da-3b033aa1e4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:34:58.903073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:58.908330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d5cde-f206-442b-b464-1b29a219597e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:58.967726 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 394/797] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:34:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:34:59.047110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d5cde-f206-442b-b464-1b29a219597e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:59.055558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:59.088254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:59.089266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:59.089839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:59.090337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:59.100946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:59.106063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:59.107007 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:34:59.107853 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:59.115436 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 404/798] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:34:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:59.116423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15442699-c30e-4ac3-93da-3b033aa1e4ff None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:59.231769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15442699-c30e-4ac3-93da-3b033aa1e4ff None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:59.269130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:59.270194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:59.270689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:59.271207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:59.272105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:59.272661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:59.274409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:59.284441 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 395/799] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:34:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:59.290364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:59.394875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce61543-9f37-44b7-a4c4-211785b9e98c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:59.430926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:59.432015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:59.432489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:59.433007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:59.441911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:59.447572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:59.449058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:34:59.464463 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:59.465617 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:34:59.466622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:34:59.466622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:34:59.466622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:34:59.466622 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 405/800] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:34:59.479250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:59.483805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:59.538704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:59.571394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8125f-b37d-45a3-ac17-5bf4b08bf8da None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:59.608174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:59.609217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:59.609744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:59.610212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:59.631159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:34:59.638384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:34:59.639371 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:34:59.640219 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:34:59.644632 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 396/801] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:34:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:34:59.651821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:59.740560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e843d1-9244-48e4-b334-f00516e9e828 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:59.813691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:59.814729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:59.815203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:59.815666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:59.816512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:59.817131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:59.821024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:59.840470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:59.843003 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 406/802] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:34:59.850426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:59.870596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc9535-2379-4d91-99e5-2d3217dfacad None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:34:59.935157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94fc8e-51df-4b78-9545-639756b90d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:34:59.936310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94fc8e-51df-4b78-9545-639756b90d20 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:59.936841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94fc8e-51df-4b78-9545-639756b90d20 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:59.937308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94fc8e-51df-4b78-9545-639756b90d20 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:59.938199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94fc8e-51df-4b78-9545-639756b90d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:34:59.938791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94fc8e-51df-4b78-9545-639756b90d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:34:59.940501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94fc8e-51df-4b78-9545-639756b90d20 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:34:59.951928 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 397/803] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:34:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:34:59.953875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08257d-9ac4-474d-a7e6-81b73714d4b6 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:34:59.954380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08257d-9ac4-474d-a7e6-81b73714d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:34:59.954869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08257d-9ac4-474d-a7e6-81b73714d4b6 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:34:59.956535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 398/804] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:34:59.958191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08257d-9ac4-474d-a7e6-81b73714d4b6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:34:59.966934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:34:59.995142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353f28-602b-46b0-8b87-c816cef5ba6d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.041933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20ee3d-d5ba-42a1-93af-760fca4cef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.043197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20ee3d-d5ba-42a1-93af-760fca4cef9e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.044713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20ee3d-d5ba-42a1-93af-760fca4cef9e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.045224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20ee3d-d5ba-42a1-93af-760fca4cef9e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.054686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20ee3d-d5ba-42a1-93af-760fca4cef9e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.057264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20ee3d-d5ba-42a1-93af-760fca4cef9e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1e20ee3d-d5ba-42a1-93af-760fca4cef9e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:00.072143 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:00.073140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:00.074136 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:00.080205 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 407/805] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:34:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:00.080205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f8589-e8ce-472b-9ab4-c6f14b544c4b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.080205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f8589-e8ce-472b-9ab4-c6f14b544c4b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.080205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f8589-e8ce-472b-9ab4-c6f14b544c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.087358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08257d-9ac4-474d-a7e6-81b73714d4b6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.099767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9d2f8589-e8ce-472b-9ab4-c6f14b544c4b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:00.101256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9d2f8589-e8ce-472b-9ab4-c6f14b544c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:00.122334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.123081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.123363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.123641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.134276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.135763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.156808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:00.167076 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:00.168098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:00.169043 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:00.169043 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:00.169043 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:00.169043 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 399/806] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:00.169789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.198030 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 408/807] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:00.199258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f8589-e8ce-472b-9ab4-c6f14b544c4b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.277067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:00.314358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f8589-e8ce-472b-9ab4-c6f14b544c4b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.349093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90531a-1ee4-4a99-b03e-6759e68781e8 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.364440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.365495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.365974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.366498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.367343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:00.367842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.369750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.393264 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 409/808] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:00.400248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.412667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.413284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.417992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.417992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.417992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:00.417992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.417992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.435604 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 400/809] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:00.444789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.564144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:00.575953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d8765-3506-4d38-9576-98cdd11dbdd7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.603491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:00.615925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.616964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.617242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.617554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.625540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.626990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.637035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16154-63e7-47d0-be96-fee7ca205004 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:00.638747 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:00.639672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:00.640610 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 410/810] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:00.641203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.685958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.686921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.687325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.687781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.701715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.703842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:00.724413 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:00.725669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:00.726776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:00.726776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:00.726776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:00.728711 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 401/811] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:00.729673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.775659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007c1f6-0746-4afd-9d9d-8a8497754835 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.811775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.812570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.813111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.813444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.814260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.815746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:00.825138 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:00.826355 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:35:00.827419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:00.833511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 411/812] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:00.833511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.865630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980fb59-3a55-4d8d-9b4a-dcdd61594f69 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.875370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:00.891884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:00.904123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19502275-4e77-44bc-93dc-aebbca9414e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.904744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19502275-4e77-44bc-93dc-aebbca9414e4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.905028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19502275-4e77-44bc-93dc-aebbca9414e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.905299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19502275-4e77-44bc-93dc-aebbca9414e4 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.906431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19502275-4e77-44bc-93dc-aebbca9414e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:00.906742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19502275-4e77-44bc-93dc-aebbca9414e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.908193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19502275-4e77-44bc-93dc-aebbca9414e4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.927282 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 402/813] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:00.928588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6f92e-2945-4e3e-8d7c-d3100a6f93ed None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.928868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6f92e-2945-4e3e-8d7c-d3100a6f93ed None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.929137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6f92e-2945-4e3e-8d7c-d3100a6f93ed None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.930487 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 403/814] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:00.931033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6f92e-2945-4e3e-8d7c-d3100a6f93ed None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:00.947681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920657ec-87e2-43a5-9fed-4bad799cd80b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:00.982913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:00.983524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:00.983807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:00.984076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:00.984715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:00.985016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:00.986415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:00.998278 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 412/815] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:00.999167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.046075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6f92e-2945-4e3e-8d7c-d3100a6f93ed None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.081020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487905de-8d48-4a4b-afac-9c51e135ea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.081752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487905de-8d48-4a4b-afac-9c51e135ea78 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.082028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487905de-8d48-4a4b-afac-9c51e135ea78 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.082295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487905de-8d48-4a4b-afac-9c51e135ea78 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.082929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487905de-8d48-4a4b-afac-9c51e135ea78 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:01.083229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487905de-8d48-4a4b-afac-9c51e135ea78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.084827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487905de-8d48-4a4b-afac-9c51e135ea78 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.089754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 404/816] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:01.091237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f85ab0-f201-4d5b-ba33-4619b23c6c1a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.091767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f85ab0-f201-4d5b-ba33-4619b23c6c1a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.092039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f85ab0-f201-4d5b-ba33-4619b23c6c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.108032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30599603-d149-4404-84e9-2304e73c6313 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.109749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-93f85ab0-f201-4d5b-ba33-4619b23c6c1a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:01.110921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-93f85ab0-f201-4d5b-ba33-4619b23c6c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:01.121358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.140025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.145413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.146381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.146978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.147633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.193028 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 405/817] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:01.193928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f85ab0-f201-4d5b-ba33-4619b23c6c1a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.227768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.229234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.300495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f85ab0-f201-4d5b-ba33-4619b23c6c1a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.331013 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 413/818] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:01.335068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dd242-46e9-4533-a25f-f0a9719e1998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.335785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd242-46e9-4533-a25f-f0a9719e1998 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.335964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd242-46e9-4533-a25f-f0a9719e1998 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.336182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd242-46e9-4533-a25f-f0a9719e1998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.336607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.343266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dd242-46e9-4533-a25f-f0a9719e1998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.344734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dd242-46e9-4533-a25f-f0a9719e1998 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.351003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-793dd242-46e9-4533-a25f-f0a9719e1998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:01.356173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:01.357096 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:01.358115 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 406/819] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:01.358950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e055a5-4b55-4734-b989-25834d78b438 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.358950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e055a5-4b55-4734-b989-25834d78b438 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.358950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e055a5-4b55-4734-b989-25834d78b438 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.360354 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 407/820] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:01.365108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e055a5-4b55-4734-b989-25834d78b438 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.387945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.447734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79a2f3-5eca-4a84-aeaf-52c827c1553a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.471213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e055a5-4b55-4734-b989-25834d78b438 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.484376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.484992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.485503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.485635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.486193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:01.486496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.487874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.497429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 414/821] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 17 20:35:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:01.497429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.510844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685ecc8-d3bf-4866-b755-c580cd02d564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.511481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685ecc8-d3bf-4866-b755-c580cd02d564 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.511765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685ecc8-d3bf-4866-b755-c580cd02d564 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.512027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685ecc8-d3bf-4866-b755-c580cd02d564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.523769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685ecc8-d3bf-4866-b755-c580cd02d564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.525246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685ecc8-d3bf-4866-b755-c580cd02d564 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-d685ecc8-d3bf-4866-b755-c580cd02d564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:01.539902 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:01.540918 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:01.541886 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:01.541886 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:01.541886 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:01.541886 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 408/822] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:01.542174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a09081-f87e-4fb6-8970-f5dffacdffef None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.542458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a09081-f87e-4fb6-8970-f5dffacdffef None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.542732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a09081-f87e-4fb6-8970-f5dffacdffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.560743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-79a09081-f87e-4fb6-8970-f5dffacdffef None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:01.561983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-79a09081-f87e-4fb6-8970-f5dffacdffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:01.576537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.602627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195efa3-9f42-4645-a406-07c3c93ac0b5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.628203 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 409/823] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:01.629020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a09081-f87e-4fb6-8970-f5dffacdffef None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.631222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.642488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.643109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.643399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.643709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.644497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:01.644815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.646438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.665274 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 415/824] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:01.665274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.737748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a09081-f87e-4fb6-8970-f5dffacdffef None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.773026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.773694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.773978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.774252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.774930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.776300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.777212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd02b-2f09-430a-9791-ab798f32feee None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.802376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 410/825] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:01.803449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.822137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.823652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:01.824969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:01.825224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:01.825591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] PATH_INFO: `/v3/services/network` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:01.826318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:01.828131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:01.838761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:01.840059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:01.841029 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 416/826] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:01 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:01.841875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:01.917651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:01.968738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a45739-658a-46c7-860b-612f1fa96ab4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:01.972776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa709c-ccbc-4b35-b712-eeaf757e5cf6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.008138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.008828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.009107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.009366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/a090c2f1086243a19fd4f5ade6d7560b/roles/b888b6f28fcb407795b56d37fdf7c6f6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.011419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.012052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.012393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.012664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.013327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:02.013665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.015206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.028832 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 411/827] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:02.029513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.031023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=a090c2f1086243a19fd4f5ade6d7560b, role_id=b888b6f28fcb407795b56d37fdf7c6f6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.032908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.054316 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 417/828] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:35:01 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/a090c2f1086243a19fd4f5ade6d7560b/roles/b888b6f28fcb407795b56d37fdf7c6f6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:35:02.054316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.114801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:02.162594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debe1af-475e-4d6c-a2e8-ca1bbeebddfd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.179229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996a974-3f92-4f3f-8b43-77e8879149d1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.188066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:02.216163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.218686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.218686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.218686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.218978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.219075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] RBAC: Adding query filter params (name=network) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:02.219075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.219210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.219210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.219345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.219932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.230266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 412/829] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:02.230266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.238035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:02.243157 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:02.244187 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:02.245127 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:02.247657 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 418/830] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:02.248743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.366381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fbc53-233d-4032-b8c4-657d4d4ce854 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.376711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc208-954d-455c-88ef-fe53a68314ef None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.381844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:02.404909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.406025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.406376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.406715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.407426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] RBAC: Adding query filter params (type=network) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:02.407797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.409282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.414751 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 419/831] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:02.419779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.428330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.429792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.430252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.431005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.440797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.442825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:02.458589 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:02.459557 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:02.460486 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:02.462887 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 413/832] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:02.468614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.529666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:02.531320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29132fec-23f9-4122-9c5d-862df43d5abe None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.573491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.575235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.575698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.576144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.577570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:02.578485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.580350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.593083 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 420/833] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:02.594243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.607216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c56e3a-7944-46b7-8e15-080cb27ee213 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.639468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:02.643358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.644139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.644418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.644798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.645891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.647292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.681570 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 414/834] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:02 2022] POST /identity/v3/endpoints => generated 337 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:02.681570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.784336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff098603-7d72-4554-bfb5-e61e0caf09c3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.846723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:02.859758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c40d4f-0663-40aa-a0f6-fbc357c4c2ff None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:02.883278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.883916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.884190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.884460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.885155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:02.885454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.886927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.919308 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 421/835] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:02.921870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:02.922517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:02.922780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:02.923045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:02.925979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:02.947183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:02.948995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:02.966850 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:02.967888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:02.968817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:02.968817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:02.968817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:02.970954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 415/836] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:02.977670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:03.051001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:03.063099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4565b36-d44d-4ad8-a81f-c85250b16e01 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:03.107656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:03.108296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:03.108662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:03.108933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:03.109717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=724ce8c58c1b473e9e3af18009532c9f) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:03.110350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:03.111561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:03.122073 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 422/837] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/role_assignments?user.id=724ce8c58c1b473e9e3af18009532c9f&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:03.129499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab57ead-4023-4304-ba7f-c9f527811414 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:03.169870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:03.170530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:03.170811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:03.171110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:03.171779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:03.172082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:03.173622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:03.188176 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 416/838] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:03.194113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:03.329588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885e682-64ee-4e9c-a20d-4cc6d608c451 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:03.368051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:03.369146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:03.369667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:03.370136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] PATH_INFO: `/v3/users/swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:03.379418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:03.381235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:03.393945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:03.394978 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:35:03.395943 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:03.398135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 423/839] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:35:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:03.403929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:03.415029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:03.512215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ad3d-b0d1-415d-a174-fdd1354f3a75 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:03.547930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:03.548539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:03.548864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:03.549140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:03.549902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:03.550203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:03.551635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:03.569809 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 417/840] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 17 20:35:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:03.575738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:03.595785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:03.697492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72f5f1-4976-477e-a963-6d5d89e1f56c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:03.729750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:03.730459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:03.730745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:03.731076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:03.802361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:03.803835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:03.822184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:03.913088 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 424/841] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:03.915466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:03.981767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:04.027280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5140f80-62e7-4255-a095-f92c6d811dac None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:04.067774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:04.068415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.068692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.068963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.080035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:04.081544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:04.098668 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:04.099680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:04.100559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:04.100559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:04.100559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:04.104318 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 418/842] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:04.106186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:04.222442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaf96f-e36c-41ff-b2e6-79caf9217c6e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:04.240024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:04.258798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:04.259480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.259762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.260035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.260744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:04.261049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:04.262573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:04.280471 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 425/843] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:04.282392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:04.386708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779bd48-b7d1-4eb0-921a-b79125f9bdf9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:04.403497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:04.424388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231808b5-8331-4d02-bf19-c0c52aa174dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:04.425055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231808b5-8331-4d02-bf19-c0c52aa174dc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.425337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231808b5-8331-4d02-bf19-c0c52aa174dc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.425647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231808b5-8331-4d02-bf19-c0c52aa174dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.433664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231808b5-8331-4d02-bf19-c0c52aa174dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:04.435194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231808b5-8331-4d02-bf19-c0c52aa174dc None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-231808b5-8331-4d02-bf19-c0c52aa174dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:04.447437 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:04.448317 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:04.449233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:04.453415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 419/844] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:04.454857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:04.463024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b8b2a-024c-4efd-8b6b-cffa65f2448d None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.463306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b8b2a-024c-4efd-8b6b-cffa65f2448d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.463585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b8b2a-024c-4efd-8b6b-cffa65f2448d None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.465199 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 420/845] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:04.470384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466802df-ecff-40f2-81d0-a51de3ed1c67 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.470674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466802df-ecff-40f2-81d0-a51de3ed1c67 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.470944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466802df-ecff-40f2-81d0-a51de3ed1c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.489904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-466802df-ecff-40f2-81d0-a51de3ed1c67 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:04.491021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-466802df-ecff-40f2-81d0-a51de3ed1c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:04.577370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8101184-3bff-4bf1-a839-a1a9d6c84ad3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:04.585626 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 421/846] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:04.593040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe745d27-440b-4f94-a0d4-3aab2447d252 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.593325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe745d27-440b-4f94-a0d4-3aab2447d252 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.594907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe745d27-440b-4f94-a0d4-3aab2447d252 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.595864 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 422/847] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:04.601208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232df9d-f726-4ca7-ad8a-7bbd8570edd7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.601687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232df9d-f726-4ca7-ad8a-7bbd8570edd7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.602097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232df9d-f726-4ca7-ad8a-7bbd8570edd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.621814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5232df9d-f726-4ca7-ad8a-7bbd8570edd7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:04.623225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5232df9d-f726-4ca7-ad8a-7bbd8570edd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:04.624552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:04.625273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.625937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.626317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.628038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:04.628474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:04.630031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:04.649545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 426/848] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:04.652183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:04.698366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:04.717832 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 423/849] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:04.767026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6884e3-4908-4fae-8dad-3643b0d62f5c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:04.802887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:04.803497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:04.803776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:04.804044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:04.804782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=8eea4641bfd940fe8a94ec79484ef4d3) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:04.805086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:04.806591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:04.812841 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 427/850] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:04 2022] GET /identity/v3/role_assignments?user.id=8eea4641bfd940fe8a94ec79484ef4d3&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:04.913188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232df9d-f726-4ca7-ad8a-7bbd8570edd7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:04.951972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:05.023199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232df9d-f726-4ca7-ad8a-7bbd8570edd7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.052752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:05.063254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.063907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.064180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.064451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.065654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:05.067160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:05.077578 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:05.078611 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:05.079441 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 424/851] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:05.085774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:05.182664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:05.187628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b72cf-81d0-49f7-a839-199be0c5a31a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.215969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90310cf5-c6b2-4b18-a657-4ee860fe8026 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.227120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.228368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.228819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.229281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.234331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:05.243958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:05.249989 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:05.250945 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:35:05.251775 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:05.259453 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 428/852] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:35:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:05.260514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:05.270381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.271004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.271306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.271581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.272222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:05.272525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:05.275079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:05.282144 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 425/853] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:05.290473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:05.395629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8be85-74f4-482c-886d-c1e6c6a9a429 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.398200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28092a-d78d-462a-a7ea-e4aa7525f268 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.434512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.435264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.435603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.436166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.436912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:05.437223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:05.437914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.438654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:05.438966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.439425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.440049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.441247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:05.441909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:05.443531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:05.450062 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 426/854] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:05.458157 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 429/855] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:35:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:05.459653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:05.494104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:05.578816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ad823-2bcb-4e9b-b7f2-a236fce14eea None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.621337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e072-1c0c-4319-9d7f-390a455c8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.622135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e072-1c0c-4319-9d7f-390a455c8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.622416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e072-1c0c-4319-9d7f-390a455c8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.622686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e072-1c0c-4319-9d7f-390a455c8cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.634963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25e072-1c0c-4319-9d7f-390a455c8cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:05.636475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25e072-1c0c-4319-9d7f-390a455c8cb1 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-7a25e072-1c0c-4319-9d7f-390a455c8cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:05.653242 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:05.654309 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:05.655194 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:05.655194 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:05.655194 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:05.655194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 427/856] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:05.662485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:05.700216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:05.779847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53684-7d2c-447b-a97b-cedbe1d4ab4b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.804771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807be28-6b83-45b8-929b-e9048b01e5d0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.805100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807be28-6b83-45b8-929b-e9048b01e5d0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.805319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807be28-6b83-45b8-929b-e9048b01e5d0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.806961 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 428/857] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:05.811895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a952cd-8790-4017-8cae-b5ecc61de4fb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.812181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a952cd-8790-4017-8cae-b5ecc61de4fb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.812459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a952cd-8790-4017-8cae-b5ecc61de4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.816276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.816897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.817238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.817547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.818196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:05.818675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:05.820066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:05.831242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-29a952cd-8790-4017-8cae-b5ecc61de4fb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:05.832369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-29a952cd-8790-4017-8cae-b5ecc61de4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:05.832949 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 430/858] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:05.838694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:05.917567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 429/859] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:05.931741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:05.950436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bc210-e61c-4a36-8f79-e31abbc5e472 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:05.984854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:05.985779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:05.986193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:05.986600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:05.993999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:05.995598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:06.007972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:06.009182 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:06.010397 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:06.012907 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 431/860] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:35:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:06.018878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a952cd-8790-4017-8cae-b5ecc61de4fb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:06.031994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:06.123304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a952cd-8790-4017-8cae-b5ecc61de4fb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:06.158724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:06.159581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:06.160049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:06.160457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:06.161357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:06.161829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:06.163401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.181668 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 430/861] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:06.187667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:06.247860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:06.299732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80754b-5958-4ec5-ac2e-bbb0e0ef006c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:06.329972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:06.340955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:06.341898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:06.342336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:06.342852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:06.357022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:06.367491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c256847-86fe-4e7d-ba12-1897865ec69c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:06.404800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:06.406139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:06.406591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:06.407053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:06.407778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:06.409398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:06.418888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:06.420015 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:06.421055 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:06.421830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:06.421830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.423646 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 431/862] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:06.433842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:06.533662 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 432/863] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:06.537990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:06.547592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c20c5-b1cd-4c4b-bdf2-593373b4641e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:06.582487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:06.587642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:06.590627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:06.591433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:06.591911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:06.592501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:06.593221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:06.593616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:06.595379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.605508 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 432/864] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:06.608202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:06.661560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f74a-f501-4068-b430-b2bbbea64b01 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:06.708293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:06.708931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:06.709212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:06.709491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:06.719344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62525a3c-5486-4204-8626-5c7485beb588 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:06.721705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:06.723178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:06.739399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:06.740457 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:06.741367 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:06.741367 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:06.741367 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:06.741575 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 433/865] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:06.746054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:06.766078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:06.767453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:06.767927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:06.768389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:06.769212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:06.770252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:06.772122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.779490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 433/866] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:06.862980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:06.870241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e59f2-8b56-439f-a3e7-ea31ce7a043f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:06.910695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:06.911309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:06.911635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:06.911851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:06.912496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:06.912794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:06.919860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:06.932139 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 434/867] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:06.937708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:07.008427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:07.050080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcd61d-3251-4d3d-b398-6df1d4cf9896 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:07.083761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:07.084395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:07.084667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:07.084935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:07.092012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:07.093487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:07.104721 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:07.105622 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:07.106593 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 434/868] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:07.112090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:07.211953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05c185-ec18-4e0b-b9b9-80a2682028f4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:07.245323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bc971-12d6-4a28-9db3-8eee6b863156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:07.245960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc971-12d6-4a28-9db3-8eee6b863156 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:07.246236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc971-12d6-4a28-9db3-8eee6b863156 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:07.246616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc971-12d6-4a28-9db3-8eee6b863156 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:07.247306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bc971-12d6-4a28-9db3-8eee6b863156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:07.247605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bc971-12d6-4a28-9db3-8eee6b863156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:07.249028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bc971-12d6-4a28-9db3-8eee6b863156 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:07.261486 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 435/869] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:07.267713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:07.368737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43a31e-1ff1-446d-bc34-6babc3a287cd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:07.389189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:07.403672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e8679-2453-4848-8876-f7ffab24b8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:07.404278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8679-2453-4848-8876-f7ffab24b8c2 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:07.404552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8679-2453-4848-8876-f7ffab24b8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:07.404819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8679-2453-4848-8876-f7ffab24b8c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:07.405929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e8679-2453-4848-8876-f7ffab24b8c2 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=a090c2f1086243a19fd4f5ade6d7560b) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:07.406234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e8679-2453-4848-8876-f7ffab24b8c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:07.407793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e8679-2453-4848-8876-f7ffab24b8c2 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:07.414064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 435/870] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:07 2022] GET /identity/v3/role_assignments?user.id=a090c2f1086243a19fd4f5ade6d7560b&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:07.434200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f06c0-0bcf-4de0-945f-bcc1610e7b4e None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:07.434532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f06c0-0bcf-4de0-945f-bcc1610e7b4e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:07.434750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f06c0-0bcf-4de0-945f-bcc1610e7b4e None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:07.436139 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 436/871] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:07.442446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a25d75-72f9-43fd-98ce-3d807b3c26dc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:07.442770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a25d75-72f9-43fd-98ce-3d807b3c26dc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:07.443039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a25d75-72f9-43fd-98ce-3d807b3c26dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:07.463694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e4a25d75-72f9-43fd-98ce-3d807b3c26dc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:07.464342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e4a25d75-72f9-43fd-98ce-3d807b3c26dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:07.540065 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 436/872] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:07.865510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f06c0-0bcf-4de0-945f-bcc1610e7b4e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:07.881657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:07.973299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f06c0-0bcf-4de0-945f-bcc1610e7b4e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:08.011432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:08.012306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:08.012695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:08.013064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:08.013944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:08.015896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:08.024248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b501b77-f04a-42d6-96ae-1b74e18cd62f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:08.024515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b501b77-f04a-42d6-96ae-1b74e18cd62f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:08.024787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b501b77-f04a-42d6-96ae-1b74e18cd62f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:08.026444 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 437/873] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:08.026865 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:08.028169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:08.029499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 437/874] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:08.032781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4f57-5c83-4f26-9ff8-8ed9ce76ca6f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:08.032781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4f57-5c83-4f26-9ff8-8ed9ce76ca6f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:08.032781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4f57-5c83-4f26-9ff8-8ed9ce76ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:08.038355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:08.054189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4fff4f57-5c83-4f26-9ff8-8ed9ce76ca6f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:08.055564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4fff4f57-5c83-4f26-9ff8-8ed9ce76ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:08.123309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:08.134206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 438/875] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:08.151177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8921b1f-8a2b-45f4-bc3e-693ebae2bcd8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:08.187652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:08.188265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:08.188584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:08.188853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:08.189501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:08.189803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:08.191210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:08.196120 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 438/876] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:08.201878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4f57-5c83-4f26-9ff8-8ed9ce76ca6f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:08.248302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:08.306231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4f57-5c83-4f26-9ff8-8ed9ce76ca6f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:08.345205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c27808-a05a-49ba-b86e-68bff7ad3c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:08.345888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c27808-a05a-49ba-b86e-68bff7ad3c9f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:08.346162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c27808-a05a-49ba-b86e-68bff7ad3c9f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:08.346433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c27808-a05a-49ba-b86e-68bff7ad3c9f None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:08.347068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c27808-a05a-49ba-b86e-68bff7ad3c9f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:08.347366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c27808-a05a-49ba-b86e-68bff7ad3c9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:08.348746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c27808-a05a-49ba-b86e-68bff7ad3c9f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:08.353674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 439/877] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:08.475764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:08.585120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb921965-06e9-4b01-9c7f-5cff3b4a0ca8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:08.620109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361e9c8-f34f-4e3f-8f9a-4bfbe93d99a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:08.621055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361e9c8-f34f-4e3f-8f9a-4bfbe93d99a7 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:08.621743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361e9c8-f34f-4e3f-8f9a-4bfbe93d99a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:08.622210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361e9c8-f34f-4e3f-8f9a-4bfbe93d99a7 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:08.623069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361e9c8-f34f-4e3f-8f9a-4bfbe93d99a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:08.624657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361e9c8-f34f-4e3f-8f9a-4bfbe93d99a7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:08.641522 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 439/878] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:35:08 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:09.335392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ce90-37b6-4f22-b842-288902aeb329 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:09.335918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ce90-37b6-4f22-b842-288902aeb329 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:09.335918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74ce90-37b6-4f22-b842-288902aeb329 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:09.337396 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 440/879] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:09.342980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d9279-c4aa-4712-b716-3ca23d3e2753 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:09.343497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d9279-c4aa-4712-b716-3ca23d3e2753 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:09.343965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d9279-c4aa-4712-b716-3ca23d3e2753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:09.350607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:09.364881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-470d9279-c4aa-4712-b716-3ca23d3e2753 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:09.367372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-470d9279-c4aa-4712-b716-3ca23d3e2753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:09.440407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 440/880] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:09 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:09.760039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74ce90-37b6-4f22-b842-288902aeb329 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:09.862322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74ce90-37b6-4f22-b842-288902aeb329 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:09.872884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:09.898331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64cdf3-5a3e-4471-9e68-d0f6c44b78da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:09.898945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64cdf3-5a3e-4471-9e68-d0f6c44b78da None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:09.899217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64cdf3-5a3e-4471-9e68-d0f6c44b78da None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:09.899486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64cdf3-5a3e-4471-9e68-d0f6c44b78da None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:09.900150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64cdf3-5a3e-4471-9e68-d0f6c44b78da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:09.901545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64cdf3-5a3e-4471-9e68-d0f6c44b78da None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:09.921669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 441/881] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:35:09 2022] POST /identity/v3/services => generated 269 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:10.031838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2195f-a2a3-428e-ba60-e2ae11317a3d None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:10.032349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2195f-a2a3-428e-ba60-e2ae11317a3d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:10.032349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2195f-a2a3-428e-ba60-e2ae11317a3d None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:10.033971 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 441/882] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:10.043517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae840-db2d-4219-b554-bb6cf9c8f969 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:10.044134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae840-db2d-4219-b554-bb6cf9c8f969 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:10.044623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae840-db2d-4219-b554-bb6cf9c8f969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:10.064678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-938ae840-db2d-4219-b554-bb6cf9c8f969 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:10.066265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-938ae840-db2d-4219-b554-bb6cf9c8f969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:10.150660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 442/883] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:10.472406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2195f-a2a3-428e-ba60-e2ae11317a3d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:10.575981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2195f-a2a3-428e-ba60-e2ae11317a3d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:10.603152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:10.611637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:10.612715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:10.613251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:10.613865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:10.624417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:10.629137 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:10.630122 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:10.631160 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 442/884] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 17 20:35:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:10.636333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae840-db2d-4219-b554-bb6cf9c8f969 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:10.737955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:10.745773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae840-db2d-4219-b554-bb6cf9c8f969 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:10.782947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050e65c-fb7d-450e-8af0-976ea0676ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:10.783960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e65c-fb7d-450e-8af0-976ea0676ace None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:10.784429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e65c-fb7d-450e-8af0-976ea0676ace None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:10.784889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e65c-fb7d-450e-8af0-976ea0676ace None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:10.785803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050e65c-fb7d-450e-8af0-976ea0676ace None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:10.786297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050e65c-fb7d-450e-8af0-976ea0676ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:10.789082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050e65c-fb7d-450e-8af0-976ea0676ace None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:10.798252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 443/885] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 17 20:35:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:10.804324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:10.885433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75332e-ecbf-44ef-bf84-16d883b35c97 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:10.886151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75332e-ecbf-44ef-bf84-16d883b35c97 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:10.886646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75332e-ecbf-44ef-bf84-16d883b35c97 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:10.892157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 444/886] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:10.893776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5398ff-5afc-4e3c-90ac-36b5bad9e109 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:10.894269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5398ff-5afc-4e3c-90ac-36b5bad9e109 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:10.894762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5398ff-5afc-4e3c-90ac-36b5bad9e109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:10.914111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aa5398ff-5afc-4e3c-90ac-36b5bad9e109 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:10.915631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa5398ff-5afc-4e3c-90ac-36b5bad9e109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:10.915743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fc3c7-ded6-49ea-b2e9-0dfefcd50d44 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:10.954305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:10.954941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:10.955219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:10.955483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:10.966756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:10.968498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:10.985552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:10.986527 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:10.988089 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:10.988089 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:10.988089 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:10.988089 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 443/887] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:10.992620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:10.995652 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 445/888] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:11.035920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:11.095874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98917ce-5188-42e7-a764-f2532b155459 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:11.129912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:11.130516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.130787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.131053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.131697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:11.131992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:11.133357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:11.145805 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 444/889] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:11.149549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adff758-ca09-4a77-a4cb-5f9306130c4d None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.149838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adff758-ca09-4a77-a4cb-5f9306130c4d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.150111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adff758-ca09-4a77-a4cb-5f9306130c4d None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.151232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:11.152031 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 446/890] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:11.157640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabaffd-b0ee-462b-ba56-5367332d771b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.157922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabaffd-b0ee-462b-ba56-5367332d771b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.158193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabaffd-b0ee-462b-ba56-5367332d771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.178805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aaabaffd-b0ee-462b-ba56-5367332d771b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:11.180467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aaabaffd-b0ee-462b-ba56-5367332d771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:11.258958 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 447/891] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:11.270035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:11.273348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5c89-a513-4407-9550-7a4eacd51055 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:11.310165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:11.310794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.311067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.311330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.319086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:11.320533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:11.332670 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:11.333672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:11.335336 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 445/892] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:35:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:11.339934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabaffd-b0ee-462b-ba56-5367332d771b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:11.345406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:11.362322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:11.462614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34979d9-83b8-4995-9401-5608396051c5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:11.477770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabaffd-b0ee-462b-ba56-5367332d771b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:11.506887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:11.508239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.508933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.509509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.510983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:11.513097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:11.518771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:11.519531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.519714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.519968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.520839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:11.521140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:11.522664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:11.539396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 446/893] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:35:11 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:35:11.554938 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 448/894] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 17 20:35:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:11.561309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:11.583387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:11.594069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:11.676673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:11.721430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f5760-daf0-45e9-bf85-07b79d535bc6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:11.723147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ae13-99e8-4806-972a-cdd963cdb0db None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:11.770814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:11.771422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.771692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.771961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.772554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:11.773668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:11.773947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:11.774737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.775207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.775676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:11.784817 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:11.785923 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:11.786779 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 449/895] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:11.794446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:11.861938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:11.863315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:11.883421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:11.918310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51b03e-c5f2-4b76-808b-5f423e4f02ee None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:11.954947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:11.955555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:11.955830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:11.956096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:11.956729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:11.957022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:11.958834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:11.963649 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 450/896] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:11.973505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:11.974118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:11.994709 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 447/897] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:11 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:11.996246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.119434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df364cb0-53c0-46de-80d1-607639d7a3aa None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.120451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7ade-d78a-4e48-a628-e8435c5cee62 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.140516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:12.163766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.163980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.164606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.164927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.165058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.165425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.165606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.166003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.166352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:12.166757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.168204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.177506 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 451/898] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:12.179848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.182792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.184454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:12.201021 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:12.202114 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:12.203043 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:12.203043 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:12.203043 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:12.205959 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 448/899] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:12.208284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.256522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:12.296062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae66a0-eecb-451c-b688-e776abd6a48a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.330624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.331240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.331514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.331785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.332571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] RBAC: Adding query filter params (interface=public, service_id=a699dcb84883428fa1b67c3670713303, region_id=RegionOne) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:12.332873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.334371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.336075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac35901-55a3-472c-a88f-58af7de481a5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.344960 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 452/900] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:35:12 2022] GET /identity/v3/endpoints?service_id=a699dcb84883428fa1b67c3670713303&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:12.345624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.378670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.379733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.380166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.380573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.381504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:12.381947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.384280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.399551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 449/901] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:12.410498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.446547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:12.457876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bcc75-af16-4c7c-a1c8-a515e61bc757 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.487030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:12.494202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd60c12-8418-4337-a0aa-d3c65e01a247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.495549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd60c12-8418-4337-a0aa-d3c65e01a247 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.495832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd60c12-8418-4337-a0aa-d3c65e01a247 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.496099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd60c12-8418-4337-a0aa-d3c65e01a247 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.496803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd60c12-8418-4337-a0aa-d3c65e01a247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.498210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd60c12-8418-4337-a0aa-d3c65e01a247 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.513487 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 453/902] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:35:12 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:35:12.513487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd65b2e-934b-454e-83d9-74557467afe5 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.513487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd65b2e-934b-454e-83d9-74557467afe5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.513487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd65b2e-934b-454e-83d9-74557467afe5 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.513487 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 454/903] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:12.514414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395ef82-b3d4-449e-a199-d7b855af5958 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.514616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395ef82-b3d4-449e-a199-d7b855af5958 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.514885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395ef82-b3d4-449e-a199-d7b855af5958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.536147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1395ef82-b3d4-449e-a199-d7b855af5958 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:12.537380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6e7fa-fb40-4964-b4c8-712fcc4f908d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.537520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1395ef82-b3d4-449e-a199-d7b855af5958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:12.581480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.582316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.582679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.583046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.591296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.593051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:12.606221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:12.607386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:12.608370 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:12.614092 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 450/904] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:12.615085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.632286 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 455/905] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:12.737401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:12.752433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad3808-5cc3-4839-bd2e-5584cd0e0ed0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.788394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.789008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.789281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.789608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.790327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:12.790630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.792065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.804566 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 451/906] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:12.810421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395ef82-b3d4-449e-a199-d7b855af5958 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.920150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395ef82-b3d4-449e-a199-d7b855af5958 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:12.945254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:12.957756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:12.958085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:12.958707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:12.958983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:12.959251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:12.959983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] RBAC: Adding query filter params (role.id=85e89b3e3f63492db730418fb475453d, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=a090c2f1086243a19fd4f5ade6d7560b) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:12.960286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:12.961790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:12.969514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 456/907] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:12 2022] GET /identity/v3/role_assignments?user.id=a090c2f1086243a19fd4f5ade6d7560b&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=85e89b3e3f63492db730418fb475453d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:13.074356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2c62-5270-4c28-8277-04f1d6570bc8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:13.112142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:13.112784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:13.113058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:13.113322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:13.113949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:13.115320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:13.124795 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:13.126419 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:13.128125 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 452/908] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Sep 17 20:35:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:13.134857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:13.244511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dd493-14c9-4743-8329-a8e344ba4b07 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:13.279407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:13.280399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:13.281254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:13.281788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:13.282619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:13.283138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:13.284713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:13.290225 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 457/909] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:13.297629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:13.313723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:13.402379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417df32e-1ce7-4f69-93b7-a63d9d65f3ee None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:13.443487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:13.444353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:13.444797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:13.445217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:13.446058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:13.446507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:13.448007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:13.457771 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 453/910] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:13.459761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:13.513973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:13.564535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8f21d-de3a-4c1b-b2cd-84ccf2a18cd6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:13.600920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d76229-8a48-4943-a256-ab38344a85b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:13.601581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d76229-8a48-4943-a256-ab38344a85b2 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:13.601864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d76229-8a48-4943-a256-ab38344a85b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:13.602135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d76229-8a48-4943-a256-ab38344a85b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:13.602867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d76229-8a48-4943-a256-ab38344a85b2 None admin] RBAC: Adding query filter params (interface=public, service_id=7b2a6ce8b10f48bdbd17da287b42cb76, region_id=RegionOne) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:13.603169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d76229-8a48-4943-a256-ab38344a85b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:13.604598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d76229-8a48-4943-a256-ab38344a85b2 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:13.611150 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 458/911] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:35:13 2022] GET /identity/v3/endpoints?service_id=7b2a6ce8b10f48bdbd17da287b42cb76&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:13.616831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:13.715752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:13.737835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8662-de7f-4200-8a69-a76a5a208299 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:13.790271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f2a5-3e1c-4fb1-8a84-f3aec13748ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:13.790806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f2a5-3e1c-4fb1-8a84-f3aec13748ec None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:13.791082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f2a5-3e1c-4fb1-8a84-f3aec13748ec None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:13.791350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f2a5-3e1c-4fb1-8a84-f3aec13748ec None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:13.791965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49f2a5-3e1c-4fb1-8a84-f3aec13748ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:13.793365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49f2a5-3e1c-4fb1-8a84-f3aec13748ec None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:13.803463 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 454/912] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:35:13 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:35:14.121705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5520c5f-5967-4a53-a364-53025778f922 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:14.122217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5520c5f-5967-4a53-a364-53025778f922 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:14.122217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5520c5f-5967-4a53-a364-53025778f922 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:14.123864 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 459/913] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:14.129370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de028-2249-42c2-835b-3b1b172d7be6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:14.129823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de028-2249-42c2-835b-3b1b172d7be6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:14.130027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de028-2249-42c2-835b-3b1b172d7be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:14.150582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3d5de028-2249-42c2-835b-3b1b172d7be6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:14.151777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3d5de028-2249-42c2-835b-3b1b172d7be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:14.249956 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 455/914] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:14.653277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5520c5f-5967-4a53-a364-53025778f922 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:14.769757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5520c5f-5967-4a53-a364-53025778f922 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:14.797841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:14.804886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:14.805596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:14.805876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:14.806187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:14.806818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:14.808303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:14.817911 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:14.818900 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:14.819749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 460/915] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:14.825388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de028-2249-42c2-835b-3b1b172d7be6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:14.895546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:14.931743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de028-2249-42c2-835b-3b1b172d7be6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:14.969093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ea4fd-961a-412b-824e-a3e381efc387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:14.969777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ea4fd-961a-412b-824e-a3e381efc387 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:14.970056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ea4fd-961a-412b-824e-a3e381efc387 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:14.970330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ea4fd-961a-412b-824e-a3e381efc387 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:14.970988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ea4fd-961a-412b-824e-a3e381efc387 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:14.971292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ea4fd-961a-412b-824e-a3e381efc387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:14.972717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ea4fd-961a-412b-824e-a3e381efc387 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:14.978669 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 456/916] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:14.984357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:15.087654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b69015-0b16-4394-a9a5-6f7210c0d6dd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:15.123004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d13e9-8744-404d-a20d-fe3f0f21e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:15.123643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d13e9-8744-404d-a20d-fe3f0f21e951 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.124028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d13e9-8744-404d-a20d-fe3f0f21e951 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.124221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d13e9-8744-404d-a20d-fe3f0f21e951 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.124977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d13e9-8744-404d-a20d-fe3f0f21e951 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:15.125281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d13e9-8744-404d-a20d-fe3f0f21e951 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:15.126713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435bf48-c779-45b9-a5c6-62035e054fe2 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.126824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d13e9-8744-404d-a20d-fe3f0f21e951 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:15.127304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435bf48-c779-45b9-a5c6-62035e054fe2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.127912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435bf48-c779-45b9-a5c6-62035e054fe2 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.133149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 461/917] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:15.133637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 457/918] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:15.137984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435bf48-c779-45b9-a5c6-62035e054fe2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:15.138435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e5ed-bcab-4882-acc2-8db1db593017 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.138923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e5ed-bcab-4882-acc2-8db1db593017 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.139394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e5ed-bcab-4882-acc2-8db1db593017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.160771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0608e5ed-bcab-4882-acc2-8db1db593017 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:15.162010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0608e5ed-bcab-4882-acc2-8db1db593017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:15.211740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:15.235791 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 462/919] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:15.249408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435bf48-c779-45b9-a5c6-62035e054fe2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:15.285225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:15.285877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.286157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.286435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.287145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] RBAC: Adding query filter params (interface=public, service_id=e5900057c7854d3290b915e1f9b3badf, region_id=RegionOne) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:15.287456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:15.288954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:15.294681 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 458/920] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:35:15 2022] GET /identity/v3/endpoints?service_id=e5900057c7854d3290b915e1f9b3badf&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:15.295154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:15.304862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608e5ed-bcab-4882-acc2-8db1db593017 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:15.404104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608e5ed-bcab-4882-acc2-8db1db593017 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:15.439329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6791c04-cec8-4d47-882f-d18e2694da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:15.439946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6791c04-cec8-4d47-882f-d18e2694da02 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.440230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6791c04-cec8-4d47-882f-d18e2694da02 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.440496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6791c04-cec8-4d47-882f-d18e2694da02 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.441101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6791c04-cec8-4d47-882f-d18e2694da02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:15.442505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6791c04-cec8-4d47-882f-d18e2694da02 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:15.448895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 463/921] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:35:15 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:35:15.574657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:15.646822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12c1e-6c91-4d65-a2e1-982be250e854 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.647169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12c1e-6c91-4d65-a2e1-982be250e854 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.647396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12c1e-6c91-4d65-a2e1-982be250e854 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.649191 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 464/922] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:15.654954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c6a34-8f5f-4442-81b2-5de8478f6f2c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.655286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c6a34-8f5f-4442-81b2-5de8478f6f2c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.655911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c6a34-8f5f-4442-81b2-5de8478f6f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.679595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-643c6a34-8f5f-4442-81b2-5de8478f6f2c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:15.681525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-643c6a34-8f5f-4442-81b2-5de8478f6f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:15.690177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145d945-b1f9-4756-a69c-66f631e77b4f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:15.720799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:15.730867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:15.731097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.731379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.731645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.732665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:15.733637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:15.742436 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:15.743484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:15.744339 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 459/923] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:15.749555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:15.777485 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 465/924] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:15.832344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:15.872654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372c536-ab22-464c-b75a-6ba126c46639 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:15.912034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:15.912680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:15.912952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:15.913224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:15.913928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:15.914230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:15.915640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:15.921789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 460/925] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:15.934749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c6a34-8f5f-4442-81b2-5de8478f6f2c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.037968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c6a34-8f5f-4442-81b2-5de8478f6f2c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:16.072636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:16.073249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.073559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.073834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.074564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:16.074873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:16.076257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:16.081572 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 466/926] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 17 20:35:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:16.087570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.145624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.184038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:16.217809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b52573-81b3-4808-87cc-c2fb9b48a0b2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:16.245373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:16.255890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62979dcd-4642-4446-b5f0-c208ad1580a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:16.256784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979dcd-4642-4446-b5f0-c208ad1580a7 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.257276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979dcd-4642-4446-b5f0-c208ad1580a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.257769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62979dcd-4642-4446-b5f0-c208ad1580a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.258653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62979dcd-4642-4446-b5f0-c208ad1580a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:16.260216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62979dcd-4642-4446-b5f0-c208ad1580a7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:16.276285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983558e5-930a-4ab7-8a96-9bed719e249b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:16.294341 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 461/927] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:16.326458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6bc04-8250-4183-87f8-024f873e713d None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.326777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6bc04-8250-4183-87f8-024f873e713d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.327012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6bc04-8250-4183-87f8-024f873e713d None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.329821 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 462/928] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:16.331720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:16.332431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.332713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.332979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.334509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fae43-92f9-4d3d-bb2c-d8a5b1c09a33 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.334798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fae43-92f9-4d3d-bb2c-d8a5b1c09a33 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.335075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fae43-92f9-4d3d-bb2c-d8a5b1c09a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.345691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:16.350559 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:16.351580 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:16.352415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 467/929] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 17 20:35:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:16.358000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.362169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-134fae43-92f9-4d3d-bb2c-d8a5b1c09a33 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:16.363578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-134fae43-92f9-4d3d-bb2c-d8a5b1c09a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:16.454381 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 463/930] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:16.480256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:16.494786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7495610-a3fe-4c41-ab56-8883a4ae4cb4 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:16.538279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:16.538894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.539170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.539436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.540119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:16.540417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:16.541855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:16.551626 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 468/931] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 17 20:35:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:16.557073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fae43-92f9-4d3d-bb2c-d8a5b1c09a33 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.662249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fae43-92f9-4d3d-bb2c-d8a5b1c09a33 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:16.681803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:16.705810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:16.707026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.707387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.707727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.718601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:16.720067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:16.735287 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:16.736293 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:16.737221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:16.737221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:16.737221 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:16.737221 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 464/932] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:16.743619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.782707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.859099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dc003-e46e-46ba-8b94-7e0f43ade07b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:16.894749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e079d5-0235-420d-a344-fd5f5f95cde0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:16.898233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:16.898917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.899197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.899471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.900173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:16.900480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:16.901903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:16.919604 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 469/933] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:16.925422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.938245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:16.938886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:16.939213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:16.939487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:16.940078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:16.941553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:16.942516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:16.951991 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:16.952957 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:16.953889 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 465/934] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Sep 17 20:35:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:16.962708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:16.994075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:17.050131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85df52-2f70-4f24-a376-efe22d1c82d9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.071764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392b689-c76e-4ce1-af02-94a175517488 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.087564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.088559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.089072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.089653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.097812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.099744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.108207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.108878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.109155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.109427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.110088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:17.110388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.111763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:17.114843 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:17.115788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:17.121910 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:17.122540 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 466/935] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:17.122615 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 470/936] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:35:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:17.126705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:17.126814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:17.219144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:17.229821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:17.241199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de2818-6d21-45a3-91c9-2134d64adfb2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.261264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc667b21-17d0-4ebc-a4d4-626e02200422 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.276743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.277503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.278091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.278463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.279246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:17.279663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.281583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.291532 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 467/937] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:17.292810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:17.299331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.299944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.300218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.300484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.301213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:17.301535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.302976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.322204 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 471/938] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 17 20:35:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:17.327788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:17.398284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686286-aa43-446f-96f9-5bf9dde3e19b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.437299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.438072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.438451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.438844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.439614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.441084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.447452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9cff-1344-408b-a905-9c27796cc3bc None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.467226 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 468/939] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:35:17.479343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:17.500879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.502303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.502931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.503580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.522231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.524804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:17.549039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:17.550446 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:17.551634 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:17.551634 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:17.551634 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:17.553448 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 472/940] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:17.559571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:17.578360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:17.684572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a087d-24b1-4be0-b19a-3f05d7d9a7d0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.721492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.722113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.722389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.722659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.723307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:17.723598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.724975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.737933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 469/941] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:17.743618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:17.858121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc72b-ec7b-4b53-b759-6bafe08ff9b8 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:17.900058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c0790-f174-48e6-9bff-e3b6094ad768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:17.900995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0790-f174-48e6-9bff-e3b6094ad768 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:17.901407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0790-f174-48e6-9bff-e3b6094ad768 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:17.901847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0790-f174-48e6-9bff-e3b6094ad768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:17.913664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c0790-f174-48e6-9bff-e3b6094ad768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:17.915955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c0790-f174-48e6-9bff-e3b6094ad768 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ce1c0790-f174-48e6-9bff-e3b6094ad768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:17.931654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:17.932692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:17.933667 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 473/942] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:17.942592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:18.013371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:18.044553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cf3ee-bfea-4ed4-bb1a-c3d99c6cfd06 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:18.063850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6700ab-755b-4037-aff8-e78bdfb62ada None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.064149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6700ab-755b-4037-aff8-e78bdfb62ada None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.064394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6700ab-755b-4037-aff8-e78bdfb62ada None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.066027 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 474/943] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:18.070985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0197c73-d5b6-4b22-adc0-2bb959779aa8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.071260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0197c73-d5b6-4b22-adc0-2bb959779aa8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.071529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0197c73-d5b6-4b22-adc0-2bb959779aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.079604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:18.080225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.080502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.080770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.081474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:18.081785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:18.083315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:18.089864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f0197c73-d5b6-4b22-adc0-2bb959779aa8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:18.091106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f0197c73-d5b6-4b22-adc0-2bb959779aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:18.098406 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 470/944] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:18.104292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:18.124477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:18.173769 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 475/945] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:18.211395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bc760-c8ba-4aff-84b2-3fe184e9daa9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:18.247284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:18.253195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:18.254518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.255176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.255767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/a090c2f1086243a19fd4f5ade6d7560b/roles/85e89b3e3f63492db730418fb475453d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.259222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=a090c2f1086243a19fd4f5ade6d7560b, role_id=85e89b3e3f63492db730418fb475453d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:18.261487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:18.283878 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 471/946] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:35:18 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/a090c2f1086243a19fd4f5ade6d7560b/roles/85e89b3e3f63492db730418fb475453d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:35:18.513625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0197c73-d5b6-4b22-adc0-2bb959779aa8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:18.624790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0197c73-d5b6-4b22-adc0-2bb959779aa8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:18.662312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:18.663348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.663825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.664289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.665098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:18.666724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:18.676287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:18.677454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:18.678396 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:18.681214 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 476/947] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:18.687534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:18.789209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d25e79-ce41-43ab-bf60-0b4d3433279e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:18.824364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f168ad2-6204-4982-970f-7e7cb105b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:18.825057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f168ad2-6204-4982-970f-7e7cb105b5ab None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.825334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f168ad2-6204-4982-970f-7e7cb105b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.825633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f168ad2-6204-4982-970f-7e7cb105b5ab None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.826268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f168ad2-6204-4982-970f-7e7cb105b5ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:18.826571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f168ad2-6204-4982-970f-7e7cb105b5ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:18.828144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f168ad2-6204-4982-970f-7e7cb105b5ab None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:18.833281 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 472/948] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:18.839322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:18.872507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:18.891617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8b28-6373-48a3-b19f-3b9268f10705 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.892062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8b28-6373-48a3-b19f-3b9268f10705 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.892443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8b28-6373-48a3-b19f-3b9268f10705 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.894652 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 473/949] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:18.902753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded7652-e730-499f-94e3-e3b899524b0f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:18.903155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded7652-e730-499f-94e3-e3b899524b0f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:18.903521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded7652-e730-499f-94e3-e3b899524b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:18.925619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3ded7652-e730-499f-94e3-e3b899524b0f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:18.927218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3ded7652-e730-499f-94e3-e3b899524b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:18.957028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af1870-f761-4b1e-951d-2c0984baa5f5 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:18.983641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:18.999809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:19.000516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:19.000861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:19.001201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:19.002133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:19.002507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:19.004599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:19.013485 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 477/950] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:19.014018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 474/951] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:19.015907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:19.118098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:19.125445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a1db1-096b-4a9f-9409-da6035a4b808 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:19.160662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:19.161196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:19.161494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:19.161839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:19.162515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:19.163957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:19.189340 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 478/952] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:19.374036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded7652-e730-499f-94e3-e3b899524b0f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:19.482338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded7652-e730-499f-94e3-e3b899524b0f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:19.523885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:19.524859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:19.525329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:19.525851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:19.526766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:19.528552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:19.531773 ncdv-master-1 devstack@keystone.service[99167]: WARNING keystone.server.flask.common [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 17 20:35:19.550666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.sql.core [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 20:35:19.550666 ncdv-master-1 devstack@keystone.service[99167]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 20:35:19.550666 ncdv-master-1 devstack@keystone.service[99167]: [parameters: {'id': 'bffb72a79f2b4216b4996a31e504e28d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 20:35:19.550666 ncdv-master-1 devstack@keystone.service[99167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bffb72a79f2b4216b4996a31e504e28d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bffb72a79f2b4216b4996a31e504e28d. Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:35:19.568356 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application result.read() Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 17 20:35:19.569283 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application yield resource Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise self.value Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application session.commit() Sep 17 20:35:19.570297 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.session.flush() Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application compat.raise_( Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 20:35:19.571265 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 20:35:19.572276 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application result.read() Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:35:19.573106 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [parameters: {'id': 'bffb72a79f2b4216b4996a31e504e28d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:19.575834 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:19.577761 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bffb72a79f2b4216b4996a31e504e28d. Sep 17 20:35:19.580019 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:19.582964 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 475/953] 10.222.0.26 () {60 vars in 1140 bytes} [Sat Sep 17 20:35:19 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:19.591127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:19.719339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9c88a-0163-439f-b668-8538d7eaae2d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:19.755558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3702975-c219-45ec-92e2-2a154f52a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:19.756125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3702975-c219-45ec-92e2-2a154f52a344 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:19.756397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3702975-c219-45ec-92e2-2a154f52a344 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:19.756668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3702975-c219-45ec-92e2-2a154f52a344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:19.763903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3702975-c219-45ec-92e2-2a154f52a344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:19.765363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3702975-c219-45ec-92e2-2a154f52a344 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-b3702975-c219-45ec-92e2-2a154f52a344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:19.777139 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:19.778163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:19.779106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 479/954] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:19.784892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:19.828377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:19.887287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a311dcb-a1ad-429e-9ad3-ac995b4d9c92 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:19.923330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ba7ad-31fd-420b-84f7-567d71b11c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:19.923949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ba7ad-31fd-420b-84f7-567d71b11c95 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:19.924218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ba7ad-31fd-420b-84f7-567d71b11c95 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:19.924481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ba7ad-31fd-420b-84f7-567d71b11c95 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:19.925143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ba7ad-31fd-420b-84f7-567d71b11c95 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:19.925435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ba7ad-31fd-420b-84f7-567d71b11c95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:19.927100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ba7ad-31fd-420b-84f7-567d71b11c95 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:19.948485 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 476/955] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:20.059611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60463-62e4-4ee7-88ce-4f493f667653 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:20.059989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60463-62e4-4ee7-88ce-4f493f667653 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:20.060359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60463-62e4-4ee7-88ce-4f493f667653 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:20.062194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 480/956] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:20.068151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94793098-ff29-48f9-9131-afb600f22fee None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:20.068575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94793098-ff29-48f9-9131-afb600f22fee None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:20.069020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94793098-ff29-48f9-9131-afb600f22fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:20.089707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-94793098-ff29-48f9-9131-afb600f22fee None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:20.091408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-94793098-ff29-48f9-9131-afb600f22fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:20.166159 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 477/957] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:20.486992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef60463-62e4-4ee7-88ce-4f493f667653 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:20.500738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:20.594377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef60463-62e4-4ee7-88ce-4f493f667653 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:20.634800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303fe3-e0fa-424c-8a54-1f6424c9ca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:20.635693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303fe3-e0fa-424c-8a54-1f6424c9ca14 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:20.636113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303fe3-e0fa-424c-8a54-1f6424c9ca14 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:20.636545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303fe3-e0fa-424c-8a54-1f6424c9ca14 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:20.637320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303fe3-e0fa-424c-8a54-1f6424c9ca14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:20.638878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303fe3-e0fa-424c-8a54-1f6424c9ca14 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-95303fe3-e0fa-424c-8a54-1f6424c9ca14 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:20.648107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:20.649341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:20.650448 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:20.652917 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 481/958] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:20.659198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94793098-ff29-48f9-9131-afb600f22fee None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:20.687323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:20.751897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96622-1cd4-4e14-8f5c-ff066ca5ffec None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:20.752083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96622-1cd4-4e14-8f5c-ff066ca5ffec None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:20.752390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96622-1cd4-4e14-8f5c-ff066ca5ffec None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:20.753722 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 482/959] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:20.759023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34165f7c-8609-4ed2-9f4d-f55dc7b12feb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:20.759408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34165f7c-8609-4ed2-9f4d-f55dc7b12feb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:20.759782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34165f7c-8609-4ed2-9f4d-f55dc7b12feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:20.763106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94793098-ff29-48f9-9131-afb600f22fee None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:20.778760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-34165f7c-8609-4ed2-9f4d-f55dc7b12feb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:20.780090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34165f7c-8609-4ed2-9f4d-f55dc7b12feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:20.800094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:20.800977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:20.801423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:20.801877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:20.802741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] RBAC: Adding query filter params (name=compute) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:20.803191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:20.804699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:20.813539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 478/960] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:20.819494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:20.854928 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 483/961] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:20.924316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:20.930677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909485d-49d9-46e5-87ff-bd1b477fe987 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:20.964870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:20.965546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:20.965827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:20.966099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:20.966738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:20.967040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:20.968417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:20.973301 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 479/962] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:21.180834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34165f7c-8609-4ed2-9f4d-f55dc7b12feb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:21.184436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:21.288172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34165f7c-8609-4ed2-9f4d-f55dc7b12feb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:21.324204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:21.324888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:21.325165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:21.325433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:21.336279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:21.341333 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:21.342297 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:21.343205 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 484/963] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 17 20:35:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:21.348796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:21.459424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57900d-3f61-4d31-8ee4-775fa08bffa8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:21.493732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:21.494354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:21.494634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:21.494907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:21.495580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:21.495873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:21.497514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:21.506523 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 480/964] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 17 20:35:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:21.512250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:21.583996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:21.623918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849335c-c663-43a1-bb7d-b24064f4b792 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:21.661472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c75c-1657-4e3e-a5aa-f43e17350a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:21.662136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c75c-1657-4e3e-a5aa-f43e17350a15 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:21.662407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c75c-1657-4e3e-a5aa-f43e17350a15 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:21.662673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c75c-1657-4e3e-a5aa-f43e17350a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:21.679288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c75c-1657-4e3e-a5aa-f43e17350a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:21.680793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c75c-1657-4e3e-a5aa-f43e17350a15 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-a783c75c-1657-4e3e-a5aa-f43e17350a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:21.701768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:21.702746 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:21.703653 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:21.703653 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:21.703653 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:21.705732 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 485/965] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:21.711484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:21.731669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:21.815356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4896668-0f69-40d2-9805-55483e667e43 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:21.850494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:21.851109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:21.851389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:21.851670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:21.852310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:21.852616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:21.854032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:21.861019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2e6bc-bca2-4c0d-8cd6-636a447c0077 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:21.861298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2e6bc-bca2-4c0d-8cd6-636a447c0077 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:21.861607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2e6bc-bca2-4c0d-8cd6-636a447c0077 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:21.863210 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 486/966] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:21.866980 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 481/967] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:21.871128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196e8da-1957-48cd-a020-fdf1afe00333 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:21.871469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196e8da-1957-48cd-a020-fdf1afe00333 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:21.871743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196e8da-1957-48cd-a020-fdf1afe00333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:21.876042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:21.895575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f196e8da-1957-48cd-a020-fdf1afe00333 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:21.897908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f196e8da-1957-48cd-a020-fdf1afe00333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:21.969807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:21.997483 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 487/968] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:22.002717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95054789-ab80-460c-a491-62e1e996e02c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:22.040307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:22.040997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.041273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.041696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.050608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:22.052234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:22.065245 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:22.066317 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:22.068020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 482/969] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:35:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:22.072573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196e8da-1957-48cd-a020-fdf1afe00333 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.120700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:22.178087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196e8da-1957-48cd-a020-fdf1afe00333 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:22.213088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:22.213843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.214125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.214395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.215193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:22.215500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:22.216941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:22.236254 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 488/970] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:22.241426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.345401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.348425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f99734-308b-4fb8-8fdf-0a26788e5c9c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:22.389701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39a733-8601-4b89-8498-4ab8754372be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:22.390318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a733-8601-4b89-8498-4ab8754372be None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.390598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a733-8601-4b89-8498-4ab8754372be None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.390921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a733-8601-4b89-8498-4ab8754372be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.401521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:22.449181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863ea8a-0c8c-4b98-b539-40190cf1843c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:22.459359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:22.470022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39a733-8601-4b89-8498-4ab8754372be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:22.471608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39a733-8601-4b89-8498-4ab8754372be None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:22.489643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b45102-fcdb-4b17-8628-db5f40747498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:22.490213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b45102-fcdb-4b17-8628-db5f40747498 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.490485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b45102-fcdb-4b17-8628-db5f40747498 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.490751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b45102-fcdb-4b17-8628-db5f40747498 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.491332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b45102-fcdb-4b17-8628-db5f40747498 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:22.492784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b45102-fcdb-4b17-8628-db5f40747498 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-63b45102-fcdb-4b17-8628-db5f40747498 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:22.502191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:22.503273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:22.504128 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 489/971] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:22.504963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d53a-c06a-464b-99ab-36334ea56c8e None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.504963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d53a-c06a-464b-99ab-36334ea56c8e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.504963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d53a-c06a-464b-99ab-36334ea56c8e None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.506296 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 490/972] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:22.510949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d53a-c06a-464b-99ab-36334ea56c8e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.582537 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 483/973] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:22.588718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3a02e-d8ae-4d5f-a0a8-5b0ea06d9a24 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.589003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3a02e-d8ae-4d5f-a0a8-5b0ea06d9a24 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.589281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3a02e-d8ae-4d5f-a0a8-5b0ea06d9a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.607292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-21c3a02e-d8ae-4d5f-a0a8-5b0ea06d9a24 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:22.608509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21c3a02e-d8ae-4d5f-a0a8-5b0ea06d9a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:22.619963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d53a-c06a-464b-99ab-36334ea56c8e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:22.644886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:22.670376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:22.671006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.671283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.671555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.672191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:22.672487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:22.673901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:22.678773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 491/974] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:22.679560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.700748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 484/975] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:22.702642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3a02e-d8ae-4d5f-a0a8-5b0ea06d9a24 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.718935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:22.797234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4b449-fb9a-46e4-a398-ddcc5ec4af9c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:22.833543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3a02e-d8ae-4d5f-a0a8-5b0ea06d9a24 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:22.842741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:22.844147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.844642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.845595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.860299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:22.862893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:22.869408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:22.870058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:22.870330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:22.870590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:22.871221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:22.871639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:22.873180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:22.878773 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 485/976] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:22.881654 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:22.882632 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:22.883678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:22.883678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:22.883678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:22.889640 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 492/977] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:22.889781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.892611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:22.924152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:22.994028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:23.006065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024682-4bbc-46f0-96f5-e698b28c0e35 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.012925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f63ad0-1476-4b13-8cf8-0bef2cdc1ff1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.042126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.043002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.043618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.043884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.044690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:23.045170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.046644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.054289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.054853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.055132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.055408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.056151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] RBAC: Adding query filter params (interface=admin, service_id=e5900057c7854d3290b915e1f9b3badf, region_id=RegionOne) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:23.056465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.057937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.062569 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 493/978] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:23.063737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:23.064354 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 486/979] 10.222.0.26 () {56 vars in 1249 bytes} [Sat Sep 17 20:35:22 2022] GET /identity/v3/endpoints?service_id=e5900057c7854d3290b915e1f9b3badf&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:23.069937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:23.173244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:23.174652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c3c5-e307-4314-aa26-117c50417d01 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.179901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d09e3-3bda-444b-87d2-6435bdd01ec5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.209902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.210523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.210802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.211075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.211715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.213202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.215864 ncdv-master-1 devstack@keystone.service[99166]: WARNING keystone.server.flask.common [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 17 20:35:23.216246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.216848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.217122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.217387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.218019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.219357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.233489 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 487/980] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:35:23 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:35:23.233489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:23.239238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.sql.core [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 20:35:23.239238 ncdv-master-1 devstack@keystone.service[99166]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 20:35:23.239238 ncdv-master-1 devstack@keystone.service[99166]: [parameters: {'user_id': '17169e63f4414a1abc82866f7e22375f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 20:35:23.239238 ncdv-master-1 devstack@keystone.service[99166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 20:35:23.247262 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:35:23.248281 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application yield resource Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise self.value Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application session.commit() Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 20:35:23.249680 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.flush() Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application compat.raise_( Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 20:35:23.250917 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:35:23.252058 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:35:23.253212 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [parameters: {'user_id': '17169e63f4414a1abc82866f7e22375f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:23.255439 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 17 20:35:23.257554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 494/981] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 17 20:35:23 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:23.265820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:23.267940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:23.379016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aaa30c-1123-4c54-9d0c-0b5cd47b9d15 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.421555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.422472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.423146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.423593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.428339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32817542-0123-46cc-a8cd-84273c74fd93 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.432859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.434753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:23.449315 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:23.450486 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:23.451399 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:23.453746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 488/982] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:23.459664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:23.477431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545db3-feb0-4e41-b9aa-1bd09fa12598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.478128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545db3-feb0-4e41-b9aa-1bd09fa12598 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.478399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545db3-feb0-4e41-b9aa-1bd09fa12598 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.478662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545db3-feb0-4e41-b9aa-1bd09fa12598 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.485909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:23.487969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f545db3-feb0-4e41-b9aa-1bd09fa12598 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.489172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f545db3-feb0-4e41-b9aa-1bd09fa12598 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-3f545db3-feb0-4e41-b9aa-1bd09fa12598 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:23.502347 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:23.503344 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:23.504336 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 495/983] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:35:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:23.505030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55665aa2-a993-416c-a888-e59e2dd031e4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.505446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55665aa2-a993-416c-a888-e59e2dd031e4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.505639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55665aa2-a993-416c-a888-e59e2dd031e4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.509638 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 496/984] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:23.511647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55665aa2-a993-416c-a888-e59e2dd031e4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:23.572846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dca5ba-b19f-438c-b55b-f0a427f67a98 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.577595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:23.609518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c5a7f-66c2-469a-8b0e-00dbe8bc3d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.610447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5a7f-66c2-469a-8b0e-00dbe8bc3d13 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.610862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5a7f-66c2-469a-8b0e-00dbe8bc3d13 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.611134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5a7f-66c2-469a-8b0e-00dbe8bc3d13 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.611819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c5a7f-66c2-469a-8b0e-00dbe8bc3d13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:23.612124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c5a7f-66c2-469a-8b0e-00dbe8bc3d13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.613585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c5a7f-66c2-469a-8b0e-00dbe8bc3d13 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.613689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55665aa2-a993-416c-a888-e59e2dd031e4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.626867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 489/985] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:23.628151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c4b25-9baf-463b-be9f-3038c959a3fc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.628433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c4b25-9baf-463b-be9f-3038c959a3fc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.628700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c4b25-9baf-463b-be9f-3038c959a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.647393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e42c4b25-9baf-463b-be9f-3038c959a3fc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:23.648852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e42c4b25-9baf-463b-be9f-3038c959a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:23.652421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.653105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.653384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.653692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.654398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:23.654728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.656543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.696594 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 497/986] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 17 20:35:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:23.698527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:23.754164 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 490/987] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:23.813610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a217bd-ff7f-4474-8dc1-75e4731d43c2 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:23.841899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:23.852378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858aa31-3717-4357-8b22-d2afdafbb632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:23.853238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858aa31-3717-4357-8b22-d2afdafbb632 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:23.853954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858aa31-3717-4357-8b22-d2afdafbb632 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:23.854324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858aa31-3717-4357-8b22-d2afdafbb632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:23.855177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858aa31-3717-4357-8b22-d2afdafbb632 None admin] RBAC: Adding query filter params (role.id=85e89b3e3f63492db730418fb475453d, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=a090c2f1086243a19fd4f5ade6d7560b) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:23.855794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858aa31-3717-4357-8b22-d2afdafbb632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:23.857640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858aa31-3717-4357-8b22-d2afdafbb632 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:23.864697 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 498/988] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:23 2022] GET /identity/v3/role_assignments?user.id=a090c2f1086243a19fd4f5ade6d7560b&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=85e89b3e3f63492db730418fb475453d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:24.086634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c4b25-9baf-463b-be9f-3038c959a3fc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:24.098659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:24.190578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c4b25-9baf-463b-be9f-3038c959a3fc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:24.225506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd76d4-d824-4b14-82c0-75b6c5518410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:24.226120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd76d4-d824-4b14-82c0-75b6c5518410 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:24.226395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd76d4-d824-4b14-82c0-75b6c5518410 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:24.226737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd76d4-d824-4b14-82c0-75b6c5518410 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:24.227385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd76d4-d824-4b14-82c0-75b6c5518410 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:24.228815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd76d4-d824-4b14-82c0-75b6c5518410 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:24.247919 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 491/989] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:35:24 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:25.710962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8d239-6a05-4f11-a75b-2a674c4082be None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:25.711492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8d239-6a05-4f11-a75b-2a674c4082be None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:25.711492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8d239-6a05-4f11-a75b-2a674c4082be None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:25.712882 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 499/990] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:25.718559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b56942-90e3-4c3e-902d-0bddf1f05510 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:25.719015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b56942-90e3-4c3e-902d-0bddf1f05510 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:25.719105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b56942-90e3-4c3e-902d-0bddf1f05510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:25.737757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-96b56942-90e3-4c3e-902d-0bddf1f05510 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:25.738939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-96b56942-90e3-4c3e-902d-0bddf1f05510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:25.805081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:25.819494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 492/991] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:26.137386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8d239-6a05-4f11-a75b-2a674c4082be None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:26.253821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8d239-6a05-4f11-a75b-2a674c4082be None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:26.294171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d14023-4242-4561-a314-113a7bd7f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:26.294811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d14023-4242-4561-a314-113a7bd7f865 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.295294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d14023-4242-4561-a314-113a7bd7f865 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.295597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d14023-4242-4561-a314-113a7bd7f865 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.296251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d14023-4242-4561-a314-113a7bd7f865 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:26.297679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d14023-4242-4561-a314-113a7bd7f865 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-13d14023-4242-4561-a314-113a7bd7f865 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:26.306839 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:26.309671 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:26.310692 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 500/992] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 17 20:35:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:26.314447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b56942-90e3-4c3e-902d-0bddf1f05510 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:26.324025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50010-e725-403c-befc-f2fb475473ef None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.324312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50010-e725-403c-befc-f2fb475473ef None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.324665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50010-e725-403c-befc-f2fb475473ef None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.326185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 501/993] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:26.330862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6d697-54de-455e-ac16-5c44d63f8b32 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.331147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6d697-54de-455e-ac16-5c44d63f8b32 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.331671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6d697-54de-455e-ac16-5c44d63f8b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.349660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c5f6d697-54de-455e-ac16-5c44d63f8b32 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:26.351175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c5f6d697-54de-455e-ac16-5c44d63f8b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:26.402022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:26.423483 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 502/994] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:26.433294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b56942-90e3-4c3e-902d-0bddf1f05510 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:26.469514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d25b47-4d58-41b0-92f4-9675d4c56aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:26.470138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d25b47-4d58-41b0-92f4-9675d4c56aa8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.470364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d25b47-4d58-41b0-92f4-9675d4c56aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.470667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d25b47-4d58-41b0-92f4-9675d4c56aa8 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.471302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d25b47-4d58-41b0-92f4-9675d4c56aa8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:26.471603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d25b47-4d58-41b0-92f4-9675d4c56aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:26.472975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d25b47-4d58-41b0-92f4-9675d4c56aa8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:26.478370 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 493/995] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:26.484240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6d697-54de-455e-ac16-5c44d63f8b32 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:26.585064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6d697-54de-455e-ac16-5c44d63f8b32 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:26.619232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:26.619812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.620093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.620360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.620739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada8c1-0400-4bfc-9eca-e68ff0188505 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.620998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:26.621099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada8c1-0400-4bfc-9eca-e68ff0188505 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.621268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada8c1-0400-4bfc-9eca-e68ff0188505 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.621491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:26.622833 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 494/996] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:26.622947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:26.627581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5aa3d-fe2e-470b-b6df-654a2dd34aa7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.627815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5aa3d-fe2e-470b-b6df-654a2dd34aa7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.628062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5aa3d-fe2e-470b-b6df-654a2dd34aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.633067 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 503/997] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 17 20:35:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:26.633845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:26.646157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d7c5aa3d-fe2e-470b-b6df-654a2dd34aa7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:26.647331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d7c5aa3d-fe2e-470b-b6df-654a2dd34aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:26.689666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:26.714151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:26.727678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 495/998] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:26.749427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5aa3d-fe2e-470b-b6df-654a2dd34aa7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:26.751072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebd5cd-c5c5-4816-aac3-1b32c94cc45c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:26.786936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:26.787544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.787824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.788095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.788768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:26.790172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:26.809155 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 504/999] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:35:26.859654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5aa3d-fe2e-470b-b6df-654a2dd34aa7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:26.903921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:26.904567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:26.904805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:26.905128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] PATH_INFO: `/v3/services/image` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:26.906089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:26.907506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:26.917706 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:26.918972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:26.922490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 496/1000] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Sep 17 20:35:26 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:26.926116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:27.045482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa7706-e8a9-4a4f-a64b-2e31a8c73f4b None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:27.053026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:27.056144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:27.086188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:27.086809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:27.087089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:27.087360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:27.087999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] RBAC: Adding query filter params (name=image) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:27.088357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:27.089775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:27.094557 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 505/1001] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:27.100310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:27.116118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:27.170974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cf809-9814-434d-b103-94d15daa642e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:27.208388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:27.208823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8af5e-aa6d-4fdd-8f09-822867498150 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:27.209611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:27.210096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:27.210821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:27.212493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:27.214098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:27.223434 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:27.224475 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:27.226556 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 497/1002] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:27.231069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:27.247807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:27.248600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:27.248847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:27.249179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:27.249901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] RBAC: Adding query filter params (type=image) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:27.250212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:27.251602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:27.261562 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 506/1003] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:27.363429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac12863-a9e0-4841-8703-a04a0aaaa100 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:27.379977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:27.403808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:27.404448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:27.404726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:27.404994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:27.406022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] RBAC: Adding query filter params (name=compute) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:27.406327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:27.407719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:27.413597 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 498/1004] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:27.419370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:27.457738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:27.528786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3414c-74e4-4ca3-b853-ab85af499896 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:27.563285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:27.563898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:27.564178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:27.564465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:27.565077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] RBAC: Adding query filter params (type=compute) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:27.565380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:27.566805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:27.577048 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 507/1005] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:27.578326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:27.700287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c689a-3a40-46e2-900f-a70d2796180a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:27.739817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce06034-d434-42bb-8c95-7731c60d85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:27.740495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06034-d434-42bb-8c95-7731c60d85c5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:27.740708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06034-d434-42bb-8c95-7731c60d85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:27.740957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06034-d434-42bb-8c95-7731c60d85c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:27.741692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce06034-d434-42bb-8c95-7731c60d85c5 None admin] RBAC: Adding query filter params (interface=public, service_id=9166dfdde6db44afb0b25fb460546d37, region_id=RegionOne) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:27.742048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce06034-d434-42bb-8c95-7731c60d85c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:27.743374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce06034-d434-42bb-8c95-7731c60d85c5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:27.748929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 499/1006] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:35:27 2022] GET /identity/v3/endpoints?service_id=9166dfdde6db44afb0b25fb460546d37&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:27.754643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:27.855193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:27.859182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e1d38-ac39-4368-8346-d7ee554e288c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:27.893962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518cba4-2ab5-493e-8053-ec0c6b0d8407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:27.894575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cba4-2ab5-493e-8053-ec0c6b0d8407 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:27.894840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cba4-2ab5-493e-8053-ec0c6b0d8407 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:27.895104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518cba4-2ab5-493e-8053-ec0c6b0d8407 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:27.895698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518cba4-2ab5-493e-8053-ec0c6b0d8407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:27.897063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518cba4-2ab5-493e-8053-ec0c6b0d8407 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:27.903841 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 508/1007] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:35:27 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:35:29.227929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a31f3-805d-485d-a0db-192da4cba867 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:29.228515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a31f3-805d-485d-a0db-192da4cba867 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:29.228515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a31f3-805d-485d-a0db-192da4cba867 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:29.230024 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 500/1008] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:29.236309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ca588-5607-4829-aefa-f5363614751e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:29.236891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ca588-5607-4829-aefa-f5363614751e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:29.236891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ca588-5607-4829-aefa-f5363614751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:29.257128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-785ca588-5607-4829-aefa-f5363614751e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:29.258401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-785ca588-5607-4829-aefa-f5363614751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:29.329681 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 509/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:29.656556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a31f3-805d-485d-a0db-192da4cba867 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:29.658933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabda3f-7c61-4959-a9b2-94c4823d46f0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:29.659224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabda3f-7c61-4959-a9b2-94c4823d46f0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:29.659499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabda3f-7c61-4959-a9b2-94c4823d46f0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:29.660994 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 510/1010] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:29.665844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb3a7e-5da0-488f-bfa7-8176f507ef08 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:29.666130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb3a7e-5da0-488f-bfa7-8176f507ef08 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:29.666414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb3a7e-5da0-488f-bfa7-8176f507ef08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:29.687053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-99eb3a7e-5da0-488f-bfa7-8176f507ef08 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:29.688252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-99eb3a7e-5da0-488f-bfa7-8176f507ef08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:29.708321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:29.763191 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 511/1011] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:29.790909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a31f3-805d-485d-a0db-192da4cba867 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:29.831287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:29.831914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:29.832184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:29.832447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:29.833584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:29.835013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:29.836758 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 501/1012] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:29.842509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb3a7e-5da0-488f-bfa7-8176f507ef08 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:29.869643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:29.947563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb3a7e-5da0-488f-bfa7-8176f507ef08 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:29.982388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39879b-2485-4a22-bd7d-240daa5b8ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:29.982992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39879b-2485-4a22-bd7d-240daa5b8ef6 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:29.983264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39879b-2485-4a22-bd7d-240daa5b8ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:29.983531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39879b-2485-4a22-bd7d-240daa5b8ef6 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:29.984182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39879b-2485-4a22-bd7d-240daa5b8ef6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:29.985637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39879b-2485-4a22-bd7d-240daa5b8ef6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:30.022079 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 512/1013] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:35:29 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:30.080492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:30.201155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cd7f7-9a9a-4f6e-b84e-e012e2d544f9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:30.214192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:30.241233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7b525-2a18-4be3-8cbc-c13bd7a8a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:30.241981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b525-2a18-4be3-8cbc-c13bd7a8a1a6 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:30.242232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b525-2a18-4be3-8cbc-c13bd7a8a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:30.242485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b525-2a18-4be3-8cbc-c13bd7a8a1a6 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:30.243143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7b525-2a18-4be3-8cbc-c13bd7a8a1a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:30.244537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7b525-2a18-4be3-8cbc-c13bd7a8a1a6 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:30.263214 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 502/1014] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:35:30 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:30.295118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72650393-5564-4f8d-80ec-9fd672b2c87f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:30.295708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72650393-5564-4f8d-80ec-9fd672b2c87f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:30.295708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72650393-5564-4f8d-80ec-9fd672b2c87f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:30.297081 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 513/1015] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:30.304601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72d517-d6e1-42a9-a014-93ab01e5a2c5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:30.304981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72d517-d6e1-42a9-a014-93ab01e5a2c5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:30.305187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72d517-d6e1-42a9-a014-93ab01e5a2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:30.325654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5d72d517-d6e1-42a9-a014-93ab01e5a2c5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:30.326850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5d72d517-d6e1-42a9-a014-93ab01e5a2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:30.416488 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 503/1016] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:30 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:30.737072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72650393-5564-4f8d-80ec-9fd672b2c87f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:30.817935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:30.846192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72650393-5564-4f8d-80ec-9fd672b2c87f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:30.879936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:30.880632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:30.880912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:30.881177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:30.881798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:30.883181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:30.891967 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:30.892988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:30.893909 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 514/1017] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:30.899119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72d517-d6e1-42a9-a014-93ab01e5a2c5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:30.994721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:31.001922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72d517-d6e1-42a9-a014-93ab01e5a2c5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:31.038060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:31.038670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:31.038946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:31.039217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:31.039875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:31.040179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:31.041647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:31.046880 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 504/1018] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:31.052447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:31.152631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87a331-e7b3-4ead-a4bd-64333d70c7d8 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:31.190342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a66535-82d8-4140-81fe-c75947f24d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:31.190963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a66535-82d8-4140-81fe-c75947f24d5c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:31.191235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a66535-82d8-4140-81fe-c75947f24d5c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:31.191507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a66535-82d8-4140-81fe-c75947f24d5c None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:31.192146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a66535-82d8-4140-81fe-c75947f24d5c None admin] RBAC: Adding query filter params (type=compute) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:31.192450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a66535-82d8-4140-81fe-c75947f24d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:31.193881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a66535-82d8-4140-81fe-c75947f24d5c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:31.198905 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 515/1019] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 17 20:35:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:31.204864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:31.308747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a255d-0837-4d30-9c04-d1b0b52dd8ea None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:31.342310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2011c2-c41c-4abc-9699-3115087818ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:31.342913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2011c2-c41c-4abc-9699-3115087818ca None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:31.343183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2011c2-c41c-4abc-9699-3115087818ca None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:31.343450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2011c2-c41c-4abc-9699-3115087818ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:31.344098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2011c2-c41c-4abc-9699-3115087818ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:31.345489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2011c2-c41c-4abc-9699-3115087818ca None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:31.373518 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 505/1020] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:35:32.499260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817d52b-a1d1-4a0a-964a-7a5109e14210 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:32.499857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817d52b-a1d1-4a0a-964a-7a5109e14210 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:32.499857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817d52b-a1d1-4a0a-964a-7a5109e14210 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:32.501108 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 516/1021] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:32.506582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f055a9-1b84-4b79-abcc-70046380866f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:32.507008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f055a9-1b84-4b79-abcc-70046380866f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:32.507202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f055a9-1b84-4b79-abcc-70046380866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:32.526787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f1f055a9-1b84-4b79-abcc-70046380866f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:32.528276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1f055a9-1b84-4b79-abcc-70046380866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:32.540369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:32.607918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 506/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:32.697748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea934207-52ba-4463-88fd-04a2cfe9b950 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:32.698117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea934207-52ba-4463-88fd-04a2cfe9b950 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:32.698402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea934207-52ba-4463-88fd-04a2cfe9b950 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:32.699781 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 517/1023] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:32.705273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36827d6-336c-4347-836f-f1bf6e938be3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:32.705562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36827d6-336c-4347-836f-f1bf6e938be3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:32.705868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36827d6-336c-4347-836f-f1bf6e938be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:32.724286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e36827d6-336c-4347-836f-f1bf6e938be3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:32.725491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e36827d6-336c-4347-836f-f1bf6e938be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:32.797609 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 507/1024] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:32.935558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea934207-52ba-4463-88fd-04a2cfe9b950 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:32.938608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:33.037582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea934207-52ba-4463-88fd-04a2cfe9b950 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:33.071751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:33.072299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.072573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.072839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.073979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:33.075525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:33.077263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 518/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:33.082950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36827d6-336c-4347-836f-f1bf6e938be3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:33.113378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:33.153790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:33.184777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:33.189187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36827d6-336c-4347-836f-f1bf6e938be3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:33.219683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a3065-57e1-42f0-a16c-8afd39ae5a78 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:33.228892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:33.229543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.229821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.230086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.230747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:33.232181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:33.256118 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 508/1026] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:35:33 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:33.257700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:33.258366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.258647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.258918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] PATH_INFO: `/v3/services/image` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.259525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:33.260942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:33.271598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:33.272697 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:33.273661 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:33.277807 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 519/1027] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Sep 17 20:35:33 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:33.280103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:33.391091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9ca14-3584-479e-bed9-4e6551eb82e2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:33.422113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:33.432107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:33.432744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.433017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.433290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.434051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] RBAC: Adding query filter params (name=image) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:33.434356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:33.435759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:33.441866 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 509/1028] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:33.448139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:33.573854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fcb17-018d-4cf2-8418-02ce04e16cd6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:33.607397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:33.608012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.608290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.608547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.609179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] RBAC: Adding query filter params (type=image) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:33.609517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:33.610868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:33.612066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:33.617014 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 520/1029] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:33.622813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:33.731722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6a66f-d70e-4b97-967b-d72a0386c9b1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:33.771785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37016c4f-fbb3-4ba4-af37-c2b0ff6f34f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:33.772397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37016c4f-fbb3-4ba4-af37-c2b0ff6f34f8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.772641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37016c4f-fbb3-4ba4-af37-c2b0ff6f34f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.772990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37016c4f-fbb3-4ba4-af37-c2b0ff6f34f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.773714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37016c4f-fbb3-4ba4-af37-c2b0ff6f34f8 None admin] RBAC: Adding query filter params (interface=public, service_id=c376a1b0be22448da63b9a07794238ea, region_id=RegionOne) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:33.774016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37016c4f-fbb3-4ba4-af37-c2b0ff6f34f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:33.775429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37016c4f-fbb3-4ba4-af37-c2b0ff6f34f8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:33.781302 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 510/1030] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:35:33 2022] GET /identity/v3/endpoints?service_id=c376a1b0be22448da63b9a07794238ea&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:33.787039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:33.894616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8b803-07bb-4035-890c-98a6be319ca0 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.894927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8b803-07bb-4035-890c-98a6be319ca0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.895169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8b803-07bb-4035-890c-98a6be319ca0 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.896636 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 511/1031] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:33.901316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e984a-ff9d-469b-8260-3fa6ec91f337 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.901647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e984a-ff9d-469b-8260-3fa6ec91f337 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.901918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e984a-ff9d-469b-8260-3fa6ec91f337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.904420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4bd0c-07f3-4e1b-a475-c4cf35602123 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:33.920266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ee0e984a-ff9d-469b-8260-3fa6ec91f337 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:33.921530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee0e984a-ff9d-469b-8260-3fa6ec91f337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:33.942798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:33.943490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:33.943776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:33.944045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:33.944655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:33.946084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:33.952872 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 521/1032] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:35:33 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:35:33.976959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:33.997202 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 512/1033] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:34.318568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:34.388199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:34.432740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6792ed8-29f5-456a-a9be-408ccf2b3e4d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:34.467197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:34.467835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:34.468120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:34.468383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:34.479363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:34.485505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:34.486926 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:34.487897 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 522/1034] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 17 20:35:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:34.491524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e984a-ff9d-469b-8260-3fa6ec91f337 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:34.595829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e984a-ff9d-469b-8260-3fa6ec91f337 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:34.630244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:34.631054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:34.631344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:34.631617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:34.632281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:34.632583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:34.633990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:34.642513 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 513/1035] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 17 20:35:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:34.648402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:34.763409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab368b-18de-467a-ae5b-a4d80f465746 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:34.786628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:34.799454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:34.800092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:34.800369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:34.800637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:34.811595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:34.813056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:34.827072 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:34.828117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:34.829035 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:34.829035 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:34.829035 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:34.833009 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 523/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:34.834503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:34.887668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:34.941707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9b6f4-50fa-44ec-80a4-fff15f4859d5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:34.975709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:34.976315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:34.976592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:34.976868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:34.977537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:34.977844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:34.979214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:34.991292 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 514/1037] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:34.996917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:35.106150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1506f3-cffa-4fec-966b-6fe24ffcba24 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:35.140724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:35.141275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:35.141577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:35.141852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:35.149548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:35.151015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:35.162776 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:35.163892 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:35.165429 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:35.168645 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 524/1038] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:35:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:35.169972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:35.275009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f41613-b58b-4fea-9bb2-6ac7bbdf8448 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:35.285298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:35.314533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:35.315317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:35.315698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:35.316063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:35.316922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:35.317326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:35.318953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:35.338860 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 515/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:35:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:35.344726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:35.380091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:35.449688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1405fa-b347-4888-925a-dc735a6c0f27 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:35.483166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d40a2f-d303-4082-8f14-196793980dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:35.483900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d40a2f-d303-4082-8f14-196793980dc4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:35.484283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d40a2f-d303-4082-8f14-196793980dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:35.484652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d40a2f-d303-4082-8f14-196793980dc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:35.551691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d40a2f-d303-4082-8f14-196793980dc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:35.553301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d40a2f-d303-4082-8f14-196793980dc4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:35.585775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:35.653755 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 525/1040] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:35.659985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:35.724308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3d56a-2d11-4dc4-977f-adc38edbceac None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:35.724488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3d56a-2d11-4dc4-977f-adc38edbceac None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:35.724797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3d56a-2d11-4dc4-977f-adc38edbceac None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:35.726648 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 526/1041] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:35.731496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e729-bc4a-441d-8625-009f610af9bd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:35.731767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e729-bc4a-441d-8625-009f610af9bd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:35.732038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e729-bc4a-441d-8625-009f610af9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:35.750363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a863e729-bc4a-441d-8625-009f610af9bd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:35.751545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a863e729-bc4a-441d-8625-009f610af9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:35.772394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6836beb-f3aa-4b69-acc6-eb7036b6f6db None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:35.808582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:35.809254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:35.809593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:35.809828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:35.820005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:35.821487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:35.822055 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 527/1042] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:35.843406 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:35.844427 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:35.845328 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:35.845328 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:35.845328 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:35.849316 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 516/1043] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:35.850420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863e729-bc4a-441d-8625-009f610af9bd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:35.912999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:35.956412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863e729-bc4a-441d-8625-009f610af9bd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:35.994075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:35.994586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:35.994862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:35.995126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:35.995759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:35.996052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:35.997447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:36.013740 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 528/1044] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:36.014678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.037645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:36.122345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed23a6-c05b-432b-81a7-5c5a1bfc094f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:36.149983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.155919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:36.156550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.156820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.157084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.163894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:36.165322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:36.176437 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:36.177354 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:36.178502 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 517/1045] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:36.183264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.260062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb21f9-f2bc-495c-a282-c48aafa57c21 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:36.269626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:36.286307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:36.295313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7239c-661c-4c97-8fff-7812b1644f59 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:36.297866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:36.298620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.298988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.299360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.310512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:36.315302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:36.316466 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:36.317560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:36.321631 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 529/1046] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:36.324183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.338323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:36.339068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.339610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.340068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.341313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:36.341646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:36.343172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:36.357994 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 518/1047] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:36.363769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.424231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b695d-65a6-47b4-9bb0-33850cd2cb8f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:36.458738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd55aa-382c-443b-9404-86e415aa22c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:36.459492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd55aa-382c-443b-9404-86e415aa22c7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.459774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd55aa-382c-443b-9404-86e415aa22c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.460026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd55aa-382c-443b-9404-86e415aa22c7 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.460614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7390cee-5df6-4bc1-ab9e-688ff3cfea4e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:36.460718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd55aa-382c-443b-9404-86e415aa22c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:36.460967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd55aa-382c-443b-9404-86e415aa22c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:36.462381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd55aa-382c-443b-9404-86e415aa22c7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:36.473511 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 530/1048] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:36.473511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78f254-000a-49d4-a9cf-4bb39228a767 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.473511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78f254-000a-49d4-a9cf-4bb39228a767 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.473511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78f254-000a-49d4-a9cf-4bb39228a767 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.474933 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 531/1049] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:36.477611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78f254-000a-49d4-a9cf-4bb39228a767 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.495794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a0217-6d57-4163-87bc-0efb4325c391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:36.496502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a0217-6d57-4163-87bc-0efb4325c391 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.496779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a0217-6d57-4163-87bc-0efb4325c391 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.497050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a0217-6d57-4163-87bc-0efb4325c391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.497809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a0217-6d57-4163-87bc-0efb4325c391 None admin] RBAC: Adding query filter params (role.id=85e89b3e3f63492db730418fb475453d, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=724ce8c58c1b473e9e3af18009532c9f) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:36.498123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a0217-6d57-4163-87bc-0efb4325c391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:36.499559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a0217-6d57-4163-87bc-0efb4325c391 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:36.504918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 519/1050] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/role_assignments?user.id=724ce8c58c1b473e9e3af18009532c9f&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=85e89b3e3f63492db730418fb475453d => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:36.506766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1e836-ffec-4243-bc67-197b639cb597 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.507140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1e836-ffec-4243-bc67-197b639cb597 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.507422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1e836-ffec-4243-bc67-197b639cb597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.521019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:36.528883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:36.531251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-90d1e836-ffec-4243-bc67-197b639cb597 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:36.532428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-90d1e836-ffec-4243-bc67-197b639cb597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:36.595923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78f254-000a-49d4-a9cf-4bb39228a767 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:36.610542 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 520/1051] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:36.633395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:36.634084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.634364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.634631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] PATH_INFO: `/v3/users/b3bf8ad3d952424d993770bfe1ba77e2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.651007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] RBAC: Authorizing `identity:get_user(user_id=b3bf8ad3d952424d993770bfe1ba77e2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:36.653773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:36.656575 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 532/1052] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/users/b3bf8ad3d952424d993770bfe1ba77e2 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:36.662486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1e836-ffec-4243-bc67-197b639cb597 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.773011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1e836-ffec-4243-bc67-197b639cb597 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:36.801707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:36.807977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:36.808585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:36.808887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:36.809153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:36.880911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:36.882549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:36.931658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:36.991865 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 521/1053] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:36.998577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:37.031771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:37.047275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942b5ba-f6d6-4953-82e8-625c561baf9e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:37.050782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:37.085899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:37.086527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:37.086798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:37.087062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] PATH_INFO: `/v3/services/image` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:37.087631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:37.089146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:37.101287 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:37.102535 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:35:37.104469 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:37.107878 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 533/1054] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:37.108624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:37.109650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e641-e859-426a-b305-4666a6bf8ac1 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:37.149346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:37.150124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:37.150448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:37.150810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:37.152004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] RBAC: Authorizing `identity:get_project(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:37.153635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:37.155905 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 522/1055] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:35:36 2022] GET /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:37.161657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:37.210900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732a51b-523a-476c-9a1c-be97deb44479 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:37.247458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:37.248424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:37.248897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:37.249177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:37.249850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] RBAC: Adding query filter params (name=image) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:37.250155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:37.251572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:37.257207 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 534/1056] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:37.262886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:37.269840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:37.273554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07495a-361c-4a83-906a-136fd1dfb359 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:37.296648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:37.312467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:37.313084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:37.313353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:37.313670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:37.314392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e, user.id=b3bf8ad3d952424d993770bfe1ba77e2) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:37.314697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:37.316130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:37.321722 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 523/1057] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:37 2022] GET /identity/v3/role_assignments?user.id=b3bf8ad3d952424d993770bfe1ba77e2&scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:37.388118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36403f1-4659-43c5-9ce6-f4670e2a96e4 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:37.425220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d104e2-eee3-4388-9aee-ccc215b4286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:37.425870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d104e2-eee3-4388-9aee-ccc215b4286f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:37.426141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d104e2-eee3-4388-9aee-ccc215b4286f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:37.426408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d104e2-eee3-4388-9aee-ccc215b4286f None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:37.427051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d104e2-eee3-4388-9aee-ccc215b4286f None admin] RBAC: Adding query filter params (type=image) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:37.427340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d104e2-eee3-4388-9aee-ccc215b4286f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:37.428723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d104e2-eee3-4388-9aee-ccc215b4286f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:37.434566 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 535/1058] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:37.440475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:37.542661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939f721-700e-4893-96be-5e873253b2b7 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:37.579321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b105d-0154-4ecb-ae1f-0390d4819e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:37.580034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b105d-0154-4ecb-ae1f-0390d4819e65 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:37.580410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b105d-0154-4ecb-ae1f-0390d4819e65 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:37.580781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b105d-0154-4ecb-ae1f-0390d4819e65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:37.581631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b105d-0154-4ecb-ae1f-0390d4819e65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:37.583159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b105d-0154-4ecb-ae1f-0390d4819e65 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:37.609510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 524/1059] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:35:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:38.904649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fd2ce-c994-4390-a7d9-344a12d2c7df None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:38.905190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fd2ce-c994-4390-a7d9-344a12d2c7df None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:38.905190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fd2ce-c994-4390-a7d9-344a12d2c7df None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:38.909507 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 536/1060] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:38.912579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c753cd-a46b-45e7-8068-9d9c8a2e9f06 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:38.912981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c753cd-a46b-45e7-8068-9d9c8a2e9f06 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:38.913204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c753cd-a46b-45e7-8068-9d9c8a2e9f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:38.932831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-21c753cd-a46b-45e7-8068-9d9c8a2e9f06 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:38.934062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21c753cd-a46b-45e7-8068-9d9c8a2e9f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:38.949676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:39.017581 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 525/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:39.337222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fd2ce-c994-4390-a7d9-344a12d2c7df None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:39.437567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:39.445919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fd2ce-c994-4390-a7d9-344a12d2c7df None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:39.479871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:39.480529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:39.480807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:39.481079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:39.491784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:39.496585 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:39.497812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:39.499191 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 537/1062] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 17 20:35:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:39.503782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c753cd-a46b-45e7-8068-9d9c8a2e9f06 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:39.604478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c753cd-a46b-45e7-8068-9d9c8a2e9f06 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:39.644450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b22c86-9c89-4631-a960-5b4a604ae1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:39.645059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b22c86-9c89-4631-a960-5b4a604ae1d4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:39.645328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b22c86-9c89-4631-a960-5b4a604ae1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:39.645626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b22c86-9c89-4631-a960-5b4a604ae1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:39.646291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b22c86-9c89-4631-a960-5b4a604ae1d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:39.646588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b22c86-9c89-4631-a960-5b4a604ae1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:39.647955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b22c86-9c89-4631-a960-5b4a604ae1d4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:39.656880 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 526/1063] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 17 20:35:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:39.662616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:39.749673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adc82f-781f-488d-9cc1-d60b76a1e206 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:39.750015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adc82f-781f-488d-9cc1-d60b76a1e206 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:39.750603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adc82f-781f-488d-9cc1-d60b76a1e206 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:39.752406 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 527/1064] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:39.757643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252584e-28fc-4868-b988-2f54a610e3d7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:39.757998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252584e-28fc-4868-b988-2f54a610e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:39.758330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252584e-28fc-4868-b988-2f54a610e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:39.770117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94f715-6675-4934-91c2-67badce39897 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:39.776359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b252584e-28fc-4868-b988-2f54a610e3d7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:39.777590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b252584e-28fc-4868-b988-2f54a610e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:39.779552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:39.807444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:39.808076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:39.808406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:39.808673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:39.819703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:39.821164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:39.836768 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:39.838027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:39.838940 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:39.838940 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:39.838940 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:39.842749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 538/1065] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:39.843573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:39.854572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:39.861613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 528/1066] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:39.944800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb2063-edca-40d1-bb22-cc4be5f93a1f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:39.978579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e20b8e-78c6-46ba-b359-ddadd2ee4ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:39.979181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e20b8e-78c6-46ba-b359-ddadd2ee4ae8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:39.979449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e20b8e-78c6-46ba-b359-ddadd2ee4ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:39.979711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e20b8e-78c6-46ba-b359-ddadd2ee4ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:39.980352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e20b8e-78c6-46ba-b359-ddadd2ee4ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:39.980681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e20b8e-78c6-46ba-b359-ddadd2ee4ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:39.982099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e20b8e-78c6-46ba-b359-ddadd2ee4ae8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:39.993839 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 539/1067] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:39.999637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252584e-28fc-4868-b988-2f54a610e3d7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.048869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d324-7d5f-4867-9718-d0d9d511a5aa None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.049216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d324-7d5f-4867-9718-d0d9d511a5aa None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.049425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d324-7d5f-4867-9718-d0d9d511a5aa None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.050879 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 540/1068] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:35:40.056586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d202b-07f9-4f43-b6f5-73790ea9c4b5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.056865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d202b-07f9-4f43-b6f5-73790ea9c4b5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.057139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d202b-07f9-4f43-b6f5-73790ea9c4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.076651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5b1d202b-07f9-4f43-b6f5-73790ea9c4b5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:40.078062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5b1d202b-07f9-4f43-b6f5-73790ea9c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:40.108381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252584e-28fc-4868-b988-2f54a610e3d7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.140412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.145114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.145794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.146071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.146354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.149522 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 541/1069] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:35:40.164306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.165881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.171228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.175468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d202b-07f9-4f43-b6f5-73790ea9c4b5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:40.180371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:40.181350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:40.186813 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 529/1070] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:35:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:40.189716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.283340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d202b-07f9-4f43-b6f5-73790ea9c4b5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.290279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da54e7-890d-4b76-9dbf-af02478d9573 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.318953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.319875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.320405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.320945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.323286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.323889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.324163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.324492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.325222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:40.325554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.326981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.333097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:40.338318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:40.339248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:40.340182 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:40.341518 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 542/1071] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:40.344965 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 530/1072] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:40.347995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.350306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.408062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.418641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.450945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a472ffc-ac11-44cd-a9b6-b5e2278e18ed None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.465870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ebca1-1c52-48b4-b9ed-e94ff5f930e4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.488302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.488906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.489175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.489440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.490131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:40.490428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.491792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.499669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 543/1073] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:40.499800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.500421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.500534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.500792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.500964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.513263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.514734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:40.529063 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:40.530077 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:40.531941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:40.531941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:40.531941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:40.531941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 531/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:40.531941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.604503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ef1e-9206-48b2-b2fc-f9c9eddfbbe5 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.623650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.630992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23570b-435b-4b2c-bc6d-881a9e66f5d4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.638526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.639140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.639419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.639692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.640078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.667134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.667674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.667948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.668254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] PATH_INFO: `/v3/users/b3bf8ad3d952424d993770bfe1ba77e2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.669379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] RBAC: Authorizing `identity:get_user(user_id=b3bf8ad3d952424d993770bfe1ba77e2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.670862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.672476 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 532/1075] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/users/b3bf8ad3d952424d993770bfe1ba77e2 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:40.673240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.710238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.711686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.771768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02a5a9-21fc-49d4-ac7b-402ff919290f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.807247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.807841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.808112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.808378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.809100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:40.809401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.810019 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 544/1076] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:40.810794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.810899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.822914 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 533/1077] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:40.823710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.834780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.857628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:40.909986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9e758-01a4-4ee6-ac89-097c56bd6887 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.922043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cabaa1-12b6-475c-ad8c-2599d52415de None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:40.944040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.944664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.944939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.945509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.946397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] RBAC: Authorizing `identity:get_project(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.947841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.949549 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 545/1078] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:40.950301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:40.959775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:40.960747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:40.961021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:40.961286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:40.972887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:40.974388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:40.989146 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:40.990246 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:40.991165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:40.991165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:40.991165 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:40.991165 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 534/1079] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:40.991165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:41.038141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:41.050646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa721edb-b2fc-4931-b123-a8fbf5a0b27f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:41.077944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:41.083932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:41.084583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:41.084851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:41.085117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:41.090622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cdfb8-c0bd-4977-8530-e41401bbba4b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:41.091609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:41.093046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:41.103812 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:41.104907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 546/1080] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:41.105955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:41.129454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:41.130145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:41.130419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:41.130688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e/users/b3bf8ad3d952424d993770bfe1ba77e2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:41.132566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] RBAC: Authorizing `identity:create_grant(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e, user_id=b3bf8ad3d952424d993770bfe1ba77e2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:41.134056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:41.148451 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 535/1081] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:35:40 2022] PUT /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e/users/b3bf8ad3d952424d993770bfe1ba77e2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:35:41.149415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:41.212283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d38b9-3401-415b-8276-60af7703acc4 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:41.252434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:41.253105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:41.253384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:41.253685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:41.254347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:41.254652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:41.256052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:41.269613 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 547/1082] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:41.274133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5313d-b446-47bd-8164-44e7624395ca None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:41.314692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a9d00-a7e4-4693-9e0e-61d551c87fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:41.315332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a9d00-a7e4-4693-9e0e-61d551c87fd2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:41.315602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a9d00-a7e4-4693-9e0e-61d551c87fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:41.315876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a9d00-a7e4-4693-9e0e-61d551c87fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:41.316597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a9d00-a7e4-4693-9e0e-61d551c87fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:41.316902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a9d00-a7e4-4693-9e0e-61d551c87fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:41.318444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a9d00-a7e4-4693-9e0e-61d551c87fd2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:41.337519 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 536/1083] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:41.340956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:41.353991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:41.460778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb4dc8-296d-4865-a3ec-64e6ccde74df None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:41.502803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec4cab-cd87-4905-b2c4-3d920f65cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:41.503412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec4cab-cd87-4905-b2c4-3d920f65cde8 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:41.503679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec4cab-cd87-4905-b2c4-3d920f65cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:41.503948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec4cab-cd87-4905-b2c4-3d920f65cde8 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/724ce8c58c1b473e9e3af18009532c9f/roles/85e89b3e3f63492db730418fb475453d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:41.506180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec4cab-cd87-4905-b2c4-3d920f65cde8 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=724ce8c58c1b473e9e3af18009532c9f, role_id=85e89b3e3f63492db730418fb475453d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:41.507629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec4cab-cd87-4905-b2c4-3d920f65cde8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:41.522393 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 548/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:35:41 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/724ce8c58c1b473e9e3af18009532c9f/roles/85e89b3e3f63492db730418fb475453d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:35:43.590510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1881606-34a9-412c-bb89-d8b0c1441b35 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:43.591062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1881606-34a9-412c-bb89-d8b0c1441b35 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:43.591062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1881606-34a9-412c-bb89-d8b0c1441b35 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:43.592405 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 537/1085] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:43.597914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9b93-ea64-4539-b6f8-29e64c12c34b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:43.598307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9b93-ea64-4539-b6f8-29e64c12c34b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:43.598544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d9b93-ea64-4539-b6f8-29e64c12c34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:43.618930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-502d9b93-ea64-4539-b6f8-29e64c12c34b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:43.620118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-502d9b93-ea64-4539-b6f8-29e64c12c34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:43.685843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:43.692274 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 549/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:43.893179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717975-0e97-498a-8ce8-0ce57a99ed28 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:43.893421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717975-0e97-498a-8ce8-0ce57a99ed28 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:43.893748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d717975-0e97-498a-8ce8-0ce57a99ed28 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:43.895094 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 538/1087] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:43.900422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7d102-2bf6-47ce-8d9a-b958c5802498 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:43.900701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7d102-2bf6-47ce-8d9a-b958c5802498 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:43.900970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7d102-2bf6-47ce-8d9a-b958c5802498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:43.920731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-29e7d102-2bf6-47ce-8d9a-b958c5802498 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:43.921957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-29e7d102-2bf6-47ce-8d9a-b958c5802498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:43.991526 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 550/1088] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:44.011696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d717975-0e97-498a-8ce8-0ce57a99ed28 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.023305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:44.119205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d717975-0e97-498a-8ce8-0ce57a99ed28 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.152267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.152910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.153184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.153454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.164102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:44.169070 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:44.170086 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:44.170970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 539/1089] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:44.176499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7d102-2bf6-47ce-8d9a-b958c5802498 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.280024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7d102-2bf6-47ce-8d9a-b958c5802498 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.309541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:44.310920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.318534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.319155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.319424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.319697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.320379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:44.320701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:44.322110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:44.335556 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 551/1090] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:44.336582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.378618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:44.414120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5327e-07c2-4b1f-b6f3-15f2f849c1c9 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.437331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713551a-6b97-4596-a93b-0637c0f20bca None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.448724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.449356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.449666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.449929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.460773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:35:44.465698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:44.466850 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:44.467698 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 540/1091] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:44.472851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.475111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.475748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.476075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.476343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] PATH_INFO: `/v3/users/b3bf8ad3d952424d993770bfe1ba77e2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.477596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] RBAC: Authorizing `identity:get_user(user_id=b3bf8ad3d952424d993770bfe1ba77e2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:44.479038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:44.480647 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 552/1092] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/users/b3bf8ad3d952424d993770bfe1ba77e2 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:44.486285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.545032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:44.572641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79a8c5-a4bb-4e1c-b21a-ed70f39eb39a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.585641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8ff66-44f6-43b7-a501-99827f22f6d7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.598256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:44.606987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.607614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.607882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.608153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.608803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:44.609122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:44.610542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:44.618353 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 541/1093] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:44.618503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.618936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.619216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.619483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.624256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.688752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:44.690129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:44.726459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826ef8b-8934-440b-9d78-26ed8f881c76 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.746503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:44.759702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.760336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.760610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.760883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.772397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:44.773886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:44.787941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:44.789028 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:44.789972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:44.789972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:44.789972 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:44.789972 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 542/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:44.790660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 553/1095] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:44.796383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.796509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.839363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:44.891986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ca1bd-dae5-411d-99c4-0b761a404184 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.895207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb1471-02e8-41e8-82cf-1af1da427e3a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:44.925016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.925664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.925936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.926203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.927306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] RBAC: Authorizing `identity:get_project(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:44.927927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:44.928532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:44.928743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:44.928853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:44.929066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:44.929719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:44.930036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:44.930561 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 554/1096] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:44.931381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:44.936487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.947601 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 543/1097] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:44.948495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:44.962413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:45.034869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30846529-da42-42af-9fa1-2dd4f2e2ef4d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:45.045757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7625e-a34a-4b39-99b1-8d556a739fd0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:45.047265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:45.069296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:45.069947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:45.070225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:45.070500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:45.071225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e, user.id=b3bf8ad3d952424d993770bfe1ba77e2) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:45.071530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:45.072982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:45.078773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 555/1098] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/role_assignments?user.id=b3bf8ad3d952424d993770bfe1ba77e2&scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:45.079772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:45.080393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:45.080664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:45.080927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:45.088342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:45.089896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:35:45.102655 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:45.103703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:35:45.104699 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 544/1099] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 17 20:35:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:35:45.110272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:45.216173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:45.228056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab54aa6-b324-4db9-aa98-a573e55dfdbc None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:45.267869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:45.268487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:45.268747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:45.269017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:45.269794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:45.270114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:45.271609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:45.291529 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 556/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 17 20:35:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:45.297715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:45.399399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f1cb2-5544-4af0-b653-88d01de60950 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:45.433567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:45.434214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:45.434489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:45.434724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:45.464697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:45.508330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:45.510545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:45.623564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 545/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:45.629609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:45.731997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd5bd-f23a-4bb9-872a-738788eee483 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:45.766381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:45.766944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:45.767208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:45.767475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:45.777612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:45.779096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:45.787751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:35:45.794315 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:45.795318 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:35:45.796239 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:35:45.796239 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:35:45.796239 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:45.797630 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 557/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:45.801451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:45.874189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:45.919361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59d3d-db1c-4363-86d6-9ef5f3f158a3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:45.956476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:45.957075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:45.957350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:45.957649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:45.958279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:45.958579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:45.960020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:45.971398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 546/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:45.977398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:46.083790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe85fea-9e1f-4060-9f33-19ecdfd1c44d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:46.119179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:46.119801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:46.120076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:46.120340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:46.127351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:46.128803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:35:46.140157 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:35:46.141249 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:35:46.142426 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:35:46.145647 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 558/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:35:46.147238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:46.260949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4905ac-89d7-431e-8855-e22724e53b19 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:46.284114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:46.296381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2ca5f-df4d-4a2a-bb12-8e5a4ac61ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:46.296989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ca5f-df4d-4a2a-bb12-8e5a4ac61ca1 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:46.297264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ca5f-df4d-4a2a-bb12-8e5a4ac61ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:46.297564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ca5f-df4d-4a2a-bb12-8e5a4ac61ca1 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:46.298249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2ca5f-df4d-4a2a-bb12-8e5a4ac61ca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:46.298551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2ca5f-df4d-4a2a-bb12-8e5a4ac61ca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:46.299975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2ca5f-df4d-4a2a-bb12-8e5a4ac61ca1 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:46.312034 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 547/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:35:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:46.318271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:46.379222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:46.422475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b59a-241b-4042-b796-c41dc8ea30e7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:46.456243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f50e2-dc03-44ed-abb5-a45b774abaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:46.456970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f50e2-dc03-44ed-abb5-a45b774abaab None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:46.457336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f50e2-dc03-44ed-abb5-a45b774abaab None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:46.457757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f50e2-dc03-44ed-abb5-a45b774abaab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:46.458485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f50e2-dc03-44ed-abb5-a45b774abaab None admin] RBAC: Adding query filter params (role.id=85e89b3e3f63492db730418fb475453d, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=724ce8c58c1b473e9e3af18009532c9f) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:46.458792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f50e2-dc03-44ed-abb5-a45b774abaab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:46.460264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f50e2-dc03-44ed-abb5-a45b774abaab None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:46.466775 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 559/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:46 2022] GET /identity/v3/role_assignments?user.id=724ce8c58c1b473e9e3af18009532c9f&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=85e89b3e3f63492db730418fb475453d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:47.498845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de4121-f720-4751-8a22-6ec14ec609aa None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:47.499466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de4121-f720-4751-8a22-6ec14ec609aa None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:47.499466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de4121-f720-4751-8a22-6ec14ec609aa None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:47.501385 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 548/1107] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:47.511677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d964-ccf7-4eb9-a5fd-a1eb1278949e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:47.512414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d964-ccf7-4eb9-a5fd-a1eb1278949e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:47.512602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d964-ccf7-4eb9-a5fd-a1eb1278949e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:47.539473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6227d964-ccf7-4eb9-a5fd-a1eb1278949e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:47.540837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6227d964-ccf7-4eb9-a5fd-a1eb1278949e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:47.616642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 560/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:47.944190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de4121-f720-4751-8a22-6ec14ec609aa None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:48.057213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de4121-f720-4751-8a22-6ec14ec609aa None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:48.094914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517250ad-8bd1-4e4b-ab96-260cb332583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:48.095534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517250ad-8bd1-4e4b-ab96-260cb332583a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:48.095809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517250ad-8bd1-4e4b-ab96-260cb332583a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:48.096074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517250ad-8bd1-4e4b-ab96-260cb332583a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:48.097318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517250ad-8bd1-4e4b-ab96-260cb332583a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:48.098780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517250ad-8bd1-4e4b-ab96-260cb332583a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:48.100330 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 549/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:35:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:48.106469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227d964-ccf7-4eb9-a5fd-a1eb1278949e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:48.162442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:48.209259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227d964-ccf7-4eb9-a5fd-a1eb1278949e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:48.244977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad916a-4e60-4500-ac67-6cd9d952432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:48.245614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad916a-4e60-4500-ac67-6cd9d952432e None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:48.245889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad916a-4e60-4500-ac67-6cd9d952432e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:48.246263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad916a-4e60-4500-ac67-6cd9d952432e None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:48.246882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad916a-4e60-4500-ac67-6cd9d952432e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:48.248316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad916a-4e60-4500-ac67-6cd9d952432e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:48.270535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 561/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:35:48 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:35:50.694784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbab135-7530-4f9e-afc7-6721ab9755be None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:50.695255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbab135-7530-4f9e-afc7-6721ab9755be None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:50.695349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbab135-7530-4f9e-afc7-6721ab9755be None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:50.696673 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 550/1111] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:50.702268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda51b-9887-40fb-958e-88fa86cd8a19 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:50.702712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda51b-9887-40fb-958e-88fa86cd8a19 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:50.702805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda51b-9887-40fb-958e-88fa86cd8a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:50.721890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-90cda51b-9887-40fb-958e-88fa86cd8a19 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:50.723081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-90cda51b-9887-40fb-958e-88fa86cd8a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:50.793331 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 562/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:51.115280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbab135-7530-4f9e-afc7-6721ab9755be None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:51.169644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:51.221694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbab135-7530-4f9e-afc7-6721ab9755be None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:51.258586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:51.259211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:51.259483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:51.261697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] PATH_INFO: `/v3/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:51.262887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] RBAC: Authorizing `identity:get_role(role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:51.264289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:51.268272 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 551/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:51 2022] GET /identity/v3/roles/1b7585de1b6442509e3788682caf8774 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:51.274990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda51b-9887-40fb-958e-88fa86cd8a19 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:51.320392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:51.382615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda51b-9887-40fb-958e-88fa86cd8a19 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:51.418503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:51.419127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:51.419395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:51.419658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] PATH_INFO: `/v3/users/e5c0147874e641af9170679020c4cf11` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:51.434768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] RBAC: Authorizing `identity:get_user(user_id=e5c0147874e641af9170679020c4cf11)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:51.436218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:51.437827 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 563/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:51 2022] GET /identity/v3/users/e5c0147874e641af9170679020c4cf11 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:51.443986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:51.556983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399c7de-6fe9-4c64-af19-150f974a454b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:51.577300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:51.598070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:51.598683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:51.598962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:51.599190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:51.673430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:51.675608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:51.786391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 552/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:51.793699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:51.892088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:51.896206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a81a-0999-4bed-b76f-e0802412391a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:51.935209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7672437-5e14-4642-9d3c-bf642ef83acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:51.935769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7672437-5e14-4642-9d3c-bf642ef83acd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:51.936042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7672437-5e14-4642-9d3c-bf642ef83acd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:51.936310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7672437-5e14-4642-9d3c-bf642ef83acd None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:51.937418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7672437-5e14-4642-9d3c-bf642ef83acd None admin] RBAC: Authorizing `identity:get_project(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:51.938895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7672437-5e14-4642-9d3c-bf642ef83acd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:51.940533 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 564/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:35:51 2022] GET /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:51.947212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:51.966665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:52.053216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846894a-e81c-4db3-8268-c1478178028e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:52.092640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b30c8-3967-4af0-a87e-bc29c2088d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:52.093228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b30c8-3967-4af0-a87e-bc29c2088d7a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:52.093509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b30c8-3967-4af0-a87e-bc29c2088d7a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:52.093786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b30c8-3967-4af0-a87e-bc29c2088d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:52.094501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b30c8-3967-4af0-a87e-bc29c2088d7a None admin] RBAC: Adding query filter params (role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e, user.id=e5c0147874e641af9170679020c4cf11) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:52.094796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b30c8-3967-4af0-a87e-bc29c2088d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:52.096222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b30c8-3967-4af0-a87e-bc29c2088d7a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:52.101638 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 553/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:51 2022] GET /identity/v3/role_assignments?user.id=e5c0147874e641af9170679020c4cf11&scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e&role.id=1b7585de1b6442509e3788682caf8774 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:54.443116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292027d0-c7e1-4fe9-a32c-50418598dbe6 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:54.443677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292027d0-c7e1-4fe9-a32c-50418598dbe6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:54.443677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292027d0-c7e1-4fe9-a32c-50418598dbe6 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:54.445013 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 565/1118] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:54.451523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89a826-879d-40cf-b287-07f32ba7cb27 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:54.452210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89a826-879d-40cf-b287-07f32ba7cb27 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:54.452355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89a826-879d-40cf-b287-07f32ba7cb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:54.475990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ba89a826-879d-40cf-b287-07f32ba7cb27 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:54.477171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ba89a826-879d-40cf-b287-07f32ba7cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:54.553077 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 554/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:54.877206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292027d0-c7e1-4fe9-a32c-50418598dbe6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:54.995459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292027d0-c7e1-4fe9-a32c-50418598dbe6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:55.030715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:55.031321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:55.031589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:55.031853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] PATH_INFO: `/v3/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:55.033338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] RBAC: Authorizing `identity:get_role(role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:55.034750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:55.036047 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 566/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:54 2022] GET /identity/v3/roles/1b7585de1b6442509e3788682caf8774 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:55.045062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89a826-879d-40cf-b287-07f32ba7cb27 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:55.061313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:55.152458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89a826-879d-40cf-b287-07f32ba7cb27 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:55.187569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:55.188181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:55.188456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:55.188718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] PATH_INFO: `/v3/users/e5c0147874e641af9170679020c4cf11` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:55.189893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] RBAC: Authorizing `identity:get_user(user_id=e5c0147874e641af9170679020c4cf11)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:55.191314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:55.192878 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 555/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:55 2022] GET /identity/v3/users/e5c0147874e641af9170679020c4cf11 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:55.198772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:55.238158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:55.324850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9eaf42-37fe-4ea8-9642-4c7fff8a8628 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:55.360956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c55595-fd8e-4817-ad7e-a364072e568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:55.361602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55595-fd8e-4817-ad7e-a364072e568e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:55.361883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55595-fd8e-4817-ad7e-a364072e568e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:55.362149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c55595-fd8e-4817-ad7e-a364072e568e None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:55.363380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c55595-fd8e-4817-ad7e-a364072e568e None admin] RBAC: Authorizing `identity:get_project(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:55.364852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c55595-fd8e-4817-ad7e-a364072e568e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:55.366524 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 567/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:35:55 2022] GET /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:55.376085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:55.459488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:55.481959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa435983-bb96-4399-b7b5-266b057f5995 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:55.516443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1d3df-d75b-4990-8977-77d56ebe732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:55.517096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1d3df-d75b-4990-8977-77d56ebe732a None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:55.517369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1d3df-d75b-4990-8977-77d56ebe732a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:55.517685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1d3df-d75b-4990-8977-77d56ebe732a None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e/users/e5c0147874e641af9170679020c4cf11/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:55.519850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1d3df-d75b-4990-8977-77d56ebe732a None admin] RBAC: Authorizing `identity:create_grant(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e, user_id=e5c0147874e641af9170679020c4cf11, role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:55.521286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1d3df-d75b-4990-8977-77d56ebe732a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:55.536769 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 556/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:35:55 2022] PUT /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e/users/e5c0147874e641af9170679020c4cf11/roles/1b7585de1b6442509e3788682caf8774 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:35:57.902070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0288d3-e931-4485-8482-9a208d088414 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:57.903158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0288d3-e931-4485-8482-9a208d088414 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:57.903158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0288d3-e931-4485-8482-9a208d088414 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:57.905519 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 568/1124] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:35:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:35:57.911890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f506-dcce-4c36-9779-b9387443df97 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:57.912362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f506-dcce-4c36-9779-b9387443df97 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:57.912463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f506-dcce-4c36-9779-b9387443df97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:57.939943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bbe8f506-dcce-4c36-9779-b9387443df97 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:35:57.941191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bbe8f506-dcce-4c36-9779-b9387443df97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:35:58.012054 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 557/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:35:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:35:58.334455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0288d3-e931-4485-8482-9a208d088414 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:58.460203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0288d3-e931-4485-8482-9a208d088414 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:58.461993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:58.498310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:58.498932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:58.499203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:58.499474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] PATH_INFO: `/v3/roles/1b7585de1b6442509e3788682caf8774` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:58.500900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] RBAC: Authorizing `identity:get_role(role_id=1b7585de1b6442509e3788682caf8774)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:58.502317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:58.503642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 569/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:58 2022] GET /identity/v3/roles/1b7585de1b6442509e3788682caf8774 => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:58.509820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8f506-dcce-4c36-9779-b9387443df97 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:58.578104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:58.611499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8f506-dcce-4c36-9779-b9387443df97 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:58.648070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:58.648685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:58.648958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:58.649223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] PATH_INFO: `/v3/users/e5c0147874e641af9170679020c4cf11` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:58.650485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] RBAC: Authorizing `identity:get_user(user_id=e5c0147874e641af9170679020c4cf11)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:58.651908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:58.653494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 558/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:35:58 2022] GET /identity/v3/users/e5c0147874e641af9170679020c4cf11 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:35:58.661035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:58.772921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361730e-5d4b-414a-bf20-a20547a76119 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:58.810959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:58.811556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:58.811830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:58.812098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:58.852966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:58.881861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:58.883315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:58.988211 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 570/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:35:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:35:58.994523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:59.118001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a857ab8-3e4c-4b01-9334-728692cd1291 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:59.158127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd30bf-90f0-44b9-8f75-4a1007c5e27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:59.158676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd30bf-90f0-44b9-8f75-4a1007c5e27b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:59.158949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd30bf-90f0-44b9-8f75-4a1007c5e27b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:59.159214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd30bf-90f0-44b9-8f75-4a1007c5e27b None admin] PATH_INFO: `/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:59.160380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd30bf-90f0-44b9-8f75-4a1007c5e27b None admin] RBAC: Authorizing `identity:get_project(project_id=d51910e4ef694a3c88aaa9a1ce11ad2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:59.161849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd30bf-90f0-44b9-8f75-4a1007c5e27b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:59.163468 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 559/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:35:58 2022] GET /identity/v3/projects/d51910e4ef694a3c88aaa9a1ce11ad2e => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:35:59.170239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:35:59.251792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:35:59.285612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7aee48-e9fd-44cb-a489-03f8212600d9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:35:59.322036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e3a2-729b-46ab-944c-d046cd780daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:35:59.322630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e3a2-729b-46ab-944c-d046cd780daa None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:35:59.322908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e3a2-729b-46ab-944c-d046cd780daa None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:35:59.323180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e3a2-729b-46ab-944c-d046cd780daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:35:59.323901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e3a2-729b-46ab-944c-d046cd780daa None admin] RBAC: Adding query filter params (role.id=1b7585de1b6442509e3788682caf8774, scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e, user.id=e5c0147874e641af9170679020c4cf11) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:35:59.324252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e3a2-729b-46ab-944c-d046cd780daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:35:59.325726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e3a2-729b-46ab-944c-d046cd780daa None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:35:59.332456 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 571/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:35:59 2022] GET /identity/v3/role_assignments?user.id=e5c0147874e641af9170679020c4cf11&scope.project.id=d51910e4ef694a3c88aaa9a1ce11ad2e&role.id=1b7585de1b6442509e3788682caf8774 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:01.736267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701944-0bfc-420d-9d17-cbc3f53a7ca7 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:01.736781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701944-0bfc-420d-9d17-cbc3f53a7ca7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:01.736781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701944-0bfc-420d-9d17-cbc3f53a7ca7 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:01.738212 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 560/1131] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:01.744670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235959e-a3e9-4296-82ca-a0c2a4902316 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:01.745138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235959e-a3e9-4296-82ca-a0c2a4902316 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:01.745240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235959e-a3e9-4296-82ca-a0c2a4902316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:01.773218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9235959e-a3e9-4296-82ca-a0c2a4902316 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:01.778273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9235959e-a3e9-4296-82ca-a0c2a4902316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:01.860880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 572/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:02.185826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701944-0bfc-420d-9d17-cbc3f53a7ca7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:02.196603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:02.296883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701944-0bfc-420d-9d17-cbc3f53a7ca7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:02.332406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470585f-e99c-4abe-8c6c-93070196ae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:02.333020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470585f-e99c-4abe-8c6c-93070196ae89 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:02.333293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470585f-e99c-4abe-8c6c-93070196ae89 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:02.333587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470585f-e99c-4abe-8c6c-93070196ae89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:02.334834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470585f-e99c-4abe-8c6c-93070196ae89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:02.336277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470585f-e99c-4abe-8c6c-93070196ae89 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:02.337861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 561/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:36:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:02.344752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235959e-a3e9-4296-82ca-a0c2a4902316 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:02.414102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:02.455768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235959e-a3e9-4296-82ca-a0c2a4902316 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:02.491103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe0d65-ea99-4eed-853a-32426ce7ad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:02.491705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe0d65-ea99-4eed-853a-32426ce7ad66 None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:02.491981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe0d65-ea99-4eed-853a-32426ce7ad66 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:02.492256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe0d65-ea99-4eed-853a-32426ce7ad66 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:02.492903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe0d65-ea99-4eed-853a-32426ce7ad66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:02.494394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe0d65-ea99-4eed-853a-32426ce7ad66 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:02.528215 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 573/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:36:02 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:36:04.956698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2ca0b-456f-4741-a31c-c46e20fd5731 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:04.957230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2ca0b-456f-4741-a31c-c46e20fd5731 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:04.957230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2ca0b-456f-4741-a31c-c46e20fd5731 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:04.958527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 562/1135] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:04.965056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad483f51-d855-4204-8749-781a64e8e549 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:04.965909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad483f51-d855-4204-8749-781a64e8e549 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:04.966020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad483f51-d855-4204-8749-781a64e8e549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:04.985882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ad483f51-d855-4204-8749-781a64e8e549 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:04.987082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ad483f51-d855-4204-8749-781a64e8e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:05.061018 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 574/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:05.380746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2ca0b-456f-4741-a31c-c46e20fd5731 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:05.459731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:05.488074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2ca0b-456f-4741-a31c-c46e20fd5731 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:05.524877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca1c07-72b1-4c05-947a-ad2c1673d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:05.525511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca1c07-72b1-4c05-947a-ad2c1673d67d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:05.525811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca1c07-72b1-4c05-947a-ad2c1673d67d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:05.526079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca1c07-72b1-4c05-947a-ad2c1673d67d None admin] PATH_INFO: `/v3/domains/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:05.527351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca1c07-72b1-4c05-947a-ad2c1673d67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:05.528905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca1c07-72b1-4c05-947a-ad2c1673d67d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:05.530901 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 563/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:36:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:05.537082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad483f51-d855-4204-8749-781a64e8e549 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:05.589795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:05.644989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad483f51-d855-4204-8749-781a64e8e549 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:05.680244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9f649-d63e-43f9-9081-77240e52b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:05.680861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9f649-d63e-43f9-9081-77240e52b4aa None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:05.681138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9f649-d63e-43f9-9081-77240e52b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:05.681408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9f649-d63e-43f9-9081-77240e52b4aa None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:05.682083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9f649-d63e-43f9-9081-77240e52b4aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:05.683546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9f649-d63e-43f9-9081-77240e52b4aa None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:05.714360 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 575/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:36:05 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:36:08.149701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b90ea-bf37-4472-b5ea-ee55b2ca0a2a None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:08.150250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b90ea-bf37-4472-b5ea-ee55b2ca0a2a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:08.150555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b90ea-bf37-4472-b5ea-ee55b2ca0a2a None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:08.151946 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 564/1139] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:08.159063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a0bb-a83e-49c8-8ff7-6082be732cd6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:08.161921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a0bb-a83e-49c8-8ff7-6082be732cd6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:08.161921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a0bb-a83e-49c8-8ff7-6082be732cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:08.186666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-32c3a0bb-a83e-49c8-8ff7-6082be732cd6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:08.187858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-32c3a0bb-a83e-49c8-8ff7-6082be732cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:08.258843 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 576/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:08.584950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b90ea-bf37-4472-b5ea-ee55b2ca0a2a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:08.710754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b90ea-bf37-4472-b5ea-ee55b2ca0a2a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:08.751647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:08.752549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:08.752922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:08.753280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:08.767304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:36:08.772048 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:36:08.773383 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:36:08.774684 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 565/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:36:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:36:08.779478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a0bb-a83e-49c8-8ff7-6082be732cd6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:08.811599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:08.881996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a0bb-a83e-49c8-8ff7-6082be732cd6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:08.922542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:08.923148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:08.923419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:08.923680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:08.924335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:08.924627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:08.926046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:08.934568 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 577/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:36:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:36:08.941254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:08.948119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:09.047043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806728-b003-4954-9aad-d7b7a4cad4fa None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:09.082212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:09.082827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:09.083102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:09.083371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] PATH_INFO: `/v3/users/8618df021aae428eb87341003db7aafe` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:09.096969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] RBAC: Authorizing `identity:get_user(user_id=8618df021aae428eb87341003db7aafe)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:09.098440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:09.100023 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 566/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:36:08 2022] GET /identity/v3/users/8618df021aae428eb87341003db7aafe => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:09.106254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:09.206032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:09.214035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac062c-7f03-46f4-a185-0c5277e1ab22 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:09.249650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:09.250260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:09.250536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:09.250806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:09.328035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:09.329509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:09.435925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:09.444322 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 578/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:36:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:36:09.450557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:09.512772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:09.566906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb0082-efe0-427f-91a0-e1b47ed8032c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:09.605600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594d076-32ad-4646-85a9-510412ff2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:09.606326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594d076-32ad-4646-85a9-510412ff2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:09.606602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594d076-32ad-4646-85a9-510412ff2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:09.606871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594d076-32ad-4646-85a9-510412ff2fa4 None admin] PATH_INFO: `/v3/projects/96ab9a9396784e80a1bf9ea588531e3b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:09.608135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594d076-32ad-4646-85a9-510412ff2fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=96ab9a9396784e80a1bf9ea588531e3b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:09.609606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594d076-32ad-4646-85a9-510412ff2fa4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:09.611260 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 567/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:36:09 2022] GET /identity/v3/projects/96ab9a9396784e80a1bf9ea588531e3b => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:09.621175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:09.731172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdad1d-12b5-483d-8030-da1071ebf145 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:09.766854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215bec1-8d2b-46dc-9919-ca915f9d95f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:09.767412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215bec1-8d2b-46dc-9919-ca915f9d95f8 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:09.767692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215bec1-8d2b-46dc-9919-ca915f9d95f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:09.767965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215bec1-8d2b-46dc-9919-ca915f9d95f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:09.768699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215bec1-8d2b-46dc-9919-ca915f9d95f8 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=96ab9a9396784e80a1bf9ea588531e3b, user.id=8618df021aae428eb87341003db7aafe) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:09.769009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215bec1-8d2b-46dc-9919-ca915f9d95f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:09.770635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215bec1-8d2b-46dc-9919-ca915f9d95f8 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:09.777317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 579/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:36:09 2022] GET /identity/v3/role_assignments?user.id=8618df021aae428eb87341003db7aafe&scope.project.id=96ab9a9396784e80a1bf9ea588531e3b&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:12.123977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499f77e-1362-49e0-b8ea-fb619283f512 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:12.124537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499f77e-1362-49e0-b8ea-fb619283f512 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:12.124537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499f77e-1362-49e0-b8ea-fb619283f512 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:12.125908 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 568/1147] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:12.132176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b9c9a-f914-4214-8d2e-886d1b5bb722 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:12.132886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b9c9a-f914-4214-8d2e-886d1b5bb722 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:12.133044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b9c9a-f914-4214-8d2e-886d1b5bb722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:12.160561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-588b9c9a-f914-4214-8d2e-886d1b5bb722 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:12.161773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-588b9c9a-f914-4214-8d2e-886d1b5bb722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:12.216896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:12.231340 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 580/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:12.556384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499f77e-1362-49e0-b8ea-fb619283f512 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:12.670822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499f77e-1362-49e0-b8ea-fb619283f512 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:12.699272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:12.712642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:12.713703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:12.714042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:12.714377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:12.730343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:36:12.735452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:36:12.736561 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:36:12.737426 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 569/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:36:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:36:12.742919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b9c9a-f914-4214-8d2e-886d1b5bb722 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:12.849235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b9c9a-f914-4214-8d2e-886d1b5bb722 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:12.884559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:12.885164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:12.885439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:12.885745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:12.886410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:12.886715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:12.888103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:12.897167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 581/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:36:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:12.905873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:13.016377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9cf633-5371-4ed6-b022-63a194b7c8b3 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:13.051739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:13.052352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:13.052629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:13.052900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] PATH_INFO: `/v3/users/8618df021aae428eb87341003db7aafe` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:13.054168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] RBAC: Authorizing `identity:get_user(user_id=8618df021aae428eb87341003db7aafe)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:13.055604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:13.057214 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 570/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:36:12 2022] GET /identity/v3/users/8618df021aae428eb87341003db7aafe => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:13.063515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:13.114698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:13.171971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fa8c0-9779-4d98-8c54-c3e805bbaefe None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:13.207103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7adb5-5293-471e-b1e2-8970800ce673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:13.207725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7adb5-5293-471e-b1e2-8970800ce673 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:13.208002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7adb5-5293-471e-b1e2-8970800ce673 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:13.208274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7adb5-5293-471e-b1e2-8970800ce673 None admin] PATH_INFO: `/v3/projects/96ab9a9396784e80a1bf9ea588531e3b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:13.209449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7adb5-5293-471e-b1e2-8970800ce673 None admin] RBAC: Authorizing `identity:get_project(project_id=96ab9a9396784e80a1bf9ea588531e3b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:13.210928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7adb5-5293-471e-b1e2-8970800ce673 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:13.212556 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 582/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:36:13 2022] GET /identity/v3/projects/96ab9a9396784e80a1bf9ea588531e3b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:13.219176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:13.289372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:13.339290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e19939-58a8-48dc-92ea-40d8667d497c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:13.373962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39715b2a-1829-4fab-a8a8-e069bc09919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:13.374536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39715b2a-1829-4fab-a8a8-e069bc09919f None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:13.374810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39715b2a-1829-4fab-a8a8-e069bc09919f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:13.375079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39715b2a-1829-4fab-a8a8-e069bc09919f None admin] PATH_INFO: `/v3/projects/96ab9a9396784e80a1bf9ea588531e3b/users/8618df021aae428eb87341003db7aafe/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:13.377232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39715b2a-1829-4fab-a8a8-e069bc09919f None admin] RBAC: Authorizing `identity:create_grant(project_id=96ab9a9396784e80a1bf9ea588531e3b, user_id=8618df021aae428eb87341003db7aafe, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:13.378699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39715b2a-1829-4fab-a8a8-e069bc09919f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:13.394991 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 571/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:36:13 2022] PUT /identity/v3/projects/96ab9a9396784e80a1bf9ea588531e3b/users/8618df021aae428eb87341003db7aafe/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:36:15.779664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e735-d50d-4398-99bb-23afc6301864 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:15.780909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e735-d50d-4398-99bb-23afc6301864 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:15.780909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e735-d50d-4398-99bb-23afc6301864 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:15.783146 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 583/1154] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:15.789639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dbf5a-58d5-4b6d-b2c0-ca234c548a1a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:15.790082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dbf5a-58d5-4b6d-b2c0-ca234c548a1a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:15.790196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dbf5a-58d5-4b6d-b2c0-ca234c548a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:15.810951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b87dbf5a-58d5-4b6d-b2c0-ca234c548a1a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:15.812139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b87dbf5a-58d5-4b6d-b2c0-ca234c548a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:15.883726 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 572/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:16.210828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8e735-d50d-4398-99bb-23afc6301864 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:16.330711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8e735-d50d-4398-99bb-23afc6301864 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:16.347673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:16.365215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:16.365888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:16.366163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:16.366413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:16.376837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:36:16.381560 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:36:16.382647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:36:16.386643 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 584/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:36:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:36:16.388912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dbf5a-58d5-4b6d-b2c0-ca234c548a1a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:16.435339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:16.490105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dbf5a-58d5-4b6d-b2c0-ca234c548a1a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:16.524381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:16.524975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:16.525248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:16.525543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:16.526193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:16.526493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:16.527892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:16.536109 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 573/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:36:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:16.542329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:16.641008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cea040-586c-4557-a31a-73c57f94c610 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:16.675736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:16.676357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:16.676628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:16.676896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] PATH_INFO: `/v3/users/8618df021aae428eb87341003db7aafe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:16.678067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] RBAC: Authorizing `identity:get_user(user_id=8618df021aae428eb87341003db7aafe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:16.679504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:16.681069 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 585/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:36:16 2022] GET /identity/v3/users/8618df021aae428eb87341003db7aafe => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:16.687151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:16.803343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab35c-167f-4e63-ae2d-9696c524c342 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:16.812353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:16.848291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:16.848891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:16.849165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:16.849437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:16.917000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:16.918466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:17.020056 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 574/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:36:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:36:17.027021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:17.081199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:17.136447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8224-2a65-46ea-8307-d85443b0da9d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:17.172887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab19661-855a-4118-86c5-ab40eea01baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:17.173501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab19661-855a-4118-86c5-ab40eea01baf None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:17.173807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab19661-855a-4118-86c5-ab40eea01baf None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:17.174071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab19661-855a-4118-86c5-ab40eea01baf None admin] PATH_INFO: `/v3/projects/96ab9a9396784e80a1bf9ea588531e3b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:17.175182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab19661-855a-4118-86c5-ab40eea01baf None admin] RBAC: Authorizing `identity:get_project(project_id=96ab9a9396784e80a1bf9ea588531e3b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:17.176605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab19661-855a-4118-86c5-ab40eea01baf None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:17.178271 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 586/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:36:17 2022] GET /identity/v3/projects/96ab9a9396784e80a1bf9ea588531e3b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:17.184612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:17.194789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:17.288286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ae0fc-4e27-4d4d-9f14-5594f390c580 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:17.328108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508e5d6-2c2c-4391-ba54-61e2131c285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:17.328706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e5d6-2c2c-4391-ba54-61e2131c285c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:17.328978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e5d6-2c2c-4391-ba54-61e2131c285c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:17.329245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e5d6-2c2c-4391-ba54-61e2131c285c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:17.329996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e5d6-2c2c-4391-ba54-61e2131c285c None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=96ab9a9396784e80a1bf9ea588531e3b, user.id=8618df021aae428eb87341003db7aafe) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:17.330302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e5d6-2c2c-4391-ba54-61e2131c285c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:17.331798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e5d6-2c2c-4391-ba54-61e2131c285c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:17.337751 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 575/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:36:17 2022] GET /identity/v3/role_assignments?user.id=8618df021aae428eb87341003db7aafe&scope.project.id=96ab9a9396784e80a1bf9ea588531e3b&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:19.796173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea230b-d4c1-47a5-a975-623ab8e627ae None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:19.796754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea230b-d4c1-47a5-a975-623ab8e627ae None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:19.796979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea230b-d4c1-47a5-a975-623ab8e627ae None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:19.799122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 587/1162] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:36:19.806293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ac2b-f235-42c7-95b8-cf2e1692db62 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:19.806850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ac2b-f235-42c7-95b8-cf2e1692db62 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:19.806850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ac2b-f235-42c7-95b8-cf2e1692db62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:19.826113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b205ac2b-f235-42c7-95b8-cf2e1692db62 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:19.827279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b205ac2b-f235-42c7-95b8-cf2e1692db62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:19.902622 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 576/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:36:20.239255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea230b-d4c1-47a5-a975-623ab8e627ae None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:20.365306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea230b-d4c1-47a5-a975-623ab8e627ae None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:20.379188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:20.410030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:20.410620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:20.410891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:20.411156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:20.485374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:20.486856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:20.589049 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 588/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:36:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 20:36:20.595499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ac2b-f235-42c7-95b8-cf2e1692db62 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:20.597850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:20.705153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ac2b-f235-42c7-95b8-cf2e1692db62 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:20.741074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97248cf-8f11-435f-b820-e87f68a10714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:20.741727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97248cf-8f11-435f-b820-e87f68a10714 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:20.741999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97248cf-8f11-435f-b820-e87f68a10714 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:20.742268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97248cf-8f11-435f-b820-e87f68a10714 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:20.749272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97248cf-8f11-435f-b820-e87f68a10714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:20.750769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97248cf-8f11-435f-b820-e87f68a10714 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-b97248cf-8f11-435f-b820-e87f68a10714 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:36:20.762300 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:36:20.763480 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:36:20.764362 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 577/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:36:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:36:20.771295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:20.789041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:20.876491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62c3b5-fa80-4f06-9218-8872689a5bdf None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:20.917221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b1f1-8907-4e5f-a9f5-637427728506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:20.917857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b1f1-8907-4e5f-a9f5-637427728506 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:20.918129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b1f1-8907-4e5f-a9f5-637427728506 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:20.918395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b1f1-8907-4e5f-a9f5-637427728506 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:20.919037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9b1f1-8907-4e5f-a9f5-637427728506 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:20.919334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9b1f1-8907-4e5f-a9f5-637427728506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:20.920811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9b1f1-8907-4e5f-a9f5-637427728506 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:20.928210 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 589/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 17 20:36:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:23.325341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ed8d8-8404-49d1-8f81-5a95548f6eb3 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:23.326435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ed8d8-8404-49d1-8f81-5a95548f6eb3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:23.326435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ed8d8-8404-49d1-8f81-5a95548f6eb3 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:23.328471 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 578/1167] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:23.334905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3849a3-183e-4490-aab2-95446b660c0a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:23.335569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3849a3-183e-4490-aab2-95446b660c0a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:23.335719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3849a3-183e-4490-aab2-95446b660c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:23.357334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ea3849a3-183e-4490-aab2-95446b660c0a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:23.358590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ea3849a3-183e-4490-aab2-95446b660c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:23.427171 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 590/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:23.750905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ed8d8-8404-49d1-8f81-5a95548f6eb3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:23.820795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:23.861925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ed8d8-8404-49d1-8f81-5a95548f6eb3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:23.896026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230593-bdcb-4a14-b127-818dcdebc06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:23.896612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230593-bdcb-4a14-b127-818dcdebc06e None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:23.896882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230593-bdcb-4a14-b127-818dcdebc06e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:23.897145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230593-bdcb-4a14-b127-818dcdebc06e None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:23.897835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0230593-bdcb-4a14-b127-818dcdebc06e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:23.899185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0230593-bdcb-4a14-b127-818dcdebc06e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:23.918006 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 579/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Sat Sep 17 20:36:23 2022] POST /identity/v3/domains => generated 267 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:36:26.364308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afa150-de74-496c-81d5-86f01f216a4f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:26.365174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afa150-de74-496c-81d5-86f01f216a4f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:26.365174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afa150-de74-496c-81d5-86f01f216a4f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:26.367384 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 591/1170] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:26.372942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8f344-8e5c-4d8b-a6f6-8496b02e2040 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:26.373395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8f344-8e5c-4d8b-a6f6-8496b02e2040 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:26.373510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8f344-8e5c-4d8b-a6f6-8496b02e2040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:26.393484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-12e8f344-8e5c-4d8b-a6f6-8496b02e2040 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:26.394670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-12e8f344-8e5c-4d8b-a6f6-8496b02e2040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:26.464286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 580/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:26.788367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afa150-de74-496c-81d5-86f01f216a4f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:26.802439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:26.900752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afa150-de74-496c-81d5-86f01f216a4f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:26.936066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b26e1a-c0da-4960-83e8-1609f719224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:26.936675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26e1a-c0da-4960-83e8-1609f719224c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:26.936944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26e1a-c0da-4960-83e8-1609f719224c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:26.937210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b26e1a-c0da-4960-83e8-1609f719224c None admin] PATH_INFO: `/v3/domains/e780648e8deb4ffea79827e9364a3a4d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:26.948278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b26e1a-c0da-4960-83e8-1609f719224c None admin] RBAC: Authorizing `identity:get_domain(domain_id=e780648e8deb4ffea79827e9364a3a4d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:26.949696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b26e1a-c0da-4960-83e8-1609f719224c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:26.951348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 592/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Sat Sep 17 20:36:26 2022] GET /identity/v3/domains/e780648e8deb4ffea79827e9364a3a4d => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:26.957487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8f344-8e5c-4d8b-a6f6-8496b02e2040 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:27.013028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:27.063457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8f344-8e5c-4d8b-a6f6-8496b02e2040 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:27.098134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88291137-25fc-4ebe-941b-9065288fab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:27.098736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88291137-25fc-4ebe-941b-9065288fab77 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:27.099010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88291137-25fc-4ebe-941b-9065288fab77 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:27.099275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88291137-25fc-4ebe-941b-9065288fab77 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:27.099917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88291137-25fc-4ebe-941b-9065288fab77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:27.101350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88291137-25fc-4ebe-941b-9065288fab77 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:27.136314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 581/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:36:26 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:36:29.537505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46feb6b2-852e-4070-a1ec-4c99c21b777e None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:29.538400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46feb6b2-852e-4070-a1ec-4c99c21b777e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:29.538400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46feb6b2-852e-4070-a1ec-4c99c21b777e None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:29.540455 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 593/1174] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:29.549355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ea851-552a-4498-9647-7b1213d85831 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:29.549806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ea851-552a-4498-9647-7b1213d85831 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:29.549983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ea851-552a-4498-9647-7b1213d85831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:29.574378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4b8ea851-552a-4498-9647-7b1213d85831 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:29.575579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b8ea851-552a-4498-9647-7b1213d85831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:29.656743 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 582/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:29.991746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46feb6b2-852e-4070-a1ec-4c99c21b777e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:30.065170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:30.101878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46feb6b2-852e-4070-a1ec-4c99c21b777e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:30.136450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e7aee-d7f0-43bc-a161-1b5d66d06f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:30.137072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e7aee-d7f0-43bc-a161-1b5d66d06f64 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:30.137348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e7aee-d7f0-43bc-a161-1b5d66d06f64 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:30.137707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e7aee-d7f0-43bc-a161-1b5d66d06f64 None admin] PATH_INFO: `/v3/domains/e780648e8deb4ffea79827e9364a3a4d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:30.138950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e7aee-d7f0-43bc-a161-1b5d66d06f64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e780648e8deb4ffea79827e9364a3a4d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:30.140389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e7aee-d7f0-43bc-a161-1b5d66d06f64 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:30.141990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 594/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Sat Sep 17 20:36:29 2022] GET /identity/v3/domains/e780648e8deb4ffea79827e9364a3a4d => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:36:30.148776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ea851-552a-4498-9647-7b1213d85831 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:30.186926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:30.256451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ea851-552a-4498-9647-7b1213d85831 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:30.306715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96abd0-17c5-4f54-92d4-fb115b818cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:30.307326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96abd0-17c5-4f54-92d4-fb115b818cbe None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:30.307601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96abd0-17c5-4f54-92d4-fb115b818cbe None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:30.307865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96abd0-17c5-4f54-92d4-fb115b818cbe None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:30.308507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96abd0-17c5-4f54-92d4-fb115b818cbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:30.310005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96abd0-17c5-4f54-92d4-fb115b818cbe None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:30.338178 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 583/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:36:30 2022] POST /identity/v3/users => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:36:32.741727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0009b7-a2f5-4677-ad5f-4466220174c9 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:32.742306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0009b7-a2f5-4677-ad5f-4466220174c9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:32.742306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0009b7-a2f5-4677-ad5f-4466220174c9 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:32.744315 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 595/1178] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:32.750108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7c65e-7658-467e-bf48-8655aa454598 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:32.750580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7c65e-7658-467e-bf48-8655aa454598 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:32.750678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7c65e-7658-467e-bf48-8655aa454598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:32.770269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cee7c65e-7658-467e-bf48-8655aa454598 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:32.771449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cee7c65e-7658-467e-bf48-8655aa454598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:32.845423 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 584/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:33.164347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0009b7-a2f5-4677-ad5f-4466220174c9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:33.279588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0009b7-a2f5-4677-ad5f-4466220174c9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:33.335513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:33.336141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:33.336414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:33.336677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:33.339775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:33.349350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:36:33.354173 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:36:33.355807 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:36:33.356831 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 596/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:36:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:36:33.362437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7c65e-7658-467e-bf48-8655aa454598 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:33.387399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:33.468972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7c65e-7658-467e-bf48-8655aa454598 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:33.504219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:33.504816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:33.505088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:33.505355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:33.506043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:33.506347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:33.507704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:33.515872 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 585/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:36:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:33.522212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:33.628163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5620d7-846d-4a68-98eb-a6cb8aeef7a0 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:33.663671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:33.664322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:33.664773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:33.664923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] PATH_INFO: `/v3/users/c68189b6c453450f8943e29d32674243` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:33.685531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] RBAC: Authorizing `identity:get_user(user_id=c68189b6c453450f8943e29d32674243)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:33.687011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:33.689219 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 597/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:36:33 2022] GET /identity/v3/users/c68189b6c453450f8943e29d32674243 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:33.698222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:33.792556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:33.811261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbdc5-91c2-4ea2-b1e1-1b1f215c5a8a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:33.845969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:33.846570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:33.846846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:33.847111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:33.918394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:33.919889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:34.002940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:34.023672 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 586/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:36:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 20:36:34.032912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:34.095905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:34.151801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e79f7-5c51-4446-8c05-8ac36013fbdc None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:34.188284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff14cd1-2540-48d2-8c75-d7e491427228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:34.188906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14cd1-2540-48d2-8c75-d7e491427228 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:34.189185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14cd1-2540-48d2-8c75-d7e491427228 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:34.189432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14cd1-2540-48d2-8c75-d7e491427228 None admin] PATH_INFO: `/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:34.190719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff14cd1-2540-48d2-8c75-d7e491427228 None admin] RBAC: Authorizing `identity:get_project(project_id=c4244fa5b8a44deb8e9d290ff3b1ccfc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:34.192167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff14cd1-2540-48d2-8c75-d7e491427228 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:34.193842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 598/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:36:34 2022] GET /identity/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:34.203916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:34.310118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855199a3-17d8-4665-8f3a-8e276a1360e6 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:34.342732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34d6fe-7e3a-402f-9050-7fef97b8c259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:34.343316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d6fe-7e3a-402f-9050-7fef97b8c259 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:34.343592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d6fe-7e3a-402f-9050-7fef97b8c259 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:34.343860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34d6fe-7e3a-402f-9050-7fef97b8c259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:34.344572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34d6fe-7e3a-402f-9050-7fef97b8c259 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=c4244fa5b8a44deb8e9d290ff3b1ccfc, user.id=c68189b6c453450f8943e29d32674243) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:34.344880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34d6fe-7e3a-402f-9050-7fef97b8c259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:34.346400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34d6fe-7e3a-402f-9050-7fef97b8c259 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:34.351986 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 587/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:36:34 2022] GET /identity/v3/role_assignments?user.id=c68189b6c453450f8943e29d32674243&scope.project.id=c4244fa5b8a44deb8e9d290ff3b1ccfc&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:36.698780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a0fdc-4502-47fc-906f-ae7537acf222 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:36.699246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a0fdc-4502-47fc-906f-ae7537acf222 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:36.699348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a0fdc-4502-47fc-906f-ae7537acf222 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:36.701328 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 599/1186] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:36.706669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd7fed-9e9d-461d-915c-d30ee0ab8f7f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:36.707105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd7fed-9e9d-461d-915c-d30ee0ab8f7f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:36.707229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd7fed-9e9d-461d-915c-d30ee0ab8f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:36.725816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a7cd7fed-9e9d-461d-915c-d30ee0ab8f7f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:36.727017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a7cd7fed-9e9d-461d-915c-d30ee0ab8f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:36.769260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:36.804914 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 588/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:37.135699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a0fdc-4502-47fc-906f-ae7537acf222 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:37.243450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a0fdc-4502-47fc-906f-ae7537acf222 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:37.267294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:37.283010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:37.283629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:37.283904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:37.284173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:37.294927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:36:37.299728 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:36:37.300848 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:36:37.301780 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 600/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:36:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:36:37.307206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd7fed-9e9d-461d-915c-d30ee0ab8f7f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:37.411913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd7fed-9e9d-461d-915c-d30ee0ab8f7f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:37.450095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:37.450694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:37.451040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:37.451295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:37.451951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:37.452248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:37.453648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:37.461690 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 589/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:36:37.472381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:37.581138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4f3e4-167c-4328-b3ad-ca4be2f57ee9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:37.617833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:37.618444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:37.618718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:37.618988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] PATH_INFO: `/v3/users/c68189b6c453450f8943e29d32674243` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:37.620147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] RBAC: Authorizing `identity:get_user(user_id=c68189b6c453450f8943e29d32674243)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:37.621624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:37.623251 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 601/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:36:37 2022] GET /identity/v3/users/c68189b6c453450f8943e29d32674243 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:37.629404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:37.675154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:37.738704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cddbfe-8ddc-4d48-8d69-90dfce80efea None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:37.774654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526dfb8-7162-48e2-93e7-4e52a5bb4c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:37.775311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526dfb8-7162-48e2-93e7-4e52a5bb4c0e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:37.775587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526dfb8-7162-48e2-93e7-4e52a5bb4c0e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:37.775854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526dfb8-7162-48e2-93e7-4e52a5bb4c0e None admin] PATH_INFO: `/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:37.777095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526dfb8-7162-48e2-93e7-4e52a5bb4c0e None admin] RBAC: Authorizing `identity:get_project(project_id=c4244fa5b8a44deb8e9d290ff3b1ccfc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:37.778589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526dfb8-7162-48e2-93e7-4e52a5bb4c0e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:37.780640 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 590/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:36:37 2022] GET /identity/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:37.790584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:37.842802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:37.908528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb412b-13a4-4b30-9258-d35306b59e44 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:37.942557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6c835-4eaf-40ff-976a-d722bfe5f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:37.943137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6c835-4eaf-40ff-976a-d722bfe5f407 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:37.943406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6c835-4eaf-40ff-976a-d722bfe5f407 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:37.943675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6c835-4eaf-40ff-976a-d722bfe5f407 None admin] PATH_INFO: `/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc/users/c68189b6c453450f8943e29d32674243/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:37.945827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6c835-4eaf-40ff-976a-d722bfe5f407 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4244fa5b8a44deb8e9d290ff3b1ccfc, user_id=c68189b6c453450f8943e29d32674243, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:37.947246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6c835-4eaf-40ff-976a-d722bfe5f407 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:37.966187 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 602/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:36:37 2022] PUT /identity/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc/users/c68189b6c453450f8943e29d32674243/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:36:40.295840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea933f-830a-4560-b251-b4f1d32ae308 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:40.296515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea933f-830a-4560-b251-b4f1d32ae308 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:40.296515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea933f-830a-4560-b251-b4f1d32ae308 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:40.297773 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 591/1193] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:36:40.303228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d458ec0-e072-451b-9a1a-4e2d690d34bf None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:40.303649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d458ec0-e072-451b-9a1a-4e2d690d34bf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:40.303767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d458ec0-e072-451b-9a1a-4e2d690d34bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:40.324005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6d458ec0-e072-451b-9a1a-4e2d690d34bf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:36:40.325176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6d458ec0-e072-451b-9a1a-4e2d690d34bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:36:40.397878 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 603/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:36:40.744804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea933f-830a-4560-b251-b4f1d32ae308 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:40.864458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea933f-830a-4560-b251-b4f1d32ae308 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:40.872885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:40.904848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:40.905862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:40.906135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:40.906404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:40.916965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:36:40.921736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:36:40.922830 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:36:40.923821 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 592/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:36:40.929356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d458ec0-e072-451b-9a1a-4e2d690d34bf None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:40.972717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:41.034403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d458ec0-e072-451b-9a1a-4e2d690d34bf None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:41.073193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:41.073827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:41.074104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:41.074371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:41.075042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:41.075340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:41.076758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:41.084788 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 604/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:41.091035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:41.196869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6646cba-a085-4974-b2ac-1af9d7611728 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:41.237748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:41.238304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:41.238579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:41.238859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] PATH_INFO: `/v3/users/c68189b6c453450f8943e29d32674243` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:41.239962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] RBAC: Authorizing `identity:get_user(user_id=c68189b6c453450f8943e29d32674243)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:41.241406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:41.243000 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 593/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 17 20:36:41 2022] GET /identity/v3/users/c68189b6c453450f8943e29d32674243 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:41.249072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:41.354009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737f01-cadb-41a9-a3d3-daacc6dbade7 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:41.355569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:41.390466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:41.391059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:41.391329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:41.391598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:41.461450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:41.462914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:41.563772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 605/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:36:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:36:41.573169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:41.610262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:41.675730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5a1a-1c14-4707-93c7-cb702b2e3d3a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:41.711713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949016c-d60b-4688-83b6-4c8e906f5111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:41.712308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949016c-d60b-4688-83b6-4c8e906f5111 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:41.712581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949016c-d60b-4688-83b6-4c8e906f5111 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:41.712839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949016c-d60b-4688-83b6-4c8e906f5111 None admin] PATH_INFO: `/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:41.713954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949016c-d60b-4688-83b6-4c8e906f5111 None admin] RBAC: Authorizing `identity:get_project(project_id=c4244fa5b8a44deb8e9d290ff3b1ccfc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:41.715382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949016c-d60b-4688-83b6-4c8e906f5111 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:41.717065 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 594/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 17 20:36:41 2022] GET /identity/v3/projects/c4244fa5b8a44deb8e9d290ff3b1ccfc => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:41.723491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:36:41.725726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:41.824086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef335749-f45b-4477-804b-af1959817a81 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:36:41.858900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538384a-118b-4f18-9c25-0052fe28f20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:36:41.859490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538384a-118b-4f18-9c25-0052fe28f20a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:36:41.859765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538384a-118b-4f18-9c25-0052fe28f20a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:36:41.860023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538384a-118b-4f18-9c25-0052fe28f20a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:36:41.860731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538384a-118b-4f18-9c25-0052fe28f20a None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=c4244fa5b8a44deb8e9d290ff3b1ccfc, user.id=c68189b6c453450f8943e29d32674243) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:36:41.861036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538384a-118b-4f18-9c25-0052fe28f20a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:36:41.862499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538384a-118b-4f18-9c25-0052fe28f20a None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:36:41.868183 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 606/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:36:41 2022] GET /identity/v3/role_assignments?user.id=c68189b6c453450f8943e29d32674243&scope.project.id=c4244fa5b8a44deb8e9d290ff3b1ccfc&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:36:51.707018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=49 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:36:51.865553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=67 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:45.100356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d887f-cabd-491a-bd79-b78b18bc3598 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:45.100356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d887f-cabd-491a-bd79-b78b18bc3598 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:45.100356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d887f-cabd-491a-bd79-b78b18bc3598 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:45.100356 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 595/1201] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:37:45.114227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573945f-3510-4668-b3e1-aa0fa72884b6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:45.114227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573945f-3510-4668-b3e1-aa0fa72884b6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:45.114227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573945f-3510-4668-b3e1-aa0fa72884b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:45.133419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8573945f-3510-4668-b3e1-aa0fa72884b6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:37:45.134779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8573945f-3510-4668-b3e1-aa0fa72884b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:37:45.229507 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 607/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:37:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:37:45.558267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d887f-cabd-491a-bd79-b78b18bc3598 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:45.699458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d887f-cabd-491a-bd79-b78b18bc3598 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:45.747850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:45.748650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:45.748953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:45.749243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:45.766553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:45.768162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:37:45.789738 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:45.793705 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:37:45.795006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:37:45.795006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:45.795006 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:37:45.795006 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 596/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:45.797498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573945f-3510-4668-b3e1-aa0fa72884b6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:45.927010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573945f-3510-4668-b3e1-aa0fa72884b6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:45.962330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:45.970994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ab101-af71-4647-bbda-9f93091130bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:45.973615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab101-af71-4647-bbda-9f93091130bf None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:45.973615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab101-af71-4647-bbda-9f93091130bf None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:45.973615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab101-af71-4647-bbda-9f93091130bf None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:45.973615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ab101-af71-4647-bbda-9f93091130bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:45.973615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ab101-af71-4647-bbda-9f93091130bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:45.974818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ab101-af71-4647-bbda-9f93091130bf None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:45.992739 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 608/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:45.994080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:46.055447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:46.113833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e222d07-7173-4ad6-a12d-afb9218f1f99 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:46.162750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3b4d8-c113-408b-9462-b7ef75448ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:46.163407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3b4d8-c113-408b-9462-b7ef75448ba6 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:46.163686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3b4d8-c113-408b-9462-b7ef75448ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:46.163953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3b4d8-c113-408b-9462-b7ef75448ba6 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:46.164634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3b4d8-c113-408b-9462-b7ef75448ba6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:46.166240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3b4d8-c113-408b-9462-b7ef75448ba6 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:46.197582 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 597/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 17 20:37:45 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:37:48.742194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85047734-496d-4c3d-9854-ef2dcea3c4c4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:48.742194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85047734-496d-4c3d-9854-ef2dcea3c4c4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:48.744819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85047734-496d-4c3d-9854-ef2dcea3c4c4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:48.744819 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 609/1206] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:37:48.750641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899a845-2aa2-44cf-a05b-429ed3f996e7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:48.751116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899a845-2aa2-44cf-a05b-429ed3f996e7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:48.751215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899a845-2aa2-44cf-a05b-429ed3f996e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:48.772029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8899a845-2aa2-44cf-a05b-429ed3f996e7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:37:48.773216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8899a845-2aa2-44cf-a05b-429ed3f996e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:37:48.848085 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 598/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:37:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:37:49.188883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85047734-496d-4c3d-9854-ef2dcea3c4c4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:49.308732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85047734-496d-4c3d-9854-ef2dcea3c4c4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:49.350443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:49.351152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:49.351453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:49.351725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:49.362736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:37:49.367574 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:49.368538 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:37:49.369410 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:49.373587 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 610/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:37:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:49.375189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899a845-2aa2-44cf-a05b-429ed3f996e7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:49.418577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:49.483110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899a845-2aa2-44cf-a05b-429ed3f996e7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:49.520111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:49.520774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:49.521083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:49.521371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:49.522241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:49.522611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:49.524356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:49.539611 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 599/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:37:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:37:49.541020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:49.596366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:49.652980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85745f-db84-404b-87fd-14fbb4772c3a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:49.707225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:49.707869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:49.708147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:49.708418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:49.723140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:49.724775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:37:49.744163 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:49.752229 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:37:49.753151 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:37:49.753151 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:49.753151 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:49.753151 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 611/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:49.753401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:49.872454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:49.876347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824222f-8f84-4549-b494-75a333c71f7e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:49.913724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:49.914539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:49.914961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:49.915365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:49.916249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:49.916637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:49.918235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:49.933757 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 600/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:49.940409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:50.082846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1e71-4cca-4585-8066-6a233315640d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:50.084794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:50.123919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:50.124744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:50.125110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:50.125524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:50.133989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:50.135606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:37:50.147518 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:50.148747 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:37:50.150027 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:50.151098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 612/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:50.158165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:50.285872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99cf31-42f4-457c-8b1f-8b9fc459346f None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:50.327503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:50.328460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:50.328738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:50.329008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:50.329842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:50.330145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:50.331686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:50.353433 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 601/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 17 20:37:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:50.360253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:50.502425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b500f4c-c0c3-4351-940d-6733f12fa7e1 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:50.549365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:50.550017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:50.550299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:50.550574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:50.583351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:50.631228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:50.633134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:50.757408 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 613/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:37:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:37:50.763325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:50.797954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:50.900694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcec73-4455-4248-bfba-614c82a27f7e None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:50.948122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:50.948841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:50.949179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:50.949615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:50.960524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:50.962384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:37:50.982637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:50.983966 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:37:50.985096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:37:50.985096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:50.985096 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:37:50.989102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 602/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:37:50.990411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:51.057164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:51.098036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59beca9-d75a-4c2c-898f-b68119ab512c None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:51.132346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:51.132953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:51.133241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:51.133520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:51.134205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:51.134495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:51.135832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:51.152561 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 614/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:51.154140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:51.242287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:51.279414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759692c-c1a9-44c7-b75e-3487a48b97df None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:51.325003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:51.325713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:51.326077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:51.326376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:51.335119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:51.336841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:37:51.350915 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:51.352179 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:37:51.353150 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 603/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:37:51.358354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:51.473835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b078c2-527c-4d23-8450-0156bedda027 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:51.499190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:51.514587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a088761-80bc-4095-baf9-55c86078dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:51.515274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a088761-80bc-4095-baf9-55c86078dd00 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:51.515577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a088761-80bc-4095-baf9-55c86078dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:51.515924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a088761-80bc-4095-baf9-55c86078dd00 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:51.516695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a088761-80bc-4095-baf9-55c86078dd00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:51.517007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a088761-80bc-4095-baf9-55c86078dd00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:51.518510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a088761-80bc-4095-baf9-55c86078dd00 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:51.531849 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 615/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:51.538319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:51.656722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c6d1-dd01-4e4d-85c3-7f8f8dd4da6b None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:51.695104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:51.702212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da01e2-64fd-4fa6-b087-e8e63ed47398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:51.702898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da01e2-64fd-4fa6-b087-e8e63ed47398 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:51.703245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da01e2-64fd-4fa6-b087-e8e63ed47398 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:51.703584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da01e2-64fd-4fa6-b087-e8e63ed47398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:51.704379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da01e2-64fd-4fa6-b087-e8e63ed47398 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=064befaa1bd34b89bafad00c81ede856) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:51.704754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da01e2-64fd-4fa6-b087-e8e63ed47398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:51.706276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da01e2-64fd-4fa6-b087-e8e63ed47398 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:51.711957 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 604/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:37:51 2022] GET /identity/v3/role_assignments?user.id=064befaa1bd34b89bafad00c81ede856&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:37:54.227492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99819-2953-46fc-9851-892ef04f93a4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:54.228092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99819-2953-46fc-9851-892ef04f93a4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:54.228092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99819-2953-46fc-9851-892ef04f93a4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:54.229396 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 616/1220] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:37:54.234879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e98e5-1c7c-4b49-82d2-b3fc62902b33 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:54.235337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e98e5-1c7c-4b49-82d2-b3fc62902b33 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:54.235444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e98e5-1c7c-4b49-82d2-b3fc62902b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:54.254933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c05e98e5-1c7c-4b49-82d2-b3fc62902b33 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:37:54.256142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c05e98e5-1c7c-4b49-82d2-b3fc62902b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:37:54.336493 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 605/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:37:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:37:54.654004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99819-2953-46fc-9851-892ef04f93a4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:54.767571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99819-2953-46fc-9851-892ef04f93a4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:54.804096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:54.804749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:54.805032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:54.805392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] PATH_INFO: `/v3/roles/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:54.816220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:37:54.821041 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:54.822304 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:54.823642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 617/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:37:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:54.828209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e98e5-1c7c-4b49-82d2-b3fc62902b33 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:54.936490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e98e5-1c7c-4b49-82d2-b3fc62902b33 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:54.961122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:54.973776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:54.974528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:54.974863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:54.975163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:54.975915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] RBAC: Adding query filter params (name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:54.976246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:54.978216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:54.987407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 606/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:37:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:54.993254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:55.040596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:55.103682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b85bdd-38a5-4bc8-9acc-e3e3f24b06db None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:55.142235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:55.142872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:55.143146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:55.143413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:55.153769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:55.155594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:37:55.171630 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:55.172669 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:37:55.173989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:37:55.173989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:55.173989 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:55.173989 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 618/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:55.178799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:55.288379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675071c-28b6-475a-98ff-ef8244fb7547 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:55.333095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:55.333859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:55.334163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:55.334533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:55.335302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:55.335570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:55.337237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:55.350564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 607/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:37:55.357085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:55.462262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:55.471130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd70a1-8883-4ad2-af6c-3000203d5168 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:55.508347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:55.509045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:55.509354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:55.509692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] PATH_INFO: `/v3/users/placement` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:55.518141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:55.519662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:37:55.532126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:55.533906 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:55.535678 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 619/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:55.539501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:55.594301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:55.657331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df56bf-fad9-45d7-9d3b-335455f5db1a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:55.696001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:55.696686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:55.697023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:55.697351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:55.698163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:55.698539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:55.700032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:55.718139 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 608/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 17 20:37:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:55.724345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:55.839276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5e03c-a6c8-4013-82f9-3557fe8befbc None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:55.875034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:55.875673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:55.875948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:55.876263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:55.887931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:55.889413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:37:55.904946 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:55.906715 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:37:55.907982 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:37:55.907982 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:55.907982 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:55.907982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 620/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:55.913074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:56.022424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec632a-bc67-4565-8d31-525d7c0ea86a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:56.024980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:56.059965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:56.060619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:56.060886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:56.061148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:56.061979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:56.062264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:56.063696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:56.076422 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 609/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:56.082600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:56.092915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:56.190335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a12b52-c107-4048-a3ba-51eb9674d05e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:56.224593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:56.225231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:56.225544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:56.225819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:56.232448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:56.234062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:37:56.245138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:56.246201 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:37:56.247106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 621/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:56.252785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:56.359851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd2a63-c2d8-4684-bc14-c1319291aef5 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:56.395610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bc351-5269-40d1-8ec6-16dfa8e1bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:56.396315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bc351-5269-40d1-8ec6-16dfa8e1bbdd None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:56.396600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bc351-5269-40d1-8ec6-16dfa8e1bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:56.396912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bc351-5269-40d1-8ec6-16dfa8e1bbdd None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:56.397724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bc351-5269-40d1-8ec6-16dfa8e1bbdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:56.398052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bc351-5269-40d1-8ec6-16dfa8e1bbdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:56.399591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bc351-5269-40d1-8ec6-16dfa8e1bbdd None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:56.413323 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 610/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:56.419946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:56.489564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:56.525818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9be94d-81ae-4d5c-8f16-98c55a50edc6 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:56.559969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72283591-7b03-4e55-a59b-27349515e6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:56.560707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72283591-7b03-4e55-a59b-27349515e6f4 None admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:56.560954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72283591-7b03-4e55-a59b-27349515e6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:56.561248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72283591-7b03-4e55-a59b-27349515e6f4 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/064befaa1bd34b89bafad00c81ede856/roles/b888b6f28fcb407795b56d37fdf7c6f6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:56.576183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72283591-7b03-4e55-a59b-27349515e6f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=064befaa1bd34b89bafad00c81ede856, role_id=b888b6f28fcb407795b56d37fdf7c6f6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:56.577711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72283591-7b03-4e55-a59b-27349515e6f4 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:56.590683 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 622/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:37:56 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/064befaa1bd34b89bafad00c81ede856/roles/b888b6f28fcb407795b56d37fdf7c6f6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:37:59.005829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eedceb-652d-4f27-9c48-cef9b5a84cde None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:59.006418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eedceb-652d-4f27-9c48-cef9b5a84cde None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:59.006516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eedceb-652d-4f27-9c48-cef9b5a84cde None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:59.008088 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 611/1233] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:37:59.014233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f1f36-fac4-47b3-bb76-5aefce6fe579 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:59.014726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f1f36-fac4-47b3-bb76-5aefce6fe579 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:59.014823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f1f36-fac4-47b3-bb76-5aefce6fe579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:59.035276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d19f1f36-fac4-47b3-bb76-5aefce6fe579 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:37:59.036735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d19f1f36-fac4-47b3-bb76-5aefce6fe579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:37:59.115984 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 623/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:37:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:37:59.467217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eedceb-652d-4f27-9c48-cef9b5a84cde None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:59.502785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:59.586009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eedceb-652d-4f27-9c48-cef9b5a84cde None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:59.622293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:59.622925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:59.623192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:59.623463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] PATH_INFO: `/v3/roles/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:59.633377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:37:59.638108 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:59.639266 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:37:59.640334 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 612/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 17 20:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:59.645511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f1f36-fac4-47b3-bb76-5aefce6fe579 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:59.696716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:59.756764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f1f36-fac4-47b3-bb76-5aefce6fe579 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:59.794824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:59.795493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:59.795834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:59.796152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:59.796870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] RBAC: Adding query filter params (name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:37:59.797190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:59.798654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:59.807748 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 624/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 17 20:37:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:37:59.814263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:37:59.904443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:37:59.922607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce862-9be7-41cd-bceb-682c519f6bef None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:37:59.960592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:37:59.961205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:37:59.961543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:37:59.961775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:37:59.972157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:37:59.973705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:37:59.990052 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:37:59.991258 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:37:59.992190 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:37:59.992190 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:37:59.992190 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:37:59.992190 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 613/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:37:59.997676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:00.108056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88e222-633c-405e-b854-3a8dcfbd5e34 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:00.117393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:00.143736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:00.144367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:00.144628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:00.144891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:00.145588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:00.145885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:00.147265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:00.159553 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 625/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:00.165715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:00.271690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa39647-13fd-49e5-81da-5e212787366d None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:00.307825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:00.308509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:00.308799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:00.309112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:00.312147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:00.317946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:00.320021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:38:00.334322 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:00.335552 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:00.336725 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 614/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:00.341848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:00.448536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bde9de-1499-4441-94f5-740b82696781 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:00.488195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:00.488952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:00.489245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:00.489562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:00.490454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:00.490804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:00.492475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:00.513433 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 626/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 17 20:38:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:00.519874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:00.634624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f5276-3d63-4fed-b281-cc7ef1874a33 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:00.673040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:00.674210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:00.674760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:00.675252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:00.745510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:00.752621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:00.754125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:00.860767 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 615/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:38:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 20:38:00.867861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:00.897784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:00.980483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491e457-6d40-496d-81f6-67d1983b209d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:01.015442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:01.015995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:01.016270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:01.016545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:01.027167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:01.028808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:38:01.045392 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:01.046647 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:38:01.047907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:38:01.047907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:01.047907 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:01.047907 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 627/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:38:01.052705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:01.153778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:01.160947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369f419-e738-4252-94ee-b164afcf486a None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:01.196997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:01.197968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:01.198339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:01.198733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:01.199469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:01.199775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:01.201253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:01.213419 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 616/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:01.219552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:01.312214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:01.337758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eb1a6-7639-4c50-b151-8baaf335e928 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:01.384701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:01.385335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:01.385642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:01.385916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:01.397036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:01.398716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:38:01.411467 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:01.412803 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:01.414156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 628/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:01.419073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:01.539024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd2142-3fa8-49e3-8169-1a9f1035afba None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:01.577591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc4adb-a978-4bb3-9420-82d8591d9ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:01.578194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc4adb-a978-4bb3-9420-82d8591d9ec7 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:01.578481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc4adb-a978-4bb3-9420-82d8591d9ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:01.578741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc4adb-a978-4bb3-9420-82d8591d9ec7 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:01.579556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc4adb-a978-4bb3-9420-82d8591d9ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:01.579854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc4adb-a978-4bb3-9420-82d8591d9ec7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:01.581599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc4adb-a978-4bb3-9420-82d8591d9ec7 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:01.594935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 617/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:01.601610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:01.714005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0edef1-282d-4652-99f2-8fb57e761eac None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:01.749904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28dcff-ab10-4c6b-b6d5-ec108c7699f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:01.750576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28dcff-ab10-4c6b-b6d5-ec108c7699f1 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:01.750863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28dcff-ab10-4c6b-b6d5-ec108c7699f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:01.751118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28dcff-ab10-4c6b-b6d5-ec108c7699f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:01.751865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28dcff-ab10-4c6b-b6d5-ec108c7699f1 None admin] RBAC: Adding query filter params (role.id=b888b6f28fcb407795b56d37fdf7c6f6, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=064befaa1bd34b89bafad00c81ede856) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:01.752149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28dcff-ab10-4c6b-b6d5-ec108c7699f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:01.753621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28dcff-ab10-4c6b-b6d5-ec108c7699f1 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:01.759522 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 629/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:38:01 2022] GET /identity/v3/role_assignments?user.id=064befaa1bd34b89bafad00c81ede856&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=b888b6f28fcb407795b56d37fdf7c6f6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:04.332657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4b321-2c82-49f3-a7fb-008d8e31ee69 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:04.333279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4b321-2c82-49f3-a7fb-008d8e31ee69 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:04.333402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4b321-2c82-49f3-a7fb-008d8e31ee69 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:04.334756 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 618/1247] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:38:04.342394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d65ca-5009-49b4-ae3f-d1f98590ec84 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:04.343060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d65ca-5009-49b4-ae3f-d1f98590ec84 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:04.343164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d65ca-5009-49b4-ae3f-d1f98590ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:04.351796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:04.370265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-340d65ca-5009-49b4-ae3f-d1f98590ec84 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:38:04.371850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-340d65ca-5009-49b4-ae3f-d1f98590ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:38:04.449813 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 630/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:38:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:38:04.766050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4b321-2c82-49f3-a7fb-008d8e31ee69 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:04.786442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:04.871065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4b321-2c82-49f3-a7fb-008d8e31ee69 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:04.909397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:04.910059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:04.910332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:04.910595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:04.921103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:38:04.925888 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:04.927054 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:04.928097 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 619/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:04.933144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d65ca-5009-49b4-ae3f-d1f98590ec84 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:05.032973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d65ca-5009-49b4-ae3f-d1f98590ec84 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:05.063201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:05.069594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:05.070213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:05.070493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:05.070758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:05.071429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:05.071731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:05.073118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:05.082082 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 631/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:05.088264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:05.177085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:05.201572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44106ae-35a7-45d3-91d1-c7d59b196a31 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:05.236088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:05.236698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:05.236974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:05.237240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:05.248133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:05.249611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:38:05.265484 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:05.266579 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:38:05.267728 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:38:05.267728 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:05.267728 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:05.267728 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 620/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:05.273151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:05.390931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b792-d3ca-42f1-bf2d-b66a5515645e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:05.428301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:05.428898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:05.429174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:05.429442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:05.430114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:05.430417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:05.431787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:05.443723 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 632/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:05.449898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:05.562180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4904c-6b97-42a9-b2c8-58b79f02a4cb None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:05.589112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:05.597640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:05.598258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:05.598529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:05.598796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:05.606041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:05.607471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:38:05.618545 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:05.619622 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:05.620861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 621/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:05.625810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:05.682861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:05.729841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f5c52-d8b0-4a9e-be4b-19f8ae85ab70 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:05.768497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:05.769100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:05.769379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:05.769687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:05.770422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:05.770716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:05.772188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:05.792886 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 633/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 17 20:38:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:05.799245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:05.910622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25090a-7b57-4b9e-afd5-bbe42eeeb450 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:05.945812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:05.946338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:05.946651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:05.946897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:06.007945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:06.022948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:06.024401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:06.122526 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 622/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:38:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:38:06.128840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:06.238915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9249-7850-49ed-89b4-cb3c3bc0746f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:06.252792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:06.276323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:06.276941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:06.277214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:06.277502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:06.288563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:06.290102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:38:06.307377 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:06.308788 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:38:06.310125 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:38:06.310125 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:06.310125 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:06.310125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 634/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:06.315649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:06.411410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:06.426657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a998fb2-030f-4808-9013-979479c46da0 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:06.465587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:06.466182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:06.466446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:06.466712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:06.467355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:06.467648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:06.469013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:06.480308 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 623/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:38:06.490186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:06.605332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85a00-ac71-4f20-ba78-00b4bdd0ef90 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:06.640677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:06.641279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:06.641585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:06.641856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:06.648769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:06.650243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:38:06.662098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:06.663101 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:06.664190 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 635/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:06.669885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:06.772289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cddbb-4006-4d44-805e-9a8bf03ed5f2 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:06.808213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711781f7-6885-4617-a25a-839b9f9461d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:06.808848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711781f7-6885-4617-a25a-839b9f9461d4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:06.809114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711781f7-6885-4617-a25a-839b9f9461d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:06.809376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711781f7-6885-4617-a25a-839b9f9461d4 None admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:06.810081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711781f7-6885-4617-a25a-839b9f9461d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:06.810378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711781f7-6885-4617-a25a-839b9f9461d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:06.811852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711781f7-6885-4617-a25a-839b9f9461d4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:06.823739 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 624/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:06.833716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:06.864784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:06.943035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741474-3146-4a31-bd2f-0d385e23f138 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:06.986250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96757937-161c-4153-b0b3-12d59a1f818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:06.986838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96757937-161c-4153-b0b3-12d59a1f818b None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:06.987112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96757937-161c-4153-b0b3-12d59a1f818b None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:06.987371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96757937-161c-4153-b0b3-12d59a1f818b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:06.988092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96757937-161c-4153-b0b3-12d59a1f818b None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=064befaa1bd34b89bafad00c81ede856) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:06.988392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96757937-161c-4153-b0b3-12d59a1f818b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:06.989861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96757937-161c-4153-b0b3-12d59a1f818b None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:06.995705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 636/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:38:06 2022] GET /identity/v3/role_assignments?user.id=064befaa1bd34b89bafad00c81ede856&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:09.432972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d937c3-df60-4081-add7-5d76636bf2d4 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:09.433573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d937c3-df60-4081-add7-5d76636bf2d4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:09.433573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d937c3-df60-4081-add7-5d76636bf2d4 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:09.434862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 625/1261] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:38:09.441145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48b09-6796-46b7-91f2-c8efd59451bc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:09.441863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48b09-6796-46b7-91f2-c8efd59451bc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:09.442059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48b09-6796-46b7-91f2-c8efd59451bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:09.466347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9cb48b09-6796-46b7-91f2-c8efd59451bc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:38:09.467522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9cb48b09-6796-46b7-91f2-c8efd59451bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:38:09.552988 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 637/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:38:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:38:09.871146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d937c3-df60-4081-add7-5d76636bf2d4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:09.884017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:09.981149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d937c3-df60-4081-add7-5d76636bf2d4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:10.017424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:10.018068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:10.018348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:10.018608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:10.029485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:38:10.034590 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:10.035688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:38:10.036804 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:10.040544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 626/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:38:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:10.041808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb48b09-6796-46b7-91f2-c8efd59451bc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:10.062523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:10.147652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb48b09-6796-46b7-91f2-c8efd59451bc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:10.182435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:10.183033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:10.183303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:10.183563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:10.184217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:10.184511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:10.185896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:10.194020 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 638/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:38:10.199937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:10.271701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:10.307055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf524-5800-412d-a25c-c655fbabd139 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:10.340290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:10.340914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:10.341184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:10.341448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:10.351512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:10.352988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:38:10.367470 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:10.368559 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:38:10.369672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:38:10.369672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:10.369672 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:10.369672 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 627/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:10.375244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:10.466021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:10.485704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807a4b1-ec0f-433e-bc3e-f314dfa6c2f9 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:10.521883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:10.522764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:10.523167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:10.523592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:10.524606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:10.525072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:10.527311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:10.546719 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 639/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:10.552574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:10.658434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bebe9-4f71-4475-a0a7-042c37d736aa None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:10.679919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:10.695467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:10.696097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:10.696360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:10.696624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:10.704176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:10.705645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:38:10.716476 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:10.717553 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:10.718586 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 628/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:10.724250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:10.828561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbb82-71bf-460a-9160-e5d870189b9d None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:10.864226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:10.864818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:10.865093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:10.865370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:10.866127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:10.866431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:10.867865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:10.876586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:10.885724 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 640/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 17 20:38:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:10.894774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:11.012070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573b991-df7b-4056-8f7e-3cd7bc8d12ba None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:11.047079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:11.047685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:11.047960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:11.048205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:11.058894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:11.060330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:38:11.074604 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:11.075690 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:38:11.076615 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:38:11.076615 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:11.076615 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:11.076615 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 629/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:11.082370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:11.199798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e73f6-3861-42f4-9cd5-72ca9304dd38 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:11.235392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:11.235994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:11.236266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:11.236534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:11.237169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:11.237502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:11.238874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:11.251374 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 641/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:11.263547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:11.300607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:11.382488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175566f-a989-4197-9db4-24b175e1e693 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:11.417393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:11.418052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:11.418326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:11.418591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:11.425493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:11.426972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:38:11.438784 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:11.439806 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:11.440716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 630/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:11.448006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:11.508180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:11.552521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc78ff3-5abc-4d61-964c-83cefde228f2 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:11.587161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b2f2a-fde8-4328-a3dc-f2bf650ce9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:11.587759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2f2a-fde8-4328-a3dc-f2bf650ce9fa None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:11.588029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2f2a-fde8-4328-a3dc-f2bf650ce9fa None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:11.588296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2f2a-fde8-4328-a3dc-f2bf650ce9fa None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:11.588977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2f2a-fde8-4328-a3dc-f2bf650ce9fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:11.589276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2f2a-fde8-4328-a3dc-f2bf650ce9fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:11.590760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2f2a-fde8-4328-a3dc-f2bf650ce9fa None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:11.602809 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 642/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:38:11.609248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:11.716176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:11.726395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c83fc2-8ed7-4ad8-94a0-402b46519ef4 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:11.760596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efee4f-df14-4fa2-aae2-da63990e9191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:11.761248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efee4f-df14-4fa2-aae2-da63990e9191 None admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:11.761544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efee4f-df14-4fa2-aae2-da63990e9191 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:11.761822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efee4f-df14-4fa2-aae2-da63990e9191 None admin] PATH_INFO: `/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/064befaa1bd34b89bafad00c81ede856/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:11.763816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efee4f-df14-4fa2-aae2-da63990e9191 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e0a10f1c5b4b76b9d08df11fb7a377, user_id=064befaa1bd34b89bafad00c81ede856, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:11.765354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efee4f-df14-4fa2-aae2-da63990e9191 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:11.781880 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 631/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 17 20:38:11 2022] PUT /identity/v3/projects/58e0a10f1c5b4b76b9d08df11fb7a377/users/064befaa1bd34b89bafad00c81ede856/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:38:14.128183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb32c9-2f8f-4213-8e6b-6386500ad273 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:14.128722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb32c9-2f8f-4213-8e6b-6386500ad273 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:14.128722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb32c9-2f8f-4213-8e6b-6386500ad273 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:14.130122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 643/1274] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:38:14.136357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161dbf5-fb7c-49bd-b206-539dccc6a3f4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:14.136794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161dbf5-fb7c-49bd-b206-539dccc6a3f4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:14.136920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161dbf5-fb7c-49bd-b206-539dccc6a3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:14.158762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8161dbf5-fb7c-49bd-b206-539dccc6a3f4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:38:14.159968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8161dbf5-fb7c-49bd-b206-539dccc6a3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:38:14.232751 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 632/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:38:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:38:14.580003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb32c9-2f8f-4213-8e6b-6386500ad273 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:14.692346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb32c9-2f8f-4213-8e6b-6386500ad273 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:14.715024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:14.739545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:14.740162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:14.740432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:14.740698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:14.751267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:38:14.755997 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:14.757066 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:14.758328 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 644/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 17 20:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:14.763169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161dbf5-fb7c-49bd-b206-539dccc6a3f4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:14.850076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:14.871526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161dbf5-fb7c-49bd-b206-539dccc6a3f4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:14.908713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:14.909328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:14.909644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:14.909913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:14.910579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:14.910880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:14.912311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:14.921052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 633/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 17 20:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:14.927202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:15.030716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61924c0a-d28b-4dd4-849b-2f0b1f22777a None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:15.064366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:15.064999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:15.065271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:15.065607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:15.076142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:15.077812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:38:15.092039 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:15.093244 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:38:15.098723 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:38:15.098723 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:15.098723 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:15.098723 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 645/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:15.099534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:15.195212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad13f7-3e20-4774-b2da-74c429924af8 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:15.232509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:15.233109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:15.233380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:15.233683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:15.234312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:15.234605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:15.235968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:15.247568 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 634/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:15.253728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:15.286765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:15.368847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788d37a-ee02-4898-9bdc-bb3182a77c52 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:15.403294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:15.403936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:15.404205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:15.404464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] PATH_INFO: `/v3/users/placement` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:15.412444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:15.413927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:38:15.424932 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:15.426063 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:15.427126 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 646/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:15.432720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:15.438923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:15.545209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165542b0-fcc0-4882-8158-c9925cc8672c None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:15.580646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:15.581240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:15.581524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:15.581819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:15.582532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:15.582839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:15.584262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:15.602033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 635/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 17 20:38:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:38:15.610573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:15.713205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:15.723757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc312ddb-963b-4555-becb-059dcea67bdd None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:15.758260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:15.758848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:15.759116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:15.759383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:15.828554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:15.829967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:15.918840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:15.931748 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 647/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 17 20:38:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:38:15.938726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:16.013604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:16.060081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27581b30-31de-4b26-90e8-5445e2ee0106 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:16.099793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:16.100370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:16.100639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:16.100902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:16.111340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:16.112774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:38:16.127583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:16.128736 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:38:16.129866 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:38:16.129866 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:38:16.129866 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:16.129866 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 636/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:16.134806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:16.235860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac6a2d-c765-41be-b213-7211d8d7177e None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:16.271507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:16.272098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:16.272354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:16.272621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:16.273257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:16.273588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:16.274959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:16.286567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 648/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 17 20:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:16.292950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:16.401141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632afc6-971d-4d50-aa27-32a6ed83a676 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:16.422930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:16.444795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:16.445420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:16.445726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:16.445991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:16.452896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:16.454374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:38:16.465611 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:16.466712 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 20:38:16.467643 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 637/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 17 20:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:16.474761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:16.529630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:16.581986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a1603-466d-41ca-9669-284fb4fb2232 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:16.616437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1566f9c-be4f-416d-83c0-c649b5561f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:16.617036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1566f9c-be4f-416d-83c0-c649b5561f09 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:16.617306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1566f9c-be4f-416d-83c0-c649b5561f09 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:16.617606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1566f9c-be4f-416d-83c0-c649b5561f09 None admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:16.618283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1566f9c-be4f-416d-83c0-c649b5561f09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:16.618614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1566f9c-be4f-416d-83c0-c649b5561f09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:16.620049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1566f9c-be4f-416d-83c0-c649b5561f09 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:16.632054 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 649/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 17 20:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:16.638322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:16.753856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce117d2-db74-4c7f-bb77-07460f534b37 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:16.789448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6249fbb-ba23-4cb5-89db-f19ae9a6af91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:16.790073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6249fbb-ba23-4cb5-89db-f19ae9a6af91 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:16.790346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6249fbb-ba23-4cb5-89db-f19ae9a6af91 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:16.790624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6249fbb-ba23-4cb5-89db-f19ae9a6af91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:16.791343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6249fbb-ba23-4cb5-89db-f19ae9a6af91 None admin] RBAC: Adding query filter params (role.id=6583e7a6ce674f6da4f19e31fe025509, scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377, user.id=064befaa1bd34b89bafad00c81ede856) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:16.791656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6249fbb-ba23-4cb5-89db-f19ae9a6af91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:16.793075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6249fbb-ba23-4cb5-89db-f19ae9a6af91 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:16.798976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 638/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 17 20:38:16 2022] GET /identity/v3/role_assignments?user.id=064befaa1bd34b89bafad00c81ede856&scope.project.id=58e0a10f1c5b4b76b9d08df11fb7a377&role.id=6583e7a6ce674f6da4f19e31fe025509 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:19.197862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569f4be-e653-4e40-a449-86d8990b1c9a None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:19.198455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569f4be-e653-4e40-a449-86d8990b1c9a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:19.198455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569f4be-e653-4e40-a449-86d8990b1c9a None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:19.199748 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 650/1288] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:38:19.206161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7526f-2606-42d3-a892-9b1aad9d5d0e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:19.206875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7526f-2606-42d3-a892-9b1aad9d5d0e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:19.207056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7526f-2606-42d3-a892-9b1aad9d5d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:19.235844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-66b7526f-2606-42d3-a892-9b1aad9d5d0e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:38:19.237021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-66b7526f-2606-42d3-a892-9b1aad9d5d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:38:19.266991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:19.309117 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 639/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:38:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:38:19.630947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569f4be-e653-4e40-a449-86d8990b1c9a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:19.740500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569f4be-e653-4e40-a449-86d8990b1c9a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:19.751280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:19.784553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:19.785182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:19.785484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:19.785758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:19.786348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:19.787735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:19.796633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:19.797830 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:19.798885 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 651/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:38:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:19.804012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7526f-2606-42d3-a892-9b1aad9d5d0e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:19.899736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7526f-2606-42d3-a892-9b1aad9d5d0e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:19.939747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15b306-3358-44fc-8113-a9d9a3113777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:19.940346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15b306-3358-44fc-8113-a9d9a3113777 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:19.940617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15b306-3358-44fc-8113-a9d9a3113777 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:19.940881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15b306-3358-44fc-8113-a9d9a3113777 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:19.941515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15b306-3358-44fc-8113-a9d9a3113777 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:19.941833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15b306-3358-44fc-8113-a9d9a3113777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:19.943208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15b306-3358-44fc-8113-a9d9a3113777 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:19.948141 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 640/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 17 20:38:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:19.954101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:20.067692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fe22b-07c9-41a7-aae0-badac6528993 None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:20.103793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de60b1-de7d-4f4f-9d6b-e7891d07186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:20.104390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de60b1-de7d-4f4f-9d6b-e7891d07186e None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:20.104660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de60b1-de7d-4f4f-9d6b-e7891d07186e None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:20.104927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de60b1-de7d-4f4f-9d6b-e7891d07186e None admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:20.105579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de60b1-de7d-4f4f-9d6b-e7891d07186e None admin] RBAC: Adding query filter params (type=placement) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:20.105880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de60b1-de7d-4f4f-9d6b-e7891d07186e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:20.107283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de60b1-de7d-4f4f-9d6b-e7891d07186e None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:20.112023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 652/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 17 20:38:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:22.507868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7bcc-87d2-4d34-b198-db7ea3f95cc8 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:22.508749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7bcc-87d2-4d34-b198-db7ea3f95cc8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:22.508749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7bcc-87d2-4d34-b198-db7ea3f95cc8 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:22.510890 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 641/1293] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:38:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:38:22.516526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ef16a-c2fb-40e4-b6f3-d49b8c9dbec6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:22.516974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ef16a-c2fb-40e4-b6f3-d49b8c9dbec6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:22.517083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ef16a-c2fb-40e4-b6f3-d49b8c9dbec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:22.538313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-837ef16a-c2fb-40e4-b6f3-d49b8c9dbec6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:38:22.539508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-837ef16a-c2fb-40e4-b6f3-d49b8c9dbec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:38:22.573002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:22.615750 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 653/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:38:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:38:22.936080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc7bcc-87d2-4d34-b198-db7ea3f95cc8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:22.976687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:23.056429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc7bcc-87d2-4d34-b198-db7ea3f95cc8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:23.098328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab6ada-0a68-44ad-9dcb-bc9f8a8b4f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:23.099184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6ada-0a68-44ad-9dcb-bc9f8a8b4f87 None admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:23.099543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6ada-0a68-44ad-9dcb-bc9f8a8b4f87 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:23.099908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6ada-0a68-44ad-9dcb-bc9f8a8b4f87 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:23.100926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab6ada-0a68-44ad-9dcb-bc9f8a8b4f87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:23.103113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab6ada-0a68-44ad-9dcb-bc9f8a8b4f87 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:23.120686 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 642/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 17 20:38:22 2022] POST /identity/v3/services => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:38:25.512571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b317-664f-494e-bf5f-49621f406fbf None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:25.513135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b317-664f-494e-bf5f-49621f406fbf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:25.513135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b317-664f-494e-bf5f-49621f406fbf None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:25.514520 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 654/1296] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:38:25.520178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c384-ca9c-45b5-8cf4-728195ae5910 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:25.520571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c384-ca9c-45b5-8cf4-728195ae5910 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:25.520720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c384-ca9c-45b5-8cf4-728195ae5910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:25.547418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f368c384-ca9c-45b5-8cf4-728195ae5910 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:38:25.548612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f368c384-ca9c-45b5-8cf4-728195ae5910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:38:25.624724 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 643/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:38:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:38:25.952638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b317-664f-494e-bf5f-49621f406fbf None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:26.071452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b317-664f-494e-bf5f-49621f406fbf None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:26.115564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:26.116191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:26.116474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:26.116745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] PATH_INFO: `/v3/services/placement` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:26.117329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:26.118819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:26.127976 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:26.129098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:26.130248 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 655/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:38:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:26.135193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c384-ca9c-45b5-8cf4-728195ae5910 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:26.155591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:26.240425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c384-ca9c-45b5-8cf4-728195ae5910 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:26.275767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:26.276368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:26.276646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:26.276921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:26.277575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:26.277885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:26.279260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:26.284809 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 644/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 17 20:38:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:26.294182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:26.329635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:26.403836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313403e-ed4f-4caf-bcdb-3f8e7bcd710f None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:26.442065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0e127-2c0d-4b42-b5b8-81ba60ce4661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:26.442656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0e127-2c0d-4b42-b5b8-81ba60ce4661 None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:26.442946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0e127-2c0d-4b42-b5b8-81ba60ce4661 None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:26.443202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0e127-2c0d-4b42-b5b8-81ba60ce4661 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:26.443902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0e127-2c0d-4b42-b5b8-81ba60ce4661 None admin] RBAC: Adding query filter params (interface=public, service_id=4607582162714a13aa6ca3df974805d7, region_id=RegionOne) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:26.444205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0e127-2c0d-4b42-b5b8-81ba60ce4661 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:26.445603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0e127-2c0d-4b42-b5b8-81ba60ce4661 None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:26.450687 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 656/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 17 20:38:26 2022] GET /identity/v3/endpoints?service_id=4607582162714a13aa6ca3df974805d7&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:26.457159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:26.545538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:26.565928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b584ff-87fe-4f83-b733-ae409a4c4675 None admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:26.601113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c68df-9725-430d-aee0-bb8df0d7c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:26.601746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c68df-9725-430d-aee0-bb8df0d7c911 None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:26.602025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c68df-9725-430d-aee0-bb8df0d7c911 None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:26.602290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c68df-9725-430d-aee0-bb8df0d7c911 None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:26.602893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c68df-9725-430d-aee0-bb8df0d7c911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:26.604259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c68df-9725-430d-aee0-bb8df0d7c911 None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:26.611136 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 645/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 17 20:38:26 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:38:28.965210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebc609-20bc-47c2-a55c-0d2bc77d2182 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:28.965771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebc609-20bc-47c2-a55c-0d2bc77d2182 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:28.965771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebc609-20bc-47c2-a55c-0d2bc77d2182 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:28.967032 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 657/1302] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:38:28.973429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05987f41-ba6f-4328-a091-4ac4697e5772 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:28.974209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05987f41-ba6f-4328-a091-4ac4697e5772 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:28.974380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05987f41-ba6f-4328-a091-4ac4697e5772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:28.998430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-05987f41-ba6f-4328-a091-4ac4697e5772 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:38:28.999629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-05987f41-ba6f-4328-a091-4ac4697e5772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:38:29.071348 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 646/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:38:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:38:29.391247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebc609-20bc-47c2-a55c-0d2bc77d2182 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:29.496497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:29.503930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebc609-20bc-47c2-a55c-0d2bc77d2182 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:29.538138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:29.538755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:29.539026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:29.539269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] PATH_INFO: `/v3/services/placement` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:29.539850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:29.541215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:38:29.549884 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:38:29.551144 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 20:38:29.552228 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 658/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 17 20:38:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:38:29.557138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05987f41-ba6f-4328-a091-4ac4697e5772 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:29.637646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:29.662095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05987f41-ba6f-4328-a091-4ac4697e5772 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:29.699773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92767de-2fee-4722-ae7c-0c21f8d94a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:29.700379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92767de-2fee-4722-ae7c-0c21f8d94a1b None admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:29.700651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92767de-2fee-4722-ae7c-0c21f8d94a1b None admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:29.700919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92767de-2fee-4722-ae7c-0c21f8d94a1b None admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:29.701582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92767de-2fee-4722-ae7c-0c21f8d94a1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:38:29.701886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92767de-2fee-4722-ae7c-0c21f8d94a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:29.703311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92767de-2fee-4722-ae7c-0c21f8d94a1b None admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:29.708340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 647/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 17 20:38:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:38:29.717528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:38:29.826765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30630aa5-eb37-4d00-beb9-9ae03b9050ca None admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:38:29.861841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a4829-04ef-4a21-974f-0e81018850bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:38:29.862421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4829-04ef-4a21-974f-0e81018850bc None admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:38:29.862695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4829-04ef-4a21-974f-0e81018850bc None admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:38:29.862963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4829-04ef-4a21-974f-0e81018850bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:38:29.863610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a4829-04ef-4a21-974f-0e81018850bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:38:29.865002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a4829-04ef-4a21-974f-0e81018850bc None admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:38:29.891752 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 659/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:38:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:38:39.705448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=30 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:38:39.884946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:42.596789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1907a4-bfd8-4bbd-b805-048394156115 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:42.597291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1907a4-bfd8-4bbd-b805-048394156115 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:42.597380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1907a4-bfd8-4bbd-b805-048394156115 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:42.604237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027967d-149b-4265-beb0-918d2ab88848 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:42.604670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027967d-149b-4265-beb0-918d2ab88848 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:42.604866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027967d-149b-4265-beb0-918d2ab88848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:42.609918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 648/1307] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:39:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:39:42.717168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1027967d-149b-4265-beb0-918d2ab88848 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:39:42.718464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1027967d-149b-4265-beb0-918d2ab88848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:42.899719 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 660/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:39:43.244797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1658e3-44ef-4a5e-9a01-67114e196305 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:43.245081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1658e3-44ef-4a5e-9a01-67114e196305 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:43.245741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1658e3-44ef-4a5e-9a01-67114e196305 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:43.247203 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 649/1309] 10.222.0.26 () {54 vars in 963 bytes} [Sat Sep 17 20:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:39:43.253558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27a39-0378-4d0e-8acc-9fd018cbda27 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:43.254130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27a39-0378-4d0e-8acc-9fd018cbda27 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:43.254465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27a39-0378-4d0e-8acc-9fd018cbda27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:43.336812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9fb27a39-0378-4d0e-8acc-9fd018cbda27 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:39:43.338333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9fb27a39-0378-4d0e-8acc-9fd018cbda27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:43.360905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:43.424931 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 661/1310] 10.222.0.26 () {58 vars in 1060 bytes} [Sat Sep 17 20:39:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:39:43.435519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1658e3-44ef-4a5e-9a01-67114e196305 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:39:43.437341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c1658e3-44ef-4a5e-9a01-67114e196305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:43.569311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1658e3-44ef-4a5e-9a01-67114e196305 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:39:43.654199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ded61-e2b3-40fb-9407-12beb00977ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:39:43.654872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ded61-e2b3-40fb-9407-12beb00977ab service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:43.655537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ded61-e2b3-40fb-9407-12beb00977ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:43.655917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ded61-e2b3-40fb-9407-12beb00977ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:43.657668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a99ded61-e2b3-40fb-9407-12beb00977ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:43.725981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:43.811461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ded61-e2b3-40fb-9407-12beb00977ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:39:43.812948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ded61-e2b3-40fb-9407-12beb00977ab service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:39:43.835220 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 650/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:39:46.486181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55e34-65bf-4e0a-acdd-75b4b05eec81 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:46.486730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55e34-65bf-4e0a-acdd-75b4b05eec81 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:46.486730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f55e34-65bf-4e0a-acdd-75b4b05eec81 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:46.488390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 662/1312] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:39:46.493789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a03f-780e-4c6e-86bb-ada2ad66e92e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:46.494627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a03f-780e-4c6e-86bb-ada2ad66e92e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:46.494988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a03f-780e-4c6e-86bb-ada2ad66e92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:46.583707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-43d7a03f-780e-4c6e-86bb-ada2ad66e92e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:39:46.584929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43d7a03f-780e-4c6e-86bb-ada2ad66e92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:46.693029 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 651/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:39:47.028007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f55e34-65bf-4e0a-acdd-75b4b05eec81 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:39:47.029647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17f55e34-65bf-4e0a-acdd-75b4b05eec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:47.147148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f55e34-65bf-4e0a-acdd-75b4b05eec81 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:39:47.228235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:47.230646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5341e-a295-4dfa-a70e-74b511f32eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:39:47.231301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5341e-a295-4dfa-a70e-74b511f32eed service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:47.231636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5341e-a295-4dfa-a70e-74b511f32eed service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:47.232598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5341e-a295-4dfa-a70e-74b511f32eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:47.234934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4a5341e-a295-4dfa-a70e-74b511f32eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:47.348560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5341e-a295-4dfa-a70e-74b511f32eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:39:47.350142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5341e-a295-4dfa-a70e-74b511f32eed service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:39:47.369447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 663/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:39:50.101156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b2a4-f2c4-4eb7-85ff-df9ab238f8ed None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:50.101740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b2a4-f2c4-4eb7-85ff-df9ab238f8ed None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:50.101740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b2a4-f2c4-4eb7-85ff-df9ab238f8ed None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:50.103608 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 652/1315] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:39:50.112239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07261-e028-4c22-9bd3-4e093cf9837e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:50.112658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07261-e028-4c22-9bd3-4e093cf9837e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:50.112771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07261-e028-4c22-9bd3-4e093cf9837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:50.180023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b8c07261-e028-4c22-9bd3-4e093cf9837e None None] MFA Rules not processed for user `a451c98d481d4228a63985e55b2d8871`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:39:50.181220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b8c07261-e028-4c22-9bd3-4e093cf9837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:50.233803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:50.285166 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 664/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:39:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:39:50.632751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b2a4-f2c4-4eb7-85ff-df9ab238f8ed None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:39:50.635275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dca1b2a4-f2c4-4eb7-85ff-df9ab238f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:50.649412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:50.750938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b2a4-f2c4-4eb7-85ff-df9ab238f8ed None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:39:50.829362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1e14c-a808-4e33-840b-ca0cf455f795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:39:50.830007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1e14c-a808-4e33-840b-ca0cf455f795 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:50.830285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1e14c-a808-4e33-840b-ca0cf455f795 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:50.830546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1e14c-a808-4e33-840b-ca0cf455f795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:50.832131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cad1e14c-a808-4e33-840b-ca0cf455f795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:50.896494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:50.951285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1e14c-a808-4e33-840b-ca0cf455f795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:39:50.952743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1e14c-a808-4e33-840b-ca0cf455f795 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:39:50.973713 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 653/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:39:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:39:53.904809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec77bb-3773-4ac2-821b-a4829b2f67db None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:53.906113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec77bb-3773-4ac2-821b-a4829b2f67db None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:53.906113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec77bb-3773-4ac2-821b-a4829b2f67db None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:53.908251 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 665/1318] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:39:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:39:53.914764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1a23-33db-47e0-94b1-659ee4eb5ec0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:53.915225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1a23-33db-47e0-94b1-659ee4eb5ec0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:53.915323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1a23-33db-47e0-94b1-659ee4eb5ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:53.986307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e22d1a23-33db-47e0-94b1-659ee4eb5ec0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:39:53.987433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e22d1a23-33db-47e0-94b1-659ee4eb5ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:54.095731 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 654/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:39:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:39:54.452865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec77bb-3773-4ac2-821b-a4829b2f67db None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:39:54.455364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5fec77bb-3773-4ac2-821b-a4829b2f67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:54.585294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec77bb-3773-4ac2-821b-a4829b2f67db None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:39:54.660416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c71335-2d04-4b66-9405-d7eb812af645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:39:54.661033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c71335-2d04-4b66-9405-d7eb812af645 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:54.661311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c71335-2d04-4b66-9405-d7eb812af645 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:54.661611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c71335-2d04-4b66-9405-d7eb812af645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:54.663206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-44c71335-2d04-4b66-9405-d7eb812af645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:54.670220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:54.784549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c71335-2d04-4b66-9405-d7eb812af645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:39:54.786399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c71335-2d04-4b66-9405-d7eb812af645 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:39:54.804431 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 666/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:39:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:39:57.825092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a677a7-3b0b-4b88-bfe6-fa0013ac2ccf None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:57.825701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a677a7-3b0b-4b88-bfe6-fa0013ac2ccf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:57.825701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a677a7-3b0b-4b88-bfe6-fa0013ac2ccf None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:57.827071 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 655/1321] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:39:57.832740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8133a7-6b8f-4f7d-aa7a-716271cdbf13 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:57.833153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8133a7-6b8f-4f7d-aa7a-716271cdbf13 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:57.833311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8133a7-6b8f-4f7d-aa7a-716271cdbf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:57.902668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3b8133a7-6b8f-4f7d-aa7a-716271cdbf13 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:39:57.903866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b8133a7-6b8f-4f7d-aa7a-716271cdbf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:57.966114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:58.007212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 667/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:39:58.343730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a677a7-3b0b-4b88-bfe6-fa0013ac2ccf None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:39:58.345352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a4a677a7-3b0b-4b88-bfe6-fa0013ac2ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:58.366674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:58.458654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a677a7-3b0b-4b88-bfe6-fa0013ac2ccf None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:39:58.536549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6b793-b2ce-4c09-909f-ab99cf60de94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:39:58.537161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b793-b2ce-4c09-909f-ab99cf60de94 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:39:58.537435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b793-b2ce-4c09-909f-ab99cf60de94 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:39:58.537732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b793-b2ce-4c09-909f-ab99cf60de94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:39:58.539310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-59d6b793-b2ce-4c09-909f-ab99cf60de94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:39:58.616804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:39:58.656110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6b793-b2ce-4c09-909f-ab99cf60de94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:39:58.657701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6b793-b2ce-4c09-909f-ab99cf60de94 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:39:58.676287 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 656/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:40:01.148793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab7c43-a5b7-4605-8814-3602d302d75e None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:01.149377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab7c43-a5b7-4605-8814-3602d302d75e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:01.149377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab7c43-a5b7-4605-8814-3602d302d75e None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:01.150749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 668/1324] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:01.156505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae9cd-bf80-4907-8c60-39eea644f22a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:01.156987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae9cd-bf80-4907-8c60-39eea644f22a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:01.156987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae9cd-bf80-4907-8c60-39eea644f22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:01.228035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d14ae9cd-bf80-4907-8c60-39eea644f22a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:01.229257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d14ae9cd-bf80-4907-8c60-39eea644f22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:01.330865 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 657/1325] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:01.685390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd48ea4-ee05-43f9-904e-6622151f3ce1 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:01.685795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd48ea4-ee05-43f9-904e-6622151f3ce1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:01.685989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd48ea4-ee05-43f9-904e-6622151f3ce1 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:01.687503 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 669/1326] 10.222.0.26 () {54 vars in 963 bytes} [Sat Sep 17 20:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:01.693744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14af9f2-a78e-4611-b720-1e0295f9c8a7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:01.694026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14af9f2-a78e-4611-b720-1e0295f9c8a7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:01.694284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14af9f2-a78e-4611-b720-1e0295f9c8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:01.728698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:01.775637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f14af9f2-a78e-4611-b720-1e0295f9c8a7 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:01.776859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f14af9f2-a78e-4611-b720-1e0295f9c8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:01.852618 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 658/1327] 10.222.0.26 () {58 vars in 1060 bytes} [Sat Sep 17 20:40:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:01.861737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd48ea4-ee05-43f9-904e-6622151f3ce1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:01.863261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bdd48ea4-ee05-43f9-904e-6622151f3ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:01.969919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd48ea4-ee05-43f9-904e-6622151f3ce1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:02.048523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f14fb0-480b-4795-953b-3d559eeafed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:02.049177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14fb0-480b-4795-953b-3d559eeafed2 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:02.049400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14fb0-480b-4795-953b-3d559eeafed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:02.049723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f14fb0-480b-4795-953b-3d559eeafed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:02.051526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-27f14fb0-480b-4795-953b-3d559eeafed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:02.072263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:02.181322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f14fb0-480b-4795-953b-3d559eeafed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:02.182821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f14fb0-480b-4795-953b-3d559eeafed2 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:02.202189 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 670/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:04.855264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3555a11-bc7a-42a4-8e5e-40305f2cc64f None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:04.855876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3555a11-bc7a-42a4-8e5e-40305f2cc64f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:04.855876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3555a11-bc7a-42a4-8e5e-40305f2cc64f None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:04.857213 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 659/1329] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:04.863255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930dc3f-f2ef-4b1c-b824-b5613815e867 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:04.863838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930dc3f-f2ef-4b1c-b824-b5613815e867 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:04.863948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930dc3f-f2ef-4b1c-b824-b5613815e867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:04.932789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7930dc3f-f2ef-4b1c-b824-b5613815e867 None None] MFA Rules not processed for user `a451c98d481d4228a63985e55b2d8871`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:04.934055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7930dc3f-f2ef-4b1c-b824-b5613815e867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:05.002471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:05.041884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 671/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:05.384722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3555a11-bc7a-42a4-8e5e-40305f2cc64f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:05.386342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a3555a11-bc7a-42a4-8e5e-40305f2cc64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:05.505390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3555a11-bc7a-42a4-8e5e-40305f2cc64f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:05.526964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:05.583737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c130ce0-30ae-40f2-92d6-7a63a325dee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:05.584354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130ce0-30ae-40f2-92d6-7a63a325dee7 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:05.584623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130ce0-30ae-40f2-92d6-7a63a325dee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:05.584905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130ce0-30ae-40f2-92d6-7a63a325dee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:05.586508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c130ce0-30ae-40f2-92d6-7a63a325dee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:05.705930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c130ce0-30ae-40f2-92d6-7a63a325dee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:05.707567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c130ce0-30ae-40f2-92d6-7a63a325dee7 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:05.725273 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 660/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:08.469987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf52e-86e8-4ad5-be6e-44a1411d4d66 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:08.470530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf52e-86e8-4ad5-be6e-44a1411d4d66 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:08.470530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf52e-86e8-4ad5-be6e-44a1411d4d66 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:08.472153 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 672/1332] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:08.478006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ddce2-063d-4ab2-b0aa-8c9e76447a88 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:08.478521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ddce2-063d-4ab2-b0aa-8c9e76447a88 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:08.478521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ddce2-063d-4ab2-b0aa-8c9e76447a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:08.549823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:08.552808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fd9ddce2-063d-4ab2-b0aa-8c9e76447a88 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:08.554374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd9ddce2-063d-4ab2-b0aa-8c9e76447a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:08.674154 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 661/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:08.993650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf52e-86e8-4ad5-be6e-44a1411d4d66 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:08.995106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cefaf52e-86e8-4ad5-be6e-44a1411d4d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:09.104070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf52e-86e8-4ad5-be6e-44a1411d4d66 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:09.181530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafee8a-7f8d-4dc1-a661-0673a193c1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:09.182151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafee8a-7f8d-4dc1-a661-0673a193c1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:09.182429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafee8a-7f8d-4dc1-a661-0673a193c1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:09.182699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafee8a-7f8d-4dc1-a661-0673a193c1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:09.184314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3cafee8a-7f8d-4dc1-a661-0673a193c1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:09.215154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:09.305216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafee8a-7f8d-4dc1-a661-0673a193c1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:09.306706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafee8a-7f8d-4dc1-a661-0673a193c1f0 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:09.324642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 673/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:11.754215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f294b-6d5f-4b43-815b-aed9b9207a4b None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:11.754767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f294b-6d5f-4b43-815b-aed9b9207a4b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:11.754767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f294b-6d5f-4b43-815b-aed9b9207a4b None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:11.756104 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 662/1335] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:11.761863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ba573-7836-49b4-937e-2c637db79e60 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:11.762313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ba573-7836-49b4-937e-2c637db79e60 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:11.762405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ba573-7836-49b4-937e-2c637db79e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:11.840261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-113ba573-7836-49b4-937e-2c637db79e60 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:11.841506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-113ba573-7836-49b4-937e-2c637db79e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:11.931759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:11.963697 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 674/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:12.311360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f294b-6d5f-4b43-815b-aed9b9207a4b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:12.313000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-271f294b-6d5f-4b43-815b-aed9b9207a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:12.427274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f294b-6d5f-4b43-815b-aed9b9207a4b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:12.460002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:12.517874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58970b43-6871-450a-bc5f-b2d0f25c218b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:12.518651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58970b43-6871-450a-bc5f-b2d0f25c218b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:12.518931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58970b43-6871-450a-bc5f-b2d0f25c218b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:12.519185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58970b43-6871-450a-bc5f-b2d0f25c218b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:12.521002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58970b43-6871-450a-bc5f-b2d0f25c218b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:12.641396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58970b43-6871-450a-bc5f-b2d0f25c218b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:12.642859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58970b43-6871-450a-bc5f-b2d0f25c218b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:12.661432 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 663/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:15.358233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ba395-4c41-4075-b146-dbb5e4f09ae3 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:15.358843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ba395-4c41-4075-b146-dbb5e4f09ae3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:15.358843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ba395-4c41-4075-b146-dbb5e4f09ae3 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:15.360621 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 675/1338] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:15.366105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a44f13-bce5-42a0-9c84-2af5d7da363a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:15.366612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a44f13-bce5-42a0-9c84-2af5d7da363a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:15.366612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a44f13-bce5-42a0-9c84-2af5d7da363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:15.433630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b5a44f13-bce5-42a0-9c84-2af5d7da363a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:15.434910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b5a44f13-bce5-42a0-9c84-2af5d7da363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:15.443495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:15.545684 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 664/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:15.878362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ba395-4c41-4075-b146-dbb5e4f09ae3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:15.880066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9d9ba395-4c41-4075-b146-dbb5e4f09ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:15.993695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ba395-4c41-4075-b146-dbb5e4f09ae3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:16.071472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f6906-ca09-42e5-b3ae-9b601fffc2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:16.072080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6906-ca09-42e5-b3ae-9b601fffc2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:16.072353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6906-ca09-42e5-b3ae-9b601fffc2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:16.072621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6906-ca09-42e5-b3ae-9b601fffc2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:16.074247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0e0f6906-ca09-42e5-b3ae-9b601fffc2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:16.112984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:16.191493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f6906-ca09-42e5-b3ae-9b601fffc2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:16.192949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f6906-ca09-42e5-b3ae-9b601fffc2c8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:16.212499 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 676/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:20.573571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830418-606a-4707-afb6-fd3e95d4d514 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:20.574155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830418-606a-4707-afb6-fd3e95d4d514 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:20.574155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830418-606a-4707-afb6-fd3e95d4d514 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:20.576192 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 665/1341] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:20.582122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633ea8a-e01d-4cdd-bd8e-da27afaada65 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:20.582632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633ea8a-e01d-4cdd-bd8e-da27afaada65 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:20.582913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633ea8a-e01d-4cdd-bd8e-da27afaada65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:20.652271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7633ea8a-e01d-4cdd-bd8e-da27afaada65 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:20.653398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7633ea8a-e01d-4cdd-bd8e-da27afaada65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:20.751212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:20.762649 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 677/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:21.103194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830418-606a-4707-afb6-fd3e95d4d514 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:21.104823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d830418-606a-4707-afb6-fd3e95d4d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:21.214381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830418-606a-4707-afb6-fd3e95d4d514 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:21.255867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:21.297088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4c8a-2938-4b15-b8d3-45408c7b1938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:21.297760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4c8a-2938-4b15-b8d3-45408c7b1938 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:21.298016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4c8a-2938-4b15-b8d3-45408c7b1938 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:21.298319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4c8a-2938-4b15-b8d3-45408c7b1938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:21.299920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-872a4c8a-2938-4b15-b8d3-45408c7b1938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:21.415439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4c8a-2938-4b15-b8d3-45408c7b1938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:21.416923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4c8a-2938-4b15-b8d3-45408c7b1938 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:21.435948 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 666/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:24.492580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360f64-aa8a-48d5-8bae-67550a54d0d4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:24.493141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360f64-aa8a-48d5-8bae-67550a54d0d4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:24.493141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db360f64-aa8a-48d5-8bae-67550a54d0d4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:24.494502 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 678/1344] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:24.500548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bc966-0686-4ccf-a412-baefce15dc78 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:24.501030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bc966-0686-4ccf-a412-baefce15dc78 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:24.501130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bc966-0686-4ccf-a412-baefce15dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:24.569622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5a1bc966-0686-4ccf-a412-baefce15dc78 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:24.570842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5a1bc966-0686-4ccf-a412-baefce15dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:24.589817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:24.676558 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 667/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:25.015820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360f64-aa8a-48d5-8bae-67550a54d0d4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:25.017393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-db360f64-aa8a-48d5-8bae-67550a54d0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:25.127459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db360f64-aa8a-48d5-8bae-67550a54d0d4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:25.206420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c4cf0-da12-4c98-ac05-d0532d252de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:25.207039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c4cf0-da12-4c98-ac05-d0532d252de2 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:25.207373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c4cf0-da12-4c98-ac05-d0532d252de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:25.207633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c4cf0-da12-4c98-ac05-d0532d252de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:25.209226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e37c4cf0-da12-4c98-ac05-d0532d252de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:25.253874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:25.321451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c4cf0-da12-4c98-ac05-d0532d252de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:25.322868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c4cf0-da12-4c98-ac05-d0532d252de2 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:25.342031 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 679/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:29.697662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9c089-a6c1-49aa-b9d1-48c65b1eba2e None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:29.698132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9c089-a6c1-49aa-b9d1-48c65b1eba2e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:29.698246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9c089-a6c1-49aa-b9d1-48c65b1eba2e None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:29.699603 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 668/1347] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:29.705225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c459e40-3aa3-4f3d-95a9-9a3bf123e3d7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:29.705742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c459e40-3aa3-4f3d-95a9-9a3bf123e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:29.705844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c459e40-3aa3-4f3d-95a9-9a3bf123e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:29.773588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6c459e40-3aa3-4f3d-95a9-9a3bf123e3d7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:29.774740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6c459e40-3aa3-4f3d-95a9-9a3bf123e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:29.879636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:29.885513 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 680/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:30.224737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9c089-a6c1-49aa-b9d1-48c65b1eba2e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:30.226982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98d9c089-a6c1-49aa-b9d1-48c65b1eba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:30.339273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9c089-a6c1-49aa-b9d1-48c65b1eba2e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:30.389153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:30.417953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d041415-2515-4aff-9bf3-59b5e09aead9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:30.418788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d041415-2515-4aff-9bf3-59b5e09aead9 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:30.419073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d041415-2515-4aff-9bf3-59b5e09aead9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:30.419348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d041415-2515-4aff-9bf3-59b5e09aead9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:30.421010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d041415-2515-4aff-9bf3-59b5e09aead9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:30.537506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d041415-2515-4aff-9bf3-59b5e09aead9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:30.538976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d041415-2515-4aff-9bf3-59b5e09aead9 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:30.559489 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 669/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:35.924926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58ef3-a766-461e-8f09-bd6985705369 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:35.925643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58ef3-a766-461e-8f09-bd6985705369 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:35.925643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58ef3-a766-461e-8f09-bd6985705369 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:35.942084 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 681/1350] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:40:35 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:35.949517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f9e17-1913-4946-a500-67c3e077e37c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:35.950043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f9e17-1913-4946-a500-67c3e077e37c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:35.950043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f9e17-1913-4946-a500-67c3e077e37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:36.064607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9e9f9e17-1913-4946-a500-67c3e077e37c None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:36.065956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e9f9e17-1913-4946-a500-67c3e077e37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:36.085763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:36.185169 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 670/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:40:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:40.520562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0ccb-0b0d-44d3-a148-c7245bc1b6af None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:40.521416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0ccb-0b0d-44d3-a148-c7245bc1b6af None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:40.521768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e0ccb-0b0d-44d3-a148-c7245bc1b6af None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:40.523313 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 682/1352] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:40.528349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ffbb-2f36-4534-bc8f-91eb49bf496b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:40.528808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ffbb-2f36-4534-bc8f-91eb49bf496b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:40.528893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ffbb-2f36-4534-bc8f-91eb49bf496b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:40.609658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c2f3ffbb-2f36-4534-bc8f-91eb49bf496b None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:40.610876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c2f3ffbb-2f36-4534-bc8f-91eb49bf496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:40.716297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:40.757122 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 671/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:40:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:42.380416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1473878-8131-46b7-bce4-6987409f6848 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:42.381030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1473878-8131-46b7-bce4-6987409f6848 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:42.381030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1473878-8131-46b7-bce4-6987409f6848 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:42.382403 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 683/1354] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:42.388173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736c29-3d34-4792-944b-718acce99b32 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:42.388699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736c29-3d34-4792-944b-718acce99b32 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:42.388699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736c29-3d34-4792-944b-718acce99b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:42.469019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-80736c29-3d34-4792-944b-718acce99b32 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:42.470337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-80736c29-3d34-4792-944b-718acce99b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:42.576479 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 672/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:40:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:46.328757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188189-4c03-4c72-ac5f-6efef1106401 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:46.329929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188189-4c03-4c72-ac5f-6efef1106401 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:46.330441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188189-4c03-4c72-ac5f-6efef1106401 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:46.334167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 684/1356] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:46.340743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5f06-7c9c-427a-ae16-2c8b2ba01189 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:46.341663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5f06-7c9c-427a-ae16-2c8b2ba01189 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:46.343147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5f06-7c9c-427a-ae16-2c8b2ba01189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:46.385143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:46.428377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e28f5f06-7c9c-427a-ae16-2c8b2ba01189 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:46.429774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e28f5f06-7c9c-427a-ae16-2c8b2ba01189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:46.528859 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 673/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:46.883623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cb0c3-7e01-43f1-827b-3231d065d560 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:46.884024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cb0c3-7e01-43f1-827b-3231d065d560 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:46.884391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cb0c3-7e01-43f1-827b-3231d065d560 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:46.885880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 685/1358] 10.222.0.26 () {54 vars in 965 bytes} [Sat Sep 17 20:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:46.894995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f41287-20cc-432f-bac7-acb4630f10c1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:46.895341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f41287-20cc-432f-bac7-acb4630f10c1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:46.895717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f41287-20cc-432f-bac7-acb4630f10c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:46.990279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-26f41287-20cc-432f-bac7-acb4630f10c1 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:46.991558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-26f41287-20cc-432f-bac7-acb4630f10c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:47.043642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:47.118001 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 674/1359] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 17 20:40:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:40:47.127143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cb0c3-7e01-43f1-827b-3231d065d560 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:47.128728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-078cb0c3-7e01-43f1-827b-3231d065d560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:47.287828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cb0c3-7e01-43f1-827b-3231d065d560 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:47.364832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:47.408140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e660-ba84-4e2a-b83b-d87966fe907c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:47.408809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e660-ba84-4e2a-b83b-d87966fe907c service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:47.409204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e660-ba84-4e2a-b83b-d87966fe907c service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:47.409505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e660-ba84-4e2a-b83b-d87966fe907c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:47.411390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc49e660-ba84-4e2a-b83b-d87966fe907c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:47.507542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e660-ba84-4e2a-b83b-d87966fe907c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:47.509329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49e660-ba84-4e2a-b83b-d87966fe907c service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:47.529016 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 686/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:40:50.149547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9677-7ca0-4187-ab2e-d599735380d5 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:50.150159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9677-7ca0-4187-ab2e-d599735380d5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:50.150159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9677-7ca0-4187-ab2e-d599735380d5 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:50.151824 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 675/1361] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:50.158339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda472b-c34a-4fd4-9231-088fb28735a1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:50.158915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda472b-c34a-4fd4-9231-088fb28735a1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:50.158915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda472b-c34a-4fd4-9231-088fb28735a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:50.237811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8bda472b-c34a-4fd4-9231-088fb28735a1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:50.238946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8bda472b-c34a-4fd4-9231-088fb28735a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:50.281120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:50.350364 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 687/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:50.689780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a9677-7ca0-4187-ab2e-d599735380d5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:50.692890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd8a9677-7ca0-4187-ab2e-d599735380d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:50.857237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a9677-7ca0-4187-ab2e-d599735380d5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:50.866982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:50.974211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede138b-d87c-4839-ab50-882e4251e03d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:50.974368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede138b-d87c-4839-ab50-882e4251e03d service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:50.974943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede138b-d87c-4839-ab50-882e4251e03d service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:50.974943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede138b-d87c-4839-ab50-882e4251e03d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:50.976470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5ede138b-d87c-4839-ab50-882e4251e03d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:51.071711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede138b-d87c-4839-ab50-882e4251e03d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:51.073311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede138b-d87c-4839-ab50-882e4251e03d service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:51.092544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 676/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:53.797693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f7237-adc3-46b9-b714-a18136bbe266 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:53.798226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f7237-adc3-46b9-b714-a18136bbe266 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:53.798226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f7237-adc3-46b9-b714-a18136bbe266 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:53.799708 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 688/1364] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:40:53.805030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e0355-0536-499f-ba88-c992d12e7102 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:53.805508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e0355-0536-499f-ba88-c992d12e7102 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:53.805799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e0355-0536-499f-ba88-c992d12e7102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:53.823213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:53.881665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-752e0355-0536-499f-ba88-c992d12e7102 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:53.883166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-752e0355-0536-499f-ba88-c992d12e7102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:54.003953 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 677/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:54.411928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d946f0-d821-433b-b358-1ae79889190c None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:54.412577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d946f0-d821-433b-b358-1ae79889190c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:54.412850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d946f0-d821-433b-b358-1ae79889190c None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:54.415212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 689/1366] 10.222.0.26 () {54 vars in 965 bytes} [Sat Sep 17 20:40:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:54.422434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb192-8e62-41ca-ae98-391e9748e1c6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:54.422953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb192-8e62-41ca-ae98-391e9748e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:54.423420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb192-8e62-41ca-ae98-391e9748e1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:54.537044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c8deb192-8e62-41ca-ae98-391e9748e1c6 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:54.538539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8deb192-8e62-41ca-ae98-391e9748e1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:54.590478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:54.700333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d946f0-d821-433b-b358-1ae79889190c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:54.702318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25d946f0-d821-433b-b358-1ae79889190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:54.708148 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 678/1367] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 17 20:40:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:54.919425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d946f0-d821-433b-b358-1ae79889190c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:54.947095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:55.083732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e1259-a0f1-450b-8cfd-2b1c9794626d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:55.084515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e1259-a0f1-450b-8cfd-2b1c9794626d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:55.084884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e1259-a0f1-450b-8cfd-2b1c9794626d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:55.085248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e1259-a0f1-450b-8cfd-2b1c9794626d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:55.095864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-525e1259-a0f1-450b-8cfd-2b1c9794626d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:55.240367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e1259-a0f1-450b-8cfd-2b1c9794626d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:55.242482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e1259-a0f1-450b-8cfd-2b1c9794626d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:55.275736 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 690/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:58.092339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e7b8d-3419-4ddd-87c6-12a60e4ff1f7 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:58.092891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e7b8d-3419-4ddd-87c6-12a60e4ff1f7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:58.092891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e7b8d-3419-4ddd-87c6-12a60e4ff1f7 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:58.095224 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 679/1369] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:58.101324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2a48f-3122-4ca0-91bf-a7c00fc9cecc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:58.101847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2a48f-3122-4ca0-91bf-a7c00fc9cecc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:58.102353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2a48f-3122-4ca0-91bf-a7c00fc9cecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:58.222100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:58.241561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2ed2a48f-3122-4ca0-91bf-a7c00fc9cecc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:58.243079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2ed2a48f-3122-4ca0-91bf-a7c00fc9cecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:58.389609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 691/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:40:58.740721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e7b8d-3419-4ddd-87c6-12a60e4ff1f7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:40:58.742996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f07e7b8d-3419-4ddd-87c6-12a60e4ff1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:58.898086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:58.903699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e7b8d-3419-4ddd-87c6-12a60e4ff1f7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:40:59.027853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f44bac-0451-43ae-8318-d72d81cb717e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:40:59.028719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f44bac-0451-43ae-8318-d72d81cb717e service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:59.029019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f44bac-0451-43ae-8318-d72d81cb717e service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:59.029386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f44bac-0451-43ae-8318-d72d81cb717e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:59.031529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-37f44bac-0451-43ae-8318-d72d81cb717e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:59.136854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f44bac-0451-43ae-8318-d72d81cb717e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:40:59.138509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f44bac-0451-43ae-8318-d72d81cb717e service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:40:59.159199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:40:59.162972 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 680/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:40:59.174787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ecf6f-1bac-468c-93b9-33220075379e None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:59.175736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ecf6f-1bac-468c-93b9-33220075379e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:59.176008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ecf6f-1bac-468c-93b9-33220075379e None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:59.177559 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 692/1372] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:40:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:59.183840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647853-120d-4657-a231-7b989b47f967 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:59.184124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647853-120d-4657-a231-7b989b47f967 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:59.184433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647853-120d-4657-a231-7b989b47f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:59.270111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9b647853-120d-4657-a231-7b989b47f967 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:59.271478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b647853-120d-4657-a231-7b989b47f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:59.390340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 681/1373] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:40:59.820431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076176c6-f96d-46ea-9914-45fb6e593679 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:59.821169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076176c6-f96d-46ea-9914-45fb6e593679 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:59.821660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076176c6-f96d-46ea-9914-45fb6e593679 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:59.823223 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 693/1374] 10.222.0.26 () {54 vars in 964 bytes} [Sat Sep 17 20:40:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:40:59.828629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0935450-2596-44fa-bd1f-10c225f55b51 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:40:59.828909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0935450-2596-44fa-bd1f-10c225f55b51 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:40:59.829186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0935450-2596-44fa-bd1f-10c225f55b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:40:59.911607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a0935450-2596-44fa-bd1f-10c225f55b51 None None] MFA Rules not processed for user `d45ab41902864586811c81948540fdac`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:40:59.912982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a0935450-2596-44fa-bd1f-10c225f55b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:40:59.941135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:00.002982 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 682/1375] 10.222.0.26 () {58 vars in 1061 bytes} [Sat Sep 17 20:40:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:00.011477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076176c6-f96d-46ea-9914-45fb6e593679 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:00.012993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-076176c6-f96d-46ea-9914-45fb6e593679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:00.138459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076176c6-f96d-46ea-9914-45fb6e593679 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:00.143018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:00.220443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae3264-0e2e-45de-b389-7f3403375ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:00.221179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae3264-0e2e-45de-b389-7f3403375ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:00.221453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae3264-0e2e-45de-b389-7f3403375ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:00.221607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae3264-0e2e-45de-b389-7f3403375ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:00.223277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ccae3264-0e2e-45de-b389-7f3403375ce4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:00.355976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae3264-0e2e-45de-b389-7f3403375ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:00.357653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae3264-0e2e-45de-b389-7f3403375ce4 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:00.395321 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 694/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:01.837828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fe04e-d68f-4497-b8f1-786efa9e9739 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:01.838599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fe04e-d68f-4497-b8f1-786efa9e9739 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:01.838599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fe04e-d68f-4497-b8f1-786efa9e9739 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:01.839780 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 683/1377] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:01.846188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c322-b499-42dd-a1f8-6585284d8b88 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:01.846841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c322-b499-42dd-a1f8-6585284d8b88 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:01.846841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c322-b499-42dd-a1f8-6585284d8b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:01.900358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:01.926606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0111c322-b499-42dd-a1f8-6585284d8b88 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:01.927899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0111c322-b499-42dd-a1f8-6585284d8b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:02.029204 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 695/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:02.471440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fe04e-d68f-4497-b8f1-786efa9e9739 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:02.473011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a59fe04e-d68f-4497-b8f1-786efa9e9739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:02.619338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fe04e-d68f-4497-b8f1-786efa9e9739 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:02.651814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:02.752733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cff8ee-8696-481c-912b-2ec40c06a0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:02.753348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff8ee-8696-481c-912b-2ec40c06a0e7 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:02.753665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff8ee-8696-481c-912b-2ec40c06a0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:02.754007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff8ee-8696-481c-912b-2ec40c06a0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:02.755609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8cff8ee-8696-481c-912b-2ec40c06a0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:02.859745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cff8ee-8696-481c-912b-2ec40c06a0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:02.861395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cff8ee-8696-481c-912b-2ec40c06a0e7 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:02.880225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 684/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:05.345770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eae946-292f-4187-9964-1c2ae66f4764 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:05.346585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eae946-292f-4187-9964-1c2ae66f4764 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:05.346585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eae946-292f-4187-9964-1c2ae66f4764 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:05.348281 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 696/1380] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:05.354245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e0f16-7d2b-4157-9e91-55fea2dfa491 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:05.354930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e0f16-7d2b-4157-9e91-55fea2dfa491 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:05.354930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e0f16-7d2b-4157-9e91-55fea2dfa491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:05.401209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:05.430472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-086e0f16-7d2b-4157-9e91-55fea2dfa491 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:05.431697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-086e0f16-7d2b-4157-9e91-55fea2dfa491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:05.550320 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 685/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:05.926310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eae946-292f-4187-9964-1c2ae66f4764 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:05.927365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49eae946-292f-4187-9964-1c2ae66f4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:06.047413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:06.089827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eae946-292f-4187-9964-1c2ae66f4764 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:06.218738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd526672-15a4-4efb-844a-4441517d5f0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:06.219345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526672-15a4-4efb-844a-4441517d5f0a service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:06.219622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526672-15a4-4efb-844a-4441517d5f0a service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:06.219892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526672-15a4-4efb-844a-4441517d5f0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:06.221507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dd526672-15a4-4efb-844a-4441517d5f0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:06.307468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:06.329973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd526672-15a4-4efb-844a-4441517d5f0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:06.331816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd526672-15a4-4efb-844a-4441517d5f0a service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:06.353146 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 697/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:08.831819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abede82-b5b3-4157-bd47-11b4f1c96490 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:08.832474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abede82-b5b3-4157-bd47-11b4f1c96490 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:08.832474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abede82-b5b3-4157-bd47-11b4f1c96490 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:08.833834 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 686/1383] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:08.839202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732ac9c-0730-4289-877e-0284ad7667af None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:08.839709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732ac9c-0730-4289-877e-0284ad7667af None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:08.839813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732ac9c-0730-4289-877e-0284ad7667af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:08.909943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b732ac9c-0730-4289-877e-0284ad7667af None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:08.911358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b732ac9c-0730-4289-877e-0284ad7667af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:09.021557 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 698/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:09.472664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abede82-b5b3-4157-bd47-11b4f1c96490 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:09.474758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5abede82-b5b3-4157-bd47-11b4f1c96490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:09.604494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:09.658118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abede82-b5b3-4157-bd47-11b4f1c96490 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:09.816551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec5093-9e4b-402a-8192-e71d109aad5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:09.817266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec5093-9e4b-402a-8192-e71d109aad5d service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:09.817744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec5093-9e4b-402a-8192-e71d109aad5d service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:09.818083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec5093-9e4b-402a-8192-e71d109aad5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:09.820230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4fec5093-9e4b-402a-8192-e71d109aad5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:09.912651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:09.931963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec5093-9e4b-402a-8192-e71d109aad5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:09.933667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec5093-9e4b-402a-8192-e71d109aad5d service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:09.953672 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 687/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:12.550547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5167784-8927-4ecb-967f-95d4f07318e7 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:12.551232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5167784-8927-4ecb-967f-95d4f07318e7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:12.551232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5167784-8927-4ecb-967f-95d4f07318e7 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:12.552497 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 699/1386] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:12.559787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c033-7758-4c3b-bb08-86ee7108c6fe None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:12.560283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c033-7758-4c3b-bb08-86ee7108c6fe None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:12.560487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c033-7758-4c3b-bb08-86ee7108c6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:12.643076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6bc3c033-7758-4c3b-bb08-86ee7108c6fe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:12.644360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6bc3c033-7758-4c3b-bb08-86ee7108c6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:12.747317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 688/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:13.092467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5167784-8927-4ecb-967f-95d4f07318e7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:13.094086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c5167784-8927-4ecb-967f-95d4f07318e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:13.162736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:13.268103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5167784-8927-4ecb-967f-95d4f07318e7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:13.403873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a7d2e-57be-4e87-b694-1726e1301a7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:13.404505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a7d2e-57be-4e87-b694-1726e1301a7d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:13.404787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a7d2e-57be-4e87-b694-1726e1301a7d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:13.405054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a7d2e-57be-4e87-b694-1726e1301a7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:13.407062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a20a7d2e-57be-4e87-b694-1726e1301a7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:13.422051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:13.509603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a7d2e-57be-4e87-b694-1726e1301a7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:13.511619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a7d2e-57be-4e87-b694-1726e1301a7d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:13.535781 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 700/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:16.180476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900412-6e35-4054-bcfd-def5c6ded045 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:16.181185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900412-6e35-4054-bcfd-def5c6ded045 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:16.181185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900412-6e35-4054-bcfd-def5c6ded045 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:16.182459 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 689/1389] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:16.187929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcbcd-6036-4d66-a800-4dd30a43aaac None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:16.188465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcbcd-6036-4d66-a800-4dd30a43aaac None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:16.188465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcbcd-6036-4d66-a800-4dd30a43aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:16.257115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3d8fcbcd-6036-4d66-a800-4dd30a43aaac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:16.264809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d8fcbcd-6036-4d66-a800-4dd30a43aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:16.363432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:16.375559 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 701/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:16.723911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d900412-6e35-4054-bcfd-def5c6ded045 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:16.725524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0d900412-6e35-4054-bcfd-def5c6ded045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:16.780593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:16.822528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51948269-908a-40a6-a8ef-e675eec74c29 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:16.822779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51948269-908a-40a6-a8ef-e675eec74c29 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:16.823092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51948269-908a-40a6-a8ef-e675eec74c29 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:16.824465 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 702/1391] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:16.830647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c25a89-2269-4cb7-804c-2f9c4bba15a5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:16.830984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c25a89-2269-4cb7-804c-2f9c4bba15a5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:16.831215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c25a89-2269-4cb7-804c-2f9c4bba15a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:16.882799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d900412-6e35-4054-bcfd-def5c6ded045 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:16.901207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d2c25a89-2269-4cb7-804c-2f9c4bba15a5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:16.902472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d2c25a89-2269-4cb7-804c-2f9c4bba15a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:16.998545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09568d5c-fc2c-4092-b8d5-da56cebd5a15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:16.999067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09568d5c-fc2c-4092-b8d5-da56cebd5a15 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:16.999341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09568d5c-fc2c-4092-b8d5-da56cebd5a15 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:16.999611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09568d5c-fc2c-4092-b8d5-da56cebd5a15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:17.001189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-09568d5c-fc2c-4092-b8d5-da56cebd5a15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:17.003141 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 703/1392] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:17.016478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:17.094155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09568d5c-fc2c-4092-b8d5-da56cebd5a15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:17.095781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09568d5c-fc2c-4092-b8d5-da56cebd5a15 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:17.113812 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 690/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:17.358840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a273a53-593c-478e-9607-803d83ea4e4b None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:17.359445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a273a53-593c-478e-9607-803d83ea4e4b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:17.359445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a273a53-593c-478e-9607-803d83ea4e4b None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:17.360773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 704/1394] 10.222.0.26 () {54 vars in 962 bytes} [Sat Sep 17 20:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:17.366900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e3494-d3d3-4a3e-80a3-dee8af8460b3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:17.367520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e3494-d3d3-4a3e-80a3-dee8af8460b3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:17.367520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e3494-d3d3-4a3e-80a3-dee8af8460b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:17.446373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a94e3494-d3d3-4a3e-80a3-dee8af8460b3 None None] MFA Rules not processed for user `94ce9c64a0bb478485c47ebedd5a29f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:17.447580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a94e3494-d3d3-4a3e-80a3-dee8af8460b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:17.531016 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 691/1395] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Sep 17 20:41:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:17.538968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a273a53-593c-478e-9607-803d83ea4e4b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:17.540527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3a273a53-593c-478e-9607-803d83ea4e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:17.616498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:17.648892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a273a53-593c-478e-9607-803d83ea4e4b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:17.729053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921f559-1362-4c63-819f-6a159cf82368 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:17.729702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921f559-1362-4c63-819f-6a159cf82368 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:17.729982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921f559-1362-4c63-819f-6a159cf82368 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:17.730261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921f559-1362-4c63-819f-6a159cf82368 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:17.731963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d921f559-1362-4c63-819f-6a159cf82368 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:17.843314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921f559-1362-4c63-819f-6a159cf82368 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:17.844776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921f559-1362-4c63-819f-6a159cf82368 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:17.860589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:17.864299 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 705/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:18.455295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589073e4-7208-4c11-9fb7-04962adc1a47 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:18.455795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589073e4-7208-4c11-9fb7-04962adc1a47 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:18.455902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589073e4-7208-4c11-9fb7-04962adc1a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:18.460777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:18.652212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-589073e4-7208-4c11-9fb7-04962adc1a47 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:18.653418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-589073e4-7208-4c11-9fb7-04962adc1a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:18.752203 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 692/1397] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:41:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:18.800478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a5430-1d81-4109-9029-344e6a6efda6 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:18.800978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a5430-1d81-4109-9029-344e6a6efda6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:18.801096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a5430-1d81-4109-9029-344e6a6efda6 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:18.802540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 706/1398] 10.222.0.26 () {54 vars in 943 bytes} [Sat Sep 17 20:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:18.808889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c03d-8424-49d7-87ae-d42bacffb66a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:18.809154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c03d-8424-49d7-87ae-d42bacffb66a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:18.809450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4c03d-8424-49d7-87ae-d42bacffb66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:18.886390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-38f4c03d-8424-49d7-87ae-d42bacffb66a None None] MFA Rules not processed for user `8eea4641bfd940fe8a94ec79484ef4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:18.887592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-38f4c03d-8424-49d7-87ae-d42bacffb66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:18.929529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:19.008151 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 693/1399] 10.222.0.26 () {58 vars in 1040 bytes} [Sat Sep 17 20:41:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:19.018668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a5430-1d81-4109-9029-344e6a6efda6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:19.020257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-328a5430-1d81-4109-9029-344e6a6efda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:19.181033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a5430-1d81-4109-9029-344e6a6efda6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:19.272718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:19.293370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc34e0-d827-4173-9d43-17c8da26bb60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:19.294025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc34e0-d827-4173-9d43-17c8da26bb60 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:19.294297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc34e0-d827-4173-9d43-17c8da26bb60 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:19.294565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc34e0-d827-4173-9d43-17c8da26bb60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:19.296197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c0dc34e0-d827-4173-9d43-17c8da26bb60 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:19.387052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc34e0-d827-4173-9d43-17c8da26bb60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:19.388971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc34e0-d827-4173-9d43-17c8da26bb60 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:19.397796 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 707/1400] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:41:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:41:19.531765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970849-1768-44bb-9b37-25185eb0337c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:19.532384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970849-1768-44bb-9b37-25185eb0337c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:19.533968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970849-1768-44bb-9b37-25185eb0337c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:19.535624 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 694/1401] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:19.541373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9fc07-94f2-4f65-a583-6f2df551bcb4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:19.541690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9fc07-94f2-4f65-a583-6f2df551bcb4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:19.541971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9fc07-94f2-4f65-a583-6f2df551bcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:19.621197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-81c9fc07-94f2-4f65-a583-6f2df551bcb4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:19.622570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81c9fc07-94f2-4f65-a583-6f2df551bcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:19.709266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:19.735916 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 708/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:20.078224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970849-1768-44bb-9b37-25185eb0337c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:20.079751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-45970849-1768-44bb-9b37-25185eb0337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:20.240249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970849-1768-44bb-9b37-25185eb0337c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:20.248299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:20.365634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b527b6-aede-4411-ad4c-3e4a1954957d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:20.366245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b527b6-aede-4411-ad4c-3e4a1954957d service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:20.366517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b527b6-aede-4411-ad4c-3e4a1954957d service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:20.366778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b527b6-aede-4411-ad4c-3e4a1954957d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:20.368351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-31b527b6-aede-4411-ad4c-3e4a1954957d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:20.467568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b527b6-aede-4411-ad4c-3e4a1954957d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:20.469156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b527b6-aede-4411-ad4c-3e4a1954957d service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:20.488151 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 695/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:23.057869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd69a3-ee5b-43a9-a491-5459e891c2bd None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:23.058473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd69a3-ee5b-43a9-a491-5459e891c2bd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:23.058473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd69a3-ee5b-43a9-a491-5459e891c2bd None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:23.063976 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 709/1404] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:23.065531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bef98-177f-4f81-9fa0-a42f36cda811 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:23.066058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bef98-177f-4f81-9fa0-a42f36cda811 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:23.066337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bef98-177f-4f81-9fa0-a42f36cda811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:23.081525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:23.136088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-365bef98-177f-4f81-9fa0-a42f36cda811 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:23.147884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-365bef98-177f-4f81-9fa0-a42f36cda811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:23.256290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 696/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:23.616154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd69a3-ee5b-43a9-a491-5459e891c2bd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:23.617742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-41dd69a3-ee5b-43a9-a491-5459e891c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:23.784741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd69a3-ee5b-43a9-a491-5459e891c2bd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:23.843452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:23.907924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098de29-2a88-47b2-b5cb-aea30c883895 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:23.908649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098de29-2a88-47b2-b5cb-aea30c883895 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:23.908918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098de29-2a88-47b2-b5cb-aea30c883895 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:23.909186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098de29-2a88-47b2-b5cb-aea30c883895 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:23.911058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4098de29-2a88-47b2-b5cb-aea30c883895 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:24.021815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098de29-2a88-47b2-b5cb-aea30c883895 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:24.023472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098de29-2a88-47b2-b5cb-aea30c883895 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:24.045330 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 710/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:26.534572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a36bbb-ba1c-4c56-b5f8-c7e8c851fe7a None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:26.535323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a36bbb-ba1c-4c56-b5f8-c7e8c851fe7a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:26.535323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a36bbb-ba1c-4c56-b5f8-c7e8c851fe7a None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:26.536867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 697/1407] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:26.541969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35991eb2-80b1-484a-bd2c-6c5f1b365da2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:26.542461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35991eb2-80b1-484a-bd2c-6c5f1b365da2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:26.542559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35991eb2-80b1-484a-bd2c-6c5f1b365da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:26.613860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-35991eb2-80b1-484a-bd2c-6c5f1b365da2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:26.615136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-35991eb2-80b1-484a-bd2c-6c5f1b365da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:26.634726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:26.737898 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 711/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:27.082082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a36bbb-ba1c-4c56-b5f8-c7e8c851fe7a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:27.083626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-89a36bbb-ba1c-4c56-b5f8-c7e8c851fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:27.195530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:27.293179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a36bbb-ba1c-4c56-b5f8-c7e8c851fe7a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:27.416262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4e78e-8cf8-4679-ba00-53727603ad3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:27.416880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4e78e-8cf8-4679-ba00-53727603ad3f service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:27.417179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4e78e-8cf8-4679-ba00-53727603ad3f service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:27.417452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4e78e-8cf8-4679-ba00-53727603ad3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:27.419155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-00e4e78e-8cf8-4679-ba00-53727603ad3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:27.482695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:27.524954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4e78e-8cf8-4679-ba00-53727603ad3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:27.526842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4e78e-8cf8-4679-ba00-53727603ad3f service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:27.547312 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 698/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:36.736088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=37 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:37.547444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=22 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:42.823513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f88070-86d1-4880-af38-6d0f11dc6f4c None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:42.824144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f88070-86d1-4880-af38-6d0f11dc6f4c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:42.824144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f88070-86d1-4880-af38-6d0f11dc6f4c None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:42.825403 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 712/1410] 10.222.0.26 () {54 vars in 918 bytes} [Sat Sep 17 20:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:42.830838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1664ddd-27ca-4dfa-95fd-6bf5bef95379 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:42.831266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1664ddd-27ca-4dfa-95fd-6bf5bef95379 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:42.831383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1664ddd-27ca-4dfa-95fd-6bf5bef95379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:42.911884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d1664ddd-27ca-4dfa-95fd-6bf5bef95379 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:42.913587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d1664ddd-27ca-4dfa-95fd-6bf5bef95379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:43.039125 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 699/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 17 20:41:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:43.054028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3accc2-2e74-48ec-a997-f3973cbdf8c7 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:43.054306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3accc2-2e74-48ec-a997-f3973cbdf8c7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:43.054569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3accc2-2e74-48ec-a997-f3973cbdf8c7 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:43.055909 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 713/1412] 10.222.0.26 () {54 vars in 918 bytes} [Sat Sep 17 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:43.060543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88f6fe-734f-4bb1-81e2-1f04cbcf6689 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:43.060822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88f6fe-734f-4bb1-81e2-1f04cbcf6689 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:43.061085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88f6fe-734f-4bb1-81e2-1f04cbcf6689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:43.138125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:43.141500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2a88f6fe-734f-4bb1-81e2-1f04cbcf6689 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:43.143248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a88f6fe-734f-4bb1-81e2-1f04cbcf6689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:43.285438 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 700/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 17 20:41:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:46.139616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83af7-60dd-4efe-b2e7-a74bfc545048 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:46.140611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83af7-60dd-4efe-b2e7-a74bfc545048 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:46.140611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c83af7-60dd-4efe-b2e7-a74bfc545048 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:46.142679 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:41:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:46.149036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6798a1-2abf-47df-95b5-9cd2cca6c72e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:46.149846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6798a1-2abf-47df-95b5-9cd2cca6c72e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:46.149999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6798a1-2abf-47df-95b5-9cd2cca6c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:46.223593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9b6798a1-2abf-47df-95b5-9cd2cca6c72e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:46.224886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b6798a1-2abf-47df-95b5-9cd2cca6c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:46.283803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:46.329732 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:46.672751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c83af7-60dd-4efe-b2e7-a74bfc545048 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:46.674482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c9c83af7-60dd-4efe-b2e7-a74bfc545048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:46.830935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c83af7-60dd-4efe-b2e7-a74bfc545048 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:46.915260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:46.959945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c789e9-6e7d-4421-856b-d1db7e4322d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:46.960550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c789e9-6e7d-4421-856b-d1db7e4322d9 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:46.960827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c789e9-6e7d-4421-856b-d1db7e4322d9 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:46.961083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c789e9-6e7d-4421-856b-d1db7e4322d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:46.962698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4c789e9-6e7d-4421-856b-d1db7e4322d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:47.069621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c789e9-6e7d-4421-856b-d1db7e4322d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:47.071268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c789e9-6e7d-4421-856b-d1db7e4322d9 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:47.095483 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 715/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:48.816016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b76255-474c-425a-b715-25819b05f1b2 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:48.816888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b76255-474c-425a-b715-25819b05f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:48.816888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b76255-474c-425a-b715-25819b05f1b2 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:48.819123 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:41:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:48.825047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94c056-d619-4248-8bcd-b313ee2befb4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:48.825811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94c056-d619-4248-8bcd-b313ee2befb4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:48.825965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94c056-d619-4248-8bcd-b313ee2befb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:48.909037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3a94c056-d619-4248-8bcd-b313ee2befb4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:48.910311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3a94c056-d619-4248-8bcd-b313ee2befb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:48.939038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:49.010622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:49.235133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b76255-474c-425a-b715-25819b05f1b2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:49.237508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-35b76255-474c-425a-b715-25819b05f1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:49.400072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b76255-474c-425a-b715-25819b05f1b2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:49.444501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:49.522390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dc29f-b444-471d-86cf-8ab73a0e07c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:49.522960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dc29f-b444-471d-86cf-8ab73a0e07c6 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:49.523231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dc29f-b444-471d-86cf-8ab73a0e07c6 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:49.523496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dc29f-b444-471d-86cf-8ab73a0e07c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:49.525101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-612dc29f-b444-471d-86cf-8ab73a0e07c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:49.622890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dc29f-b444-471d-86cf-8ab73a0e07c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:49.624500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dc29f-b444-471d-86cf-8ab73a0e07c6 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:49.643598 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 703/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:51.341955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f117e0e-8492-47e6-af17-b2da127a1d6a None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:51.342524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f117e0e-8492-47e6-af17-b2da127a1d6a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:51.342524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f117e0e-8492-47e6-af17-b2da127a1d6a None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:51.343768 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:51.350016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01115c45-8478-4ed2-87c1-13eaf69e43b7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:51.350525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01115c45-8478-4ed2-87c1-13eaf69e43b7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:51.350525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01115c45-8478-4ed2-87c1-13eaf69e43b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:51.417958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:51.443058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-01115c45-8478-4ed2-87c1-13eaf69e43b7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:51.444321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-01115c45-8478-4ed2-87c1-13eaf69e43b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:51.546362 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:51.709229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f117e0e-8492-47e6-af17-b2da127a1d6a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:51.710837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3f117e0e-8492-47e6-af17-b2da127a1d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:51.864946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f117e0e-8492-47e6-af17-b2da127a1d6a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:51.878471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:51.983735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dab0c1-f9b4-42f0-b621-22adec47f654 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:51.984431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dab0c1-f9b4-42f0-b621-22adec47f654 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:51.984707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dab0c1-f9b4-42f0-b621-22adec47f654 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:51.984976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dab0c1-f9b4-42f0-b621-22adec47f654 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:51.986624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25dab0c1-f9b4-42f0-b621-22adec47f654 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:52.090422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dab0c1-f9b4-42f0-b621-22adec47f654 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:52.092036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dab0c1-f9b4-42f0-b621-22adec47f654 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:52.110701 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 718/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:53.753498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7438-ecf6-4663-b4df-dddbbde03206 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:53.754103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7438-ecf6-4663-b4df-dddbbde03206 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:53.754103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7438-ecf6-4663-b4df-dddbbde03206 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:53.755391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:53.760730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4297-ea50-4c7a-87dc-37c4cf78af32 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:53.761192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4297-ea50-4c7a-87dc-37c4cf78af32 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:53.761299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4297-ea50-4c7a-87dc-37c4cf78af32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:53.788669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:53.847000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-11aa4297-ea50-4c7a-87dc-37c4cf78af32 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:53.848210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-11aa4297-ea50-4c7a-87dc-37c4cf78af32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:53.949715 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:54.123496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d7438-ecf6-4663-b4df-dddbbde03206 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:54.125884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d72d7438-ecf6-4663-b4df-dddbbde03206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:54.249544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:54.293654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d7438-ecf6-4663-b4df-dddbbde03206 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:54.427581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d40fc-f269-4888-b134-e954bc08fd92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:54.428178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d40fc-f269-4888-b134-e954bc08fd92 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:54.428449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d40fc-f269-4888-b134-e954bc08fd92 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:54.428714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d40fc-f269-4888-b134-e954bc08fd92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:54.430341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca1d40fc-f269-4888-b134-e954bc08fd92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:54.510326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:54.525383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d40fc-f269-4888-b134-e954bc08fd92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:54.527008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d40fc-f269-4888-b134-e954bc08fd92 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:54.545366 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 706/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:56.220066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d14daa-0094-4bd1-a930-e28833dc80a4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:56.220943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d14daa-0094-4bd1-a930-e28833dc80a4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:56.220943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d14daa-0094-4bd1-a930-e28833dc80a4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:56.223316 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:41:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:56.229390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38db90-62e0-4130-8119-13a1ac765263 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:56.230133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38db90-62e0-4130-8119-13a1ac765263 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:56.230287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38db90-62e0-4130-8119-13a1ac765263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:56.311740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8b38db90-62e0-4130-8119-13a1ac765263 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:56.312950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8b38db90-62e0-4130-8119-13a1ac765263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:56.415656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:56.578316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d14daa-0094-4bd1-a930-e28833dc80a4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:56.580719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f0d14daa-0094-4bd1-a930-e28833dc80a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:56.674859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:56.738218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d14daa-0094-4bd1-a930-e28833dc80a4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:56.858984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d08c2d-c7e4-46e7-9b3a-19295c59a95e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:56.859515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d08c2d-c7e4-46e7-9b3a-19295c59a95e service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:56.859798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d08c2d-c7e4-46e7-9b3a-19295c59a95e service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:56.860066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d08c2d-c7e4-46e7-9b3a-19295c59a95e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:56.861752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d2d08c2d-c7e4-46e7-9b3a-19295c59a95e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:56.918142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:56.960602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d08c2d-c7e4-46e7-9b3a-19295c59a95e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:56.962271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d08c2d-c7e4-46e7-9b3a-19295c59a95e service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:56.980997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 721/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:41:58.581836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472b0c-8c13-476b-b89f-43e2349717e2 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:58.582405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472b0c-8c13-476b-b89f-43e2349717e2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:58.582405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472b0c-8c13-476b-b89f-43e2349717e2 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:58.583716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:41:58.589738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7a699-8bad-45a1-904f-d048b7c1738e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:58.590493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7a699-8bad-45a1-904f-d048b7c1738e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:58.590493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7a699-8bad-45a1-904f-d048b7c1738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:58.679497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a3d7a699-8bad-45a1-904f-d048b7c1738e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:41:58.680617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a3d7a699-8bad-45a1-904f-d048b7c1738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:58.781044 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:41:58.945664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0472b0c-8c13-476b-b89f-43e2349717e2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:41:58.947920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0472b0c-8c13-476b-b89f-43e2349717e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:59.010591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:59.114996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0472b0c-8c13-476b-b89f-43e2349717e2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:41:59.248462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05951b33-9d04-429d-b581-819f584b3f0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:41:59.248971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05951b33-9d04-429d-b581-819f584b3f0d service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:41:59.249239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05951b33-9d04-429d-b581-819f584b3f0d service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:41:59.249495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05951b33-9d04-429d-b581-819f584b3f0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:41:59.251129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-05951b33-9d04-429d-b581-819f584b3f0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:41:59.258751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:59.354474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05951b33-9d04-429d-b581-819f584b3f0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:41:59.356076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05951b33-9d04-429d-b581-819f584b3f0d service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:41:59.374388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 709/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:00.979182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97a86d-6223-4df0-abd5-c2548f03f5c3 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:00.979711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97a86d-6223-4df0-abd5-c2548f03f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:00.979711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97a86d-6223-4df0-abd5-c2548f03f5c3 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:00.981052 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:00.986248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31f90-1886-4f50-8ffc-47830cce18cc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:00.986687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31f90-1886-4f50-8ffc-47830cce18cc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:00.986834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f31f90-1886-4f50-8ffc-47830cce18cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:01.070881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-97f31f90-1886-4f50-8ffc-47830cce18cc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:01.072351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97f31f90-1886-4f50-8ffc-47830cce18cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:01.147274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:01.177129 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:01.341034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97a86d-6223-4df0-abd5-c2548f03f5c3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:01.343445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cd97a86d-6223-4df0-abd5-c2548f03f5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:01.364781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:01.500462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97a86d-6223-4df0-abd5-c2548f03f5c3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:01.601294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:01.627772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a3a61-e111-4305-b0c4-daa152bc0a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:01.628395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3a61-e111-4305-b0c4-daa152bc0a49 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:01.628672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3a61-e111-4305-b0c4-daa152bc0a49 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:01.628942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3a61-e111-4305-b0c4-daa152bc0a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:01.630600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-706a3a61-e111-4305-b0c4-daa152bc0a49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:01.726270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a3a61-e111-4305-b0c4-daa152bc0a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:01.727883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a3a61-e111-4305-b0c4-daa152bc0a49 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:01.747332 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 724/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:03.348048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82e50-8789-4cc6-a270-c4cdaf4bb7ab None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:03.348689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82e50-8789-4cc6-a270-c4cdaf4bb7ab None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:03.348689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82e50-8789-4cc6-a270-c4cdaf4bb7ab None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:03.350741 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:42:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:03.356596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e39d2-4166-42d8-a81f-ca650d5b4b78 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:03.357009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e39d2-4166-42d8-a81f-ca650d5b4b78 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:03.357134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e39d2-4166-42d8-a81f-ca650d5b4b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:03.436669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-955e39d2-4166-42d8-a81f-ca650d5b4b78 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:03.437915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-955e39d2-4166-42d8-a81f-ca650d5b4b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:03.487722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:03.545689 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:03.706456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82e50-8789-4cc6-a270-c4cdaf4bb7ab None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:03.708822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-30a82e50-8789-4cc6-a270-c4cdaf4bb7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:03.858644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82e50-8789-4cc6-a270-c4cdaf4bb7ab None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:03.915399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:03.975164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc365e-1326-47d2-a6ba-675034c2c4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:03.975697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc365e-1326-47d2-a6ba-675034c2c4d5 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:03.975969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc365e-1326-47d2-a6ba-675034c2c4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:03.976232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc365e-1326-47d2-a6ba-675034c2c4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:03.977869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3bc365e-1326-47d2-a6ba-675034c2c4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:04.070533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc365e-1326-47d2-a6ba-675034c2c4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:04.072144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc365e-1326-47d2-a6ba-675034c2c4d5 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:04.090386 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 712/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:05.703059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fe9b9-9f8e-4e06-9004-b9e1cbbd509d None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:05.703585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fe9b9-9f8e-4e06-9004-b9e1cbbd509d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:05.703585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fe9b9-9f8e-4e06-9004-b9e1cbbd509d None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:05.704976 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:05.710286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf08ffe-efd0-4778-bf33-001c97425d67 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:05.710735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf08ffe-efd0-4778-bf33-001c97425d67 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:05.710840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf08ffe-efd0-4778-bf33-001c97425d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:05.792911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fdf08ffe-efd0-4778-bf33-001c97425d67 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:05.794158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fdf08ffe-efd0-4778-bf33-001c97425d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:05.796185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:05.896238 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:06.057931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fe9b9-9f8e-4e06-9004-b9e1cbbd509d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:06.060091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-570fe9b9-9f8e-4e06-9004-b9e1cbbd509d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:06.221344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fe9b9-9f8e-4e06-9004-b9e1cbbd509d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:06.248630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:06.350619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec935b-bbd6-419d-b4b9-91d1ed7870f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:06.351163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec935b-bbd6-419d-b4b9-91d1ed7870f2 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:06.351436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec935b-bbd6-419d-b4b9-91d1ed7870f2 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:06.351771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec935b-bbd6-419d-b4b9-91d1ed7870f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:06.353380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e5ec935b-bbd6-419d-b4b9-91d1ed7870f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:06.443992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec935b-bbd6-419d-b4b9-91d1ed7870f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:06.445750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec935b-bbd6-419d-b4b9-91d1ed7870f2 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:06.463635 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 727/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:08.790377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3014c-646f-4e9e-88b0-e90c98007785 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:08.791126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3014c-646f-4e9e-88b0-e90c98007785 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:08.791126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3014c-646f-4e9e-88b0-e90c98007785 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:08.792795 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:08.799667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7fbb-623e-4b85-8ec1-bcd0e0d4c161 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:08.800139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7fbb-623e-4b85-8ec1-bcd0e0d4c161 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:08.800331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7fbb-623e-4b85-8ec1-bcd0e0d4c161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:08.848472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:08.874269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-99ea7fbb-623e-4b85-8ec1-bcd0e0d4c161 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:08.875469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-99ea7fbb-623e-4b85-8ec1-bcd0e0d4c161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:08.993257 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:09.345759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3014c-646f-4e9e-88b0-e90c98007785 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:09.348202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98d3014c-646f-4e9e-88b0-e90c98007785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:09.459728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3014c-646f-4e9e-88b0-e90c98007785 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:09.493996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:09.538160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ea2fd-5772-452d-a25b-19a4bee6e0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:09.538772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ea2fd-5772-452d-a25b-19a4bee6e0fb service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:09.539073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ea2fd-5772-452d-a25b-19a4bee6e0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:09.539360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ea2fd-5772-452d-a25b-19a4bee6e0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:09.540956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e1ea2fd-5772-452d-a25b-19a4bee6e0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:09.653355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ea2fd-5772-452d-a25b-19a4bee6e0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:09.654827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ea2fd-5772-452d-a25b-19a4bee6e0fb service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:09.674412 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 715/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:12.234227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4b460-4481-427c-8cc8-c997c469bed8 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:12.235109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4b460-4481-427c-8cc8-c997c469bed8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:12.235109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4b460-4481-427c-8cc8-c997c469bed8 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:12.237081 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:12.243258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0fa60-1c9d-4bd2-897d-4afcf313618d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:12.243681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0fa60-1c9d-4bd2-897d-4afcf313618d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:12.243808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0fa60-1c9d-4bd2-897d-4afcf313618d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:12.313635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b9b0fa60-1c9d-4bd2-897d-4afcf313618d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:12.314851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b9b0fa60-1c9d-4bd2-897d-4afcf313618d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:12.329448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:12.429703 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:12.770749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4b460-4481-427c-8cc8-c997c469bed8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:12.773208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-16d4b460-4481-427c-8cc8-c997c469bed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:12.885632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4b460-4481-427c-8cc8-c997c469bed8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:12.899333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:12.971064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7954e9d-35c4-4aab-a178-8f6282778844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:12.971620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7954e9d-35c4-4aab-a178-8f6282778844 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:12.971897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7954e9d-35c4-4aab-a178-8f6282778844 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:12.972172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7954e9d-35c4-4aab-a178-8f6282778844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:12.973814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b7954e9d-35c4-4aab-a178-8f6282778844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:13.087871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7954e9d-35c4-4aab-a178-8f6282778844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:13.089332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7954e9d-35c4-4aab-a178-8f6282778844 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:13.107901 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 730/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:15.791746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f65b-b0bc-49df-a0d7-bee14cf7e578 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:15.792650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f65b-b0bc-49df-a0d7-bee14cf7e578 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:15.792650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831f65b-b0bc-49df-a0d7-bee14cf7e578 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:15.794860 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:15.803800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1790c-ed6f-444e-9ce1-8b300e0ae0f1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:15.804500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1790c-ed6f-444e-9ce1-8b300e0ae0f1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:15.804708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1790c-ed6f-444e-9ce1-8b300e0ae0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:15.856095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:15.875737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-32b1790c-ed6f-444e-9ce1-8b300e0ae0f1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:15.876972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-32b1790c-ed6f-444e-9ce1-8b300e0ae0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:15.979059 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:16.321491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831f65b-b0bc-49df-a0d7-bee14cf7e578 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:16.323241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7831f65b-b0bc-49df-a0d7-bee14cf7e578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:16.437544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831f65b-b0bc-49df-a0d7-bee14cf7e578 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:16.482294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:16.517411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe3484-d6a4-460d-9de0-b9ed6190707e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:16.518048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3484-d6a4-460d-9de0-b9ed6190707e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:16.518296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3484-d6a4-460d-9de0-b9ed6190707e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:16.518585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3484-d6a4-460d-9de0-b9ed6190707e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:16.520166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4cbe3484-d6a4-460d-9de0-b9ed6190707e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:16.630865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe3484-d6a4-460d-9de0-b9ed6190707e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:16.632289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe3484-d6a4-460d-9de0-b9ed6190707e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:16.652069 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 718/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:42:19.187584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aa8f9-f6b4-4533-a411-9d098800e002 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:19.188195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aa8f9-f6b4-4533-a411-9d098800e002 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:19.188195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647aa8f9-f6b4-4533-a411-9d098800e002 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:19.189520 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:42:19.195084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f029-28ee-4a39-9db6-5cdf79d39248 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:19.195524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f029-28ee-4a39-9db6-5cdf79d39248 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:19.195637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f029-28ee-4a39-9db6-5cdf79d39248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:19.262792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3af4f029-28ee-4a39-9db6-5cdf79d39248 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:19.264001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3af4f029-28ee-4a39-9db6-5cdf79d39248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:19.285819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:19.366624 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:19.708954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aa8f9-f6b4-4533-a411-9d098800e002 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:19.710596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-647aa8f9-f6b4-4533-a411-9d098800e002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:19.824389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647aa8f9-f6b4-4533-a411-9d098800e002 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:19.846179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:19.902287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2f7c-f4bd-467c-9938-e941a929a25f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:19.902895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2f7c-f4bd-467c-9938-e941a929a25f service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:19.903168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2f7c-f4bd-467c-9938-e941a929a25f service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:19.903460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2f7c-f4bd-467c-9938-e941a929a25f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:19.905374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6b8f2f7c-f4bd-467c-9938-e941a929a25f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:20.020366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f2f7c-f4bd-467c-9938-e941a929a25f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:20.021764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f2f7c-f4bd-467c-9938-e941a929a25f service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:20.040581 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 733/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:22.621789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7155f6-7353-4e1a-88a3-2ae9c1b210af None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:22.622740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7155f6-7353-4e1a-88a3-2ae9c1b210af None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:22.622740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7155f6-7353-4e1a-88a3-2ae9c1b210af None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:22.624876 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:22.633831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac878-4524-4d37-8161-e22e33b53ee1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:22.634300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac878-4524-4d37-8161-e22e33b53ee1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:22.634410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac878-4524-4d37-8161-e22e33b53ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:22.705944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:22.708860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b46ac878-4524-4d37-8161-e22e33b53ee1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:22.710717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b46ac878-4524-4d37-8161-e22e33b53ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:22.808545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:23.150297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7155f6-7353-4e1a-88a3-2ae9c1b210af None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:23.151844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cf7155f6-7353-4e1a-88a3-2ae9c1b210af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:23.263737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7155f6-7353-4e1a-88a3-2ae9c1b210af None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:23.315365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:23.340964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46948bf8-35dd-4cfe-8525-04023ed4285c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:23.341634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46948bf8-35dd-4cfe-8525-04023ed4285c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:23.341907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46948bf8-35dd-4cfe-8525-04023ed4285c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:23.342172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46948bf8-35dd-4cfe-8525-04023ed4285c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:23.343768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-46948bf8-35dd-4cfe-8525-04023ed4285c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:23.454398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46948bf8-35dd-4cfe-8525-04023ed4285c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:23.455749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46948bf8-35dd-4cfe-8525-04023ed4285c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:23.476711 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 721/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:42:26.007108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfeec-b8b7-41fa-83c4-ef285aa7e052 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:26.008023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfeec-b8b7-41fa-83c4-ef285aa7e052 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:26.008023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbfeec-b8b7-41fa-83c4-ef285aa7e052 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:26.009945 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:26.015466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c010-037b-413d-b9e4-685c886c1c11 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:26.015921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c010-037b-413d-b9e4-685c886c1c11 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:26.016038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483c010-037b-413d-b9e4-685c886c1c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:26.086614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f483c010-037b-413d-b9e4-685c886c1c11 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:26.087848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f483c010-037b-413d-b9e4-685c886c1c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:26.121351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:26.198090 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:26.533818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbfeec-b8b7-41fa-83c4-ef285aa7e052 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:26.535419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3ecbfeec-b8b7-41fa-83c4-ef285aa7e052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:26.651765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbfeec-b8b7-41fa-83c4-ef285aa7e052 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:26.681933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:26.730936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaa0b7-77e9-499b-9e3e-406c344b7d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:26.731549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaa0b7-77e9-499b-9e3e-406c344b7d96 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:26.731824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaa0b7-77e9-499b-9e3e-406c344b7d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:26.732092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaa0b7-77e9-499b-9e3e-406c344b7d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:26.733714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34aaa0b7-77e9-499b-9e3e-406c344b7d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:26.851933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaa0b7-77e9-499b-9e3e-406c344b7d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:26.853500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaa0b7-77e9-499b-9e3e-406c344b7d96 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:26.875026 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 736/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:29.310413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687e516-62be-4175-ac1d-38e9017af77c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:29.311020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687e516-62be-4175-ac1d-38e9017af77c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:29.311020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687e516-62be-4175-ac1d-38e9017af77c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:29.312291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:29.318207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170923-dd47-407d-820f-86385b232a2d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:29.318643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170923-dd47-407d-820f-86385b232a2d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:29.318764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170923-dd47-407d-820f-86385b232a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:29.390551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4e170923-dd47-407d-820f-86385b232a2d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:29.391776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4e170923-dd47-407d-820f-86385b232a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:29.401041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:29.494283 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:29.835909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687e516-62be-4175-ac1d-38e9017af77c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:29.838505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1687e516-62be-4175-ac1d-38e9017af77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:29.969605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687e516-62be-4175-ac1d-38e9017af77c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:30.045861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:30.065090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc267c08-b8f0-47bd-9d16-c6e398467e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:30.065741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc267c08-b8f0-47bd-9d16-c6e398467e7a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:30.066016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc267c08-b8f0-47bd-9d16-c6e398467e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:30.066283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc267c08-b8f0-47bd-9d16-c6e398467e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:30.067902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc267c08-b8f0-47bd-9d16-c6e398467e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:30.188619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc267c08-b8f0-47bd-9d16-c6e398467e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:30.190128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc267c08-b8f0-47bd-9d16-c6e398467e7a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:30.209249 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 724/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:32.658305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1718b5-8d3f-40be-a53f-bda89446a9cb None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:32.659224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1718b5-8d3f-40be-a53f-bda89446a9cb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:32.659224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1718b5-8d3f-40be-a53f-bda89446a9cb None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:32.661321 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:32.668658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9e455-4485-4f88-85a2-4a1c0c46f260 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:32.669118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9e455-4485-4f88-85a2-4a1c0c46f260 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:32.669118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9e455-4485-4f88-85a2-4a1c0c46f260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:32.738424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-78c9e455-4485-4f88-85a2-4a1c0c46f260 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:32.739649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78c9e455-4485-4f88-85a2-4a1c0c46f260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:32.782465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:32.841999 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:33.185201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1718b5-8d3f-40be-a53f-bda89446a9cb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:33.186814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f1718b5-8d3f-40be-a53f-bda89446a9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:33.301799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1718b5-8d3f-40be-a53f-bda89446a9cb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:33.342785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:33.380551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45b3a1-2f7e-418b-b599-b07362a86112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:33.381183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45b3a1-2f7e-418b-b599-b07362a86112 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:33.381454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45b3a1-2f7e-418b-b599-b07362a86112 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:33.381745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45b3a1-2f7e-418b-b599-b07362a86112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:33.383333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9e45b3a1-2f7e-418b-b599-b07362a86112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:33.497974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45b3a1-2f7e-418b-b599-b07362a86112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:33.499387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45b3a1-2f7e-418b-b599-b07362a86112 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:33.521932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 739/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:35.950251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c9df6-5155-4f6b-a764-db16b9e21e1b None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:35.951048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c9df6-5155-4f6b-a764-db16b9e21e1b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:35.951048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c9df6-5155-4f6b-a764-db16b9e21e1b None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:35.952838 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:35.958627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ef3f-f50b-459e-ac5f-f295279d0ac9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:35.959117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ef3f-f50b-459e-ac5f-f295279d0ac9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:35.959117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ef3f-f50b-459e-ac5f-f295279d0ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:36.037391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-eed3ef3f-f50b-459e-ac5f-f295279d0ac9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:36.038653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eed3ef3f-f50b-459e-ac5f-f295279d0ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:36.057771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:36.141970 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:42:36.483615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c9df6-5155-4f6b-a764-db16b9e21e1b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:36.486160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7b0c9df6-5155-4f6b-a764-db16b9e21e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:36.603609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c9df6-5155-4f6b-a764-db16b9e21e1b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:36.674350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:36.681495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0e47b-96f4-42b6-bb57-8446e7f7cb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:36.682155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0e47b-96f4-42b6-bb57-8446e7f7cb8c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:36.682428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0e47b-96f4-42b6-bb57-8446e7f7cb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:36.682696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0e47b-96f4-42b6-bb57-8446e7f7cb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:36.684276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-19a0e47b-96f4-42b6-bb57-8446e7f7cb8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:36.802019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0e47b-96f4-42b6-bb57-8446e7f7cb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:36.803499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0e47b-96f4-42b6-bb57-8446e7f7cb8c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:36.822984 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 727/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:39.358919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16d54f-7181-4835-8eb7-f204a32e2e77 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:39.359510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16d54f-7181-4835-8eb7-f204a32e2e77 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:39.359510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16d54f-7181-4835-8eb7-f204a32e2e77 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:39.361441 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:42:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:42:39.366976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9cda-2585-48b8-8067-abe663ae8b6f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:39.367430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9cda-2585-48b8-8067-abe663ae8b6f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:39.367537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9cda-2585-48b8-8067-abe663ae8b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:39.437790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4eed9cda-2585-48b8-8067-abe663ae8b6f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:42:39.438930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4eed9cda-2585-48b8-8067-abe663ae8b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:39.493407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:39.542665 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:42:39.873991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16d54f-7181-4835-8eb7-f204a32e2e77 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:42:39.875593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7c16d54f-7181-4835-8eb7-f204a32e2e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:39.991370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16d54f-7181-4835-8eb7-f204a32e2e77 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:42:40.043443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:40.070451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ebd81-0774-4d52-a375-446e0c6cdfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:42:40.071144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ebd81-0774-4d52-a375-446e0c6cdfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:42:40.071522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ebd81-0774-4d52-a375-446e0c6cdfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:42:40.071802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ebd81-0774-4d52-a375-446e0c6cdfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:42:40.073425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b34ebd81-0774-4d52-a375-446e0c6cdfe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:42:40.317612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ebd81-0774-4d52-a375-446e0c6cdfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:42:40.318995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ebd81-0774-4d52-a375-446e0c6cdfe8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:42:40.337292 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 742/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:42:49.541167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=22 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:50.335038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=66 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:04.708755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa059a-ee46-4db3-8259-a578593d38a9 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:04.709307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa059a-ee46-4db3-8259-a578593d38a9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:04.709307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa059a-ee46-4db3-8259-a578593d38a9 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:04.711480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 729/1471] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:43:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:43:04.718046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c2632-0d28-4949-a71f-ae8b329520b8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:04.718507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c2632-0d28-4949-a71f-ae8b329520b8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:04.718612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c2632-0d28-4949-a71f-ae8b329520b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:04.797643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-851c2632-0d28-4949-a71f-ae8b329520b8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:04.798852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-851c2632-0d28-4949-a71f-ae8b329520b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:04.897378 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 743/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:05.246122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d22f53-d791-45c9-939e-632f04000af9 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:05.246400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d22f53-d791-45c9-939e-632f04000af9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:05.246665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d22f53-d791-45c9-939e-632f04000af9 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:05.247990 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 730/1473] 10.222.0.26 () {54 vars in 962 bytes} [Sat Sep 17 20:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:43:05.253258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caba31-ce56-4c0a-8560-0170c6252457 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:05.253637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caba31-ce56-4c0a-8560-0170c6252457 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:05.253915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caba31-ce56-4c0a-8560-0170c6252457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:05.333650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-03caba31-ce56-4c0a-8560-0170c6252457 None None] MFA Rules not processed for user `94ce9c64a0bb478485c47ebedd5a29f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:05.334888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-03caba31-ce56-4c0a-8560-0170c6252457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:05.357672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:05.422348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 744/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Sep 17 20:43:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:05.430369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d22f53-d791-45c9-939e-632f04000af9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:43:05.431987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0d22f53-d791-45c9-939e-632f04000af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:05.546016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d22f53-d791-45c9-939e-632f04000af9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:43:05.620012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27d343-a576-41d4-96a2-64a35eaea6be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:43:05.620617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27d343-a576-41d4-96a2-64a35eaea6be service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:05.620886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27d343-a576-41d4-96a2-64a35eaea6be service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:05.621142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27d343-a576-41d4-96a2-64a35eaea6be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:05.622731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3c27d343-a576-41d4-96a2-64a35eaea6be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:05.679643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:05.739909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27d343-a576-41d4-96a2-64a35eaea6be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:43:05.741350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27d343-a576-41d4-96a2-64a35eaea6be service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:43:05.761109 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 731/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:43:08.250894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38be6e-6c03-4a8d-9772-34d3ee7b54bb None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:08.251485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38be6e-6c03-4a8d-9772-34d3ee7b54bb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:08.251485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38be6e-6c03-4a8d-9772-34d3ee7b54bb None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:08.252679 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 745/1476] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:43:08.259338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9422c41-486e-4b25-8839-24f27825a6c0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:08.260063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9422c41-486e-4b25-8839-24f27825a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:08.260303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9422c41-486e-4b25-8839-24f27825a6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:08.338118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f9422c41-486e-4b25-8839-24f27825a6c0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:08.339896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f9422c41-486e-4b25-8839-24f27825a6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:08.441881 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 732/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:08.766962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38be6e-6c03-4a8d-9772-34d3ee7b54bb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:43:08.768498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a38be6e-6c03-4a8d-9772-34d3ee7b54bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:08.902503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38be6e-6c03-4a8d-9772-34d3ee7b54bb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:43:08.952356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:09.000169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165847b3-14ca-47b2-a001-56f0275ecdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:43:09.000781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165847b3-14ca-47b2-a001-56f0275ecdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:09.001052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165847b3-14ca-47b2-a001-56f0275ecdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:09.001316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165847b3-14ca-47b2-a001-56f0275ecdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:09.001989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165847b3-14ca-47b2-a001-56f0275ecdf8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:43:09.003799 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-165847b3-14ca-47b2-a001-56f0275ecdf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:43:09.003799 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:43:09.003799 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:43:09.004947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165847b3-14ca-47b2-a001-56f0275ecdf8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:43:09.023778 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 746/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 17 20:43:08 2022] GET /identity/v3/projects => generated 2553 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:43:11.410199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5c376-2593-446d-8717-f297f7794b58 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:11.411009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5c376-2593-446d-8717-f297f7794b58 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:11.411009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5c376-2593-446d-8717-f297f7794b58 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:11.413039 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 733/1479] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:43:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:43:11.421502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532d2ee-bb00-4e19-a13f-6f0989bf494a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:11.421952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532d2ee-bb00-4e19-a13f-6f0989bf494a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:11.422057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532d2ee-bb00-4e19-a13f-6f0989bf494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:11.493240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d532d2ee-bb00-4e19-a13f-6f0989bf494a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:11.494531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d532d2ee-bb00-4e19-a13f-6f0989bf494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:11.595318 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 747/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:11.948519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5c376-2593-446d-8717-f297f7794b58 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:43:11.950373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e9b5c376-2593-446d-8717-f297f7794b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:11.964585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:12.107424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5c376-2593-446d-8717-f297f7794b58 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:43:12.191729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:12.226377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c49cfe-ba05-415e-8442-8dba28d62f13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:43:12.227246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c49cfe-ba05-415e-8442-8dba28d62f13 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:12.227484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c49cfe-ba05-415e-8442-8dba28d62f13 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:12.227780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c49cfe-ba05-415e-8442-8dba28d62f13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:12.229494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-02c49cfe-ba05-415e-8442-8dba28d62f13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:12.321713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c49cfe-ba05-415e-8442-8dba28d62f13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:43:12.323275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c49cfe-ba05-415e-8442-8dba28d62f13 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:43:12.342818 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 734/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:43:14.902320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa058cef-bd2a-4e27-a2d1-1cf873f9d385 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:14.902922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa058cef-bd2a-4e27-a2d1-1cf873f9d385 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:14.902922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa058cef-bd2a-4e27-a2d1-1cf873f9d385 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:14.904261 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 748/1482] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:43:14.910978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3832-510a-4364-bed0-b9fabbd8515e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:14.911686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3832-510a-4364-bed0-b9fabbd8515e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:14.911881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3832-510a-4364-bed0-b9fabbd8515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:14.986953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fd7a3832-510a-4364-bed0-b9fabbd8515e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:14.988077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd7a3832-510a-4364-bed0-b9fabbd8515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:15.037845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:15.095734 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 735/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:15.439603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa058cef-bd2a-4e27-a2d1-1cf873f9d385 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:43:15.442158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa058cef-bd2a-4e27-a2d1-1cf873f9d385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:15.468987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:15.558455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa058cef-bd2a-4e27-a2d1-1cf873f9d385 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:43:15.635949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206580d-b517-4324-a732-fce4bb42d666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:43:15.636556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206580d-b517-4324-a732-fce4bb42d666 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:15.636846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206580d-b517-4324-a732-fce4bb42d666 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:15.637113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206580d-b517-4324-a732-fce4bb42d666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:15.638771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b206580d-b517-4324-a732-fce4bb42d666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:15.720123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:15.764128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206580d-b517-4324-a732-fce4bb42d666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:43:15.766262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206580d-b517-4324-a732-fce4bb42d666 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:43:15.789206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 749/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:43:18.246704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac557-f9ee-4793-b642-30699f0185b2 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:18.247309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac557-f9ee-4793-b642-30699f0185b2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:18.247309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac557-f9ee-4793-b642-30699f0185b2 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:18.248617 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 736/1485] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:43:18.254154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319a681-a1c4-420c-a32e-7c6a3e65cc2f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:18.254622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319a681-a1c4-420c-a32e-7c6a3e65cc2f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:18.254731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319a681-a1c4-420c-a32e-7c6a3e65cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:18.325211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0319a681-a1c4-420c-a32e-7c6a3e65cc2f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:18.326445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0319a681-a1c4-420c-a32e-7c6a3e65cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:18.447096 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 750/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:18.788811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac557-f9ee-4793-b642-30699f0185b2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:43:18.791346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-745ac557-f9ee-4793-b642-30699f0185b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:18.907327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac557-f9ee-4793-b642-30699f0185b2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:43:18.981755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:18.983438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bdef8-b4c8-4c10-8aa2-00f0b0931150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:43:18.984082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bdef8-b4c8-4c10-8aa2-00f0b0931150 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:18.984396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bdef8-b4c8-4c10-8aa2-00f0b0931150 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:18.984697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bdef8-b4c8-4c10-8aa2-00f0b0931150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:18.986794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-674bdef8-b4c8-4c10-8aa2-00f0b0931150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:19.115368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bdef8-b4c8-4c10-8aa2-00f0b0931150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:43:19.116914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bdef8-b4c8-4c10-8aa2-00f0b0931150 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:43:19.136326 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 737/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:43:21.607378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6eaa-eb5e-46ac-94e0-545e1371a337 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:21.607992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6eaa-eb5e-46ac-94e0-545e1371a337 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:21.607992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6eaa-eb5e-46ac-94e0-545e1371a337 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:21.609275 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 751/1488] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:43:21.615754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29cf9-9747-4267-9b19-ef25360d5aa9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:21.616491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29cf9-9747-4267-9b19-ef25360d5aa9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:21.616676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29cf9-9747-4267-9b19-ef25360d5aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:21.685677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1cb29cf9-9747-4267-9b19-ef25360d5aa9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:21.686914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1cb29cf9-9747-4267-9b19-ef25360d5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:21.740337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:21.789129 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 738/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:22.133523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f6eaa-eb5e-46ac-94e0-545e1371a337 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:43:22.136153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2e5f6eaa-eb5e-46ac-94e0-545e1371a337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:22.181105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:22.255423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f6eaa-eb5e-46ac-94e0-545e1371a337 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:43:22.336229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2609cb-76e3-4543-88c4-4a1e63ae8ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:43:22.336860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2609cb-76e3-4543-88c4-4a1e63ae8ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:22.337128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2609cb-76e3-4543-88c4-4a1e63ae8ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:22.337391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2609cb-76e3-4543-88c4-4a1e63ae8ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:22.339071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9f2609cb-76e3-4543-88c4-4a1e63ae8ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:22.441587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:22.472086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2609cb-76e3-4543-88c4-4a1e63ae8ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:43:22.473593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2609cb-76e3-4543-88c4-4a1e63ae8ac5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:43:22.492027 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 752/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:43:25.134677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217dd2-6b16-4540-9035-be91c8450eb7 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:25.135393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217dd2-6b16-4540-9035-be91c8450eb7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:25.135393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14217dd2-6b16-4540-9035-be91c8450eb7 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:25.136643 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 739/1491] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 17 20:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:43:25.142294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5b39e-29a7-4c02-a956-ebe8d46bb87c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:25.142768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5b39e-29a7-4c02-a956-ebe8d46bb87c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:25.142882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5b39e-29a7-4c02-a956-ebe8d46bb87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:25.213233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2bc5b39e-29a7-4c02-a956-ebe8d46bb87c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:43:25.214495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2bc5b39e-29a7-4c02-a956-ebe8d46bb87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:25.312860 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 753/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 17 20:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:43:25.666729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14217dd2-6b16-4540-9035-be91c8450eb7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:43:25.669117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-14217dd2-6b16-4540-9035-be91c8450eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:25.779570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14217dd2-6b16-4540-9035-be91c8450eb7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:43:25.868054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6a94f-87f7-4f8c-82ca-0a6b7ea5f4b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:43:25.868667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a94f-87f7-4f8c-82ca-0a6b7ea5f4b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:43:25.868947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a94f-87f7-4f8c-82ca-0a6b7ea5f4b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:43:25.873498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a94f-87f7-4f8c-82ca-0a6b7ea5f4b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:43:25.873498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8e6a94f-87f7-4f8c-82ca-0a6b7ea5f4b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:43:25.880356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:25.992770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6a94f-87f7-4f8c-82ca-0a6b7ea5f4b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:43:25.994156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6a94f-87f7-4f8c-82ca-0a6b7ea5f4b8 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:43:26.013630 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 740/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:43:35.311429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=82 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:36.011172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=50 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:47.690623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e076c-a32d-4ac2-8dfc-a552cc7eb228 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:47.691282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e076c-a32d-4ac2-8dfc-a552cc7eb228 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:47.691282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e076c-a32d-4ac2-8dfc-a552cc7eb228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:47.773937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-da4e076c-a32d-4ac2-8dfc-a552cc7eb228 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:44:47.775134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-da4e076c-a32d-4ac2-8dfc-a552cc7eb228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:44:47.905876 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 754/1494] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:44:47.917484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4229c6-8c9d-4a3a-94af-3cde6017cda2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:47.918038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4229c6-8c9d-4a3a-94af-3cde6017cda2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:47.918038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4229c6-8c9d-4a3a-94af-3cde6017cda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:47.938772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1f4229c6-8c9d-4a3a-94af-3cde6017cda2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:44:47.939964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f4229c6-8c9d-4a3a-94af-3cde6017cda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:44:47.949642 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 741/1495] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:44:47.954952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e076c-a32d-4ac2-8dfc-a552cc7eb228 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:47.969593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e076c-a32d-4ac2-8dfc-a552cc7eb228 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:47.974972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:47.975566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:47.975838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:47.976104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:47.976749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:44:47.977048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:47.978117 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:47.978117 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:44:47.978117 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:44:47.979003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:47.991107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 755/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:44:47.999554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4229c6-8c9d-4a3a-94af-3cde6017cda2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:48.008980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4229c6-8c9d-4a3a-94af-3cde6017cda2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:48.015075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:48.015194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.015372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.015637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.016315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:48.018392 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:48.018392 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 20:44:48.018392 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 20:44:48.019721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:48.060251 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 742/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:44:47 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:44:48.064588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:48.074380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1943eb-3faa-4e54-8fa9-7c0ce00016c9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:48.127802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:48.138072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:48.138902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.139250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.139626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.140437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:48.141479 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:48.141479 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:44:48.141479 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:44:48.142694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:48.165708 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 756/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:44:48 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:44:48.170007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:48.180653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d5b0a-4f2b-45ea-bfc0-439a5dcfea11 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:48.186040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:48.186647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.186965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.187228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.187875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:48.188881 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:48.188881 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 20:44:48.188881 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 20:44:48.189814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:48.203213 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 743/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:44:48.208286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:48.219202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16a841-9bea-4b3e-8d69-3113eb6696ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:48.225026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c185c8-7450-4de0-8859-6ba1e8c51dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:48.225655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c185c8-7450-4de0-8859-6ba1e8c51dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.225931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c185c8-7450-4de0-8859-6ba1e8c51dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.226183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c185c8-7450-4de0-8859-6ba1e8c51dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.226824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c185c8-7450-4de0-8859-6ba1e8c51dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:48.227849 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-05c185c8-7450-4de0-8859-6ba1e8c51dc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:48.227849 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:44:48.227849 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:44:48.228753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c185c8-7450-4de0-8859-6ba1e8c51dc7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:48.242894 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 757/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:44:48.247064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:48.257104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567785a-0f85-4778-9b6b-f4fd8899a48d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:48.261943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8b2cf-3d6a-4343-b64e-4e9ada1bd150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:48.262585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8b2cf-3d6a-4343-b64e-4e9ada1bd150 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.262858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8b2cf-3d6a-4343-b64e-4e9ada1bd150 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.263125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8b2cf-3d6a-4343-b64e-4e9ada1bd150 admin admin] PATH_INFO: `/v3/projects/310240b744b641e2be4aefb07a5fa589/users/6d1a65ad125e4bddb12492937c786f2d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.278724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8b2cf-3d6a-4343-b64e-4e9ada1bd150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310240b744b641e2be4aefb07a5fa589, user_id=6d1a65ad125e4bddb12492937c786f2d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:48.280094 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-7ff8b2cf-3d6a-4343-b64e-4e9ada1bd150 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:48.280094 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 20:44:48.280094 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 20:44:48.282278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8b2cf-3d6a-4343-b64e-4e9ada1bd150 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:48.303168 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 744/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:44:48 2022] PUT /identity/v3/projects/310240b744b641e2be4aefb07a5fa589/users/6d1a65ad125e4bddb12492937c786f2d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:44:48.321503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb435187-4ee8-49ef-aff1-df058e22d85e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.321788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb435187-4ee8-49ef-aff1-df058e22d85e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.322180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb435187-4ee8-49ef-aff1-df058e22d85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.359804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bb435187-4ee8-49ef-aff1-df058e22d85e None None] MFA Rules not processed for user `6d1a65ad125e4bddb12492937c786f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:44:48.361032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bb435187-4ee8-49ef-aff1-df058e22d85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:44:48.440785 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 758/1502] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:44:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:44:48.454510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee6ea-cc8c-4ba6-ad82-bac312b7fc5d None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.454861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee6ea-cc8c-4ba6-ad82-bac312b7fc5d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.455412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee6ea-cc8c-4ba6-ad82-bac312b7fc5d None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.456828 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 745/1503] 10.222.0.26 () {54 vars in 964 bytes} [Sat Sep 17 20:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:44:48.462030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ffbe6-c9cd-4b84-b1f9-09d2e2f73a00 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.462314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ffbe6-c9cd-4b84-b1f9-09d2e2f73a00 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.462578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ffbe6-c9cd-4b84-b1f9-09d2e2f73a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.531020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ad1ffbe6-c9cd-4b84-b1f9-09d2e2f73a00 None None] MFA Rules not processed for user `d45ab41902864586811c81948540fdac`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:44:48.532211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ad1ffbe6-c9cd-4b84-b1f9-09d2e2f73a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:44:48.608030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 759/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Sat Sep 17 20:44:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:44:48.616219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbee6ea-cc8c-4ba6-ad82-bac312b7fc5d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:48.628796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbee6ea-cc8c-4ba6-ad82-bac312b7fc5d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:48.632973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:48.633626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.633891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.634161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.643425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:48.644872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:48.665516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 746/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:44:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:44:48.721263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd971192-3b9a-4693-8013-e18fc29a4d51 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.721745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd971192-3b9a-4693-8013-e18fc29a4d51 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.721921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd971192-3b9a-4693-8013-e18fc29a4d51 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.723198 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 760/1506] 10.222.0.26 () {52 vars in 830 bytes} [Sat Sep 17 20:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:44:48.727017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:48.793607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c0b14-9dd4-44f0-9063-3c8e7c92856c service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:48.798653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:48.799282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.799560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.799832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] PATH_INFO: `/v3/users/6d1a65ad125e4bddb12492937c786f2d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.801014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1a65ad125e4bddb12492937c786f2d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:48.802066 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:48.802066 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 20:44:48.802066 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 20:44:48.803289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:48.875566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1a65ad125e4bddb12492937c786f2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:48.876303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1a65ad125e4bddb12492937c786f2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:48.887512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1a65ad125e4bddb12492937c786f2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:48.888189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1a65ad125e4bddb12492937c786f2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:48.913313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1a65ad125e4bddb12492937c786f2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:48.925762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:48.936407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 747/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:44:48 2022] DELETE /identity/v3/users/6d1a65ad125e4bddb12492937c786f2d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:44:48.941233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:48.941542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:48.941817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:48.962251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:44:48.963346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:44:48.972924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:49.023879 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 761/1508] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:44:49.034176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:49.035706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:44:49.115256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06c8f5-5691-4d71-abcc-f642471dfe0a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:49.119246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7636f8-41a9-4696-b344-ba9f92b193ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:49.119869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7636f8-41a9-4696-b344-ba9f92b193ef service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:49.120142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7636f8-41a9-4696-b344-ba9f92b193ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:49.120386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7636f8-41a9-4696-b344-ba9f92b193ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:49.129863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7636f8-41a9-4696-b344-ba9f92b193ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:49.131299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7636f8-41a9-4696-b344-ba9f92b193ef service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:49.140788 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 748/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:44:49.508487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:44:49.509991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:44:49.521958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7a01c-e6d1-4994-bc37-ca6a73802c11 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:44:49.526760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:44:49.527391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:44:49.527719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:44:49.527986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] PATH_INFO: `/v3/projects/310240b744b641e2be4aefb07a5fa589` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:44:49.529098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310240b744b641e2be4aefb07a5fa589)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:44:49.530568 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:44:49.530568 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:44:49.530568 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:44:49.531930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:44:49.600005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310240b744b641e2be4aefb07a5fa589', 'request_id': 'req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:49.611409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310240b744b641e2be4aefb07a5fa589', 'request_id': 'req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:49.617853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310240b744b641e2be4aefb07a5fa589', 'request_id': 'req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:49.636326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] The token cache is being invalidate because project 310240b744b641e2be4aefb07a5fa589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:44:49.636612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:49.637339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a1fa3566-7ed1-41a1-9a18-2ca89e0feac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:44:49.638954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 762/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:44:49 2022] DELETE /identity/v3/projects/310240b744b641e2be4aefb07a5fa589 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:44:59.136710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=38 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:59.633104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:48:08.922072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16504508-8663-4ff9-ab08-04db541e1bf3 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:08.922962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16504508-8663-4ff9-ab08-04db541e1bf3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:08.922962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16504508-8663-4ff9-ab08-04db541e1bf3 None None] PATH_INFO: `/v3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:08.924836 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 749/1511] 10.222.0.60 () {54 vars in 925 bytes} [Sat Sep 17 20:48:08 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:48:08.959062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226660c-4016-4eb0-b7b7-4c6bb0273bbb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:08.959518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226660c-4016-4eb0-b7b7-4c6bb0273bbb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:08.959615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226660c-4016-4eb0-b7b7-4c6bb0273bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:09.011593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5226660c-4016-4eb0-b7b7-4c6bb0273bbb None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:48:09.012792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5226660c-4016-4eb0-b7b7-4c6bb0273bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:09.093476 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 763/1512] 10.222.0.60 () {58 vars in 1015 bytes} [Sat Sep 17 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:48:11.137273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59688461-d9d2-47b1-8bf0-00e91a09d35e None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:11.138206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59688461-d9d2-47b1-8bf0-00e91a09d35e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:11.138206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59688461-d9d2-47b1-8bf0-00e91a09d35e None None] PATH_INFO: `/v3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:11.140099 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 750/1513] 10.222.0.50 () {54 vars in 925 bytes} [Sat Sep 17 20:48:11 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:48:11.147979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814802d-2e47-4a8c-a679-97529354905e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:11.148693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814802d-2e47-4a8c-a679-97529354905e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:11.148861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814802d-2e47-4a8c-a679-97529354905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:11.175899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1814802d-2e47-4a8c-a679-97529354905e None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:48:11.177123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1814802d-2e47-4a8c-a679-97529354905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:11.187858 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 764/1514] 10.222.0.50 () {58 vars in 1015 bytes} [Sat Sep 17 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:48:12.877243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d3f5-0897-4b21-869d-e98e41e314c5 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:12.877743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d3f5-0897-4b21-869d-e98e41e314c5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:12.877850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511d3f5-0897-4b21-869d-e98e41e314c5 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:12.879238 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 751/1515] 10.222.0.26 () {54 vars in 961 bytes} [Sat Sep 17 20:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:48:12.884402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5ad1-7c2d-4407-b0b0-2d2a58d6e994 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:12.884850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5ad1-7c2d-4407-b0b0-2d2a58d6e994 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:12.884850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5ad1-7c2d-4407-b0b0-2d2a58d6e994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:12.905570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0c7b5ad1-7c2d-4407-b0b0-2d2a58d6e994 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:48:12.906774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0c7b5ad1-7c2d-4407-b0b0-2d2a58d6e994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:12.916292 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 765/1516] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Sep 17 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:48:12.924500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511d3f5-0897-4b21-869d-e98e41e314c5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:48:12.938822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511d3f5-0897-4b21-869d-e98e41e314c5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:48:12.944273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7af67-4c0d-4432-b68f-e7b7cbbd1164 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:48:12.944870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7af67-4c0d-4432-b68f-e7b7cbbd1164 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:12.945145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7af67-4c0d-4432-b68f-e7b7cbbd1164 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:12.945415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7af67-4c0d-4432-b68f-e7b7cbbd1164 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:12.954187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7af67-4c0d-4432-b68f-e7b7cbbd1164 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:48:12.955772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7af67-4c0d-4432-b68f-e7b7cbbd1164 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:48:12.965605 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 752/1517] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:48:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:48:14.876350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b5ad1-7c2d-4407-b0b0-2d2a58d6e994 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:48:14.887416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b5ad1-7c2d-4407-b0b0-2d2a58d6e994 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:48:14.893155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeb318-b8ce-4748-92be-bbf2fb27bd00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:48:14.893882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeb318-b8ce-4748-92be-bbf2fb27bd00 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:14.894233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeb318-b8ce-4748-92be-bbf2fb27bd00 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:14.894536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeb318-b8ce-4748-92be-bbf2fb27bd00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:14.902810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aeb318-b8ce-4748-92be-bbf2fb27bd00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:48:14.904820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aeb318-b8ce-4748-92be-bbf2fb27bd00 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:48:14.914490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 766/1518] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:48:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:48:22.961899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:48:24.910793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=56 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:48:50.416453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd11eb6-1de1-40e0-b414-762a67637f7e None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:50.417082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd11eb6-1de1-40e0-b414-762a67637f7e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:50.417082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd11eb6-1de1-40e0-b414-762a67637f7e None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:50.418408 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 753/1519] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:48:50.423313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a416f34-c084-42ed-8315-83a3311c9c02 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:50.423827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a416f34-c084-42ed-8315-83a3311c9c02 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:50.423827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a416f34-c084-42ed-8315-83a3311c9c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:50.444921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3a416f34-c084-42ed-8315-83a3311c9c02 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:48:50.446168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3a416f34-c084-42ed-8315-83a3311c9c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:50.507961 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 767/1520] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:48:50.677628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd11eb6-1de1-40e0-b414-762a67637f7e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:48:50.679501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bcd11eb6-1de1-40e0-b414-762a67637f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:50.785741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd11eb6-1de1-40e0-b414-762a67637f7e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:48:50.791667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa338e10-091c-41e5-84af-d16e82751b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:48:50.792275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa338e10-091c-41e5-84af-d16e82751b50 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:50.792556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa338e10-091c-41e5-84af-d16e82751b50 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:50.792828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa338e10-091c-41e5-84af-d16e82751b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:50.801033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa338e10-091c-41e5-84af-d16e82751b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:48:50.802657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa338e10-091c-41e5-84af-d16e82751b50 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:48:50.812133 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 754/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:48:52.463156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7266b-1c07-4dc9-8a68-1ed63e45c9e0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:52.463898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7266b-1c07-4dc9-8a68-1ed63e45c9e0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:52.463898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7266b-1c07-4dc9-8a68-1ed63e45c9e0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:52.465181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:48:52.471820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4019223-1b18-4baf-b8db-38c4ac06bdea None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:52.472290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4019223-1b18-4baf-b8db-38c4ac06bdea None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:52.472411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4019223-1b18-4baf-b8db-38c4ac06bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:52.500430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b4019223-1b18-4baf-b8db-38c4ac06bdea None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:48:52.501677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4019223-1b18-4baf-b8db-38c4ac06bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:52.510784 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:48:52.670885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7266b-1c07-4dc9-8a68-1ed63e45c9e0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:48:52.694082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7266b-1c07-4dc9-8a68-1ed63e45c9e0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:48:52.700782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd2f5f-a7e3-4a82-9f5e-7925942bbd13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:48:52.701391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd2f5f-a7e3-4a82-9f5e-7925942bbd13 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:52.701701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd2f5f-a7e3-4a82-9f5e-7925942bbd13 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:52.701971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd2f5f-a7e3-4a82-9f5e-7925942bbd13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:52.710799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd2f5f-a7e3-4a82-9f5e-7925942bbd13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:48:52.712407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd2f5f-a7e3-4a82-9f5e-7925942bbd13 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:48:52.722125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 769/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:48:56.922884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a122-26d3-4a14-bfab-d86b78191348 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:56.923427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a122-26d3-4a14-bfab-d86b78191348 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:56.923427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080a122-26d3-4a14-bfab-d86b78191348 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:56.924703 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:48:56.930880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ecd0c-1c3b-428f-b39f-1cf5d67f5773 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:56.931359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ecd0c-1c3b-428f-b39f-1cf5d67f5773 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:56.931454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ecd0c-1c3b-428f-b39f-1cf5d67f5773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:56.956142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7e3ecd0c-1c3b-428f-b39f-1cf5d67f5773 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:48:56.957986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7e3ecd0c-1c3b-428f-b39f-1cf5d67f5773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:56.967856 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:48:57.310858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080a122-26d3-4a14-bfab-d86b78191348 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:48:57.314278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3080a122-26d3-4a14-bfab-d86b78191348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:57.377721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080a122-26d3-4a14-bfab-d86b78191348 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:48:57.381956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6643b-a056-4ea4-95b8-c86179218ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:48:57.382595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6643b-a056-4ea4-95b8-c86179218ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:57.382873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6643b-a056-4ea4-95b8-c86179218ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:57.383143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6643b-a056-4ea4-95b8-c86179218ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:57.392595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6643b-a056-4ea4-95b8-c86179218ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:48:57.394086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6643b-a056-4ea4-95b8-c86179218ca6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:48:57.403570 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 757/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:48:59.776103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c527-3c6c-4738-bd86-5fb0535b34b4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:59.776692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c527-3c6c-4738-bd86-5fb0535b34b4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:59.776692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c527-3c6c-4738-bd86-5fb0535b34b4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:59.778051 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:48:59.784300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b4eb6-3295-47c8-8e49-4ffe65bdbcac None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:48:59.784779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b4eb6-3295-47c8-8e49-4ffe65bdbcac None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:48:59.784887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b4eb6-3295-47c8-8e49-4ffe65bdbcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:48:59.812557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-492b4eb6-3295-47c8-8e49-4ffe65bdbcac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:48:59.813812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-492b4eb6-3295-47c8-8e49-4ffe65bdbcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:48:59.823022 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:49:00.161529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c527-3c6c-4738-bd86-5fb0535b34b4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:49:00.164515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-27d0c527-3c6c-4738-bd86-5fb0535b34b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:49:00.179826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c527-3c6c-4738-bd86-5fb0535b34b4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:49:00.184247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad95c8-7af6-414c-b58a-059b2d0c1008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:49:00.184876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad95c8-7af6-414c-b58a-059b2d0c1008 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:49:00.185149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad95c8-7af6-414c-b58a-059b2d0c1008 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:49:00.185441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad95c8-7af6-414c-b58a-059b2d0c1008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:49:00.194724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad95c8-7af6-414c-b58a-059b2d0c1008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:49:00.196278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad95c8-7af6-414c-b58a-059b2d0c1008 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:49:00.206091 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 772/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:49:09.807650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=78 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:49:10.201836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=42 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:49:49.905232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe225ce-ce3b-4fb7-855f-51bbe68e9814 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:49:49.905820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe225ce-ce3b-4fb7-855f-51bbe68e9814 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:49:49.905820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe225ce-ce3b-4fb7-855f-51bbe68e9814 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:49:49.907165 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 759/1531] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:49:49.912470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56658925-4d24-49dc-883a-8068cd215fed None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:49:49.912906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56658925-4d24-49dc-883a-8068cd215fed None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:49:49.913045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56658925-4d24-49dc-883a-8068cd215fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:49:49.933787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-56658925-4d24-49dc-883a-8068cd215fed None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:49:49.934963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-56658925-4d24-49dc-883a-8068cd215fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:49:49.943960 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 773/1532] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:49:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:49:59.928526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:22.998623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb64ba-9fa4-4594-9e6b-8d1b74184f7f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:22.999167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb64ba-9fa4-4594-9e6b-8d1b74184f7f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:22.999167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb64ba-9fa4-4594-9e6b-8d1b74184f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.024174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-22bb64ba-9fa4-4594-9e6b-8d1b74184f7f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.025368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-22bb64ba-9fa4-4594-9e6b-8d1b74184f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.035800 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 760/1533] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:50:23.045326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98040850-53c5-4740-9285-40ceec31f637 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.046402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98040850-53c5-4740-9285-40ceec31f637 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.046877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98040850-53c5-4740-9285-40ceec31f637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.078467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-98040850-53c5-4740-9285-40ceec31f637 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.080031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-98040850-53c5-4740-9285-40ceec31f637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.089003 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 774/1534] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:50:23.094210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb64ba-9fa4-4594-9e6b-8d1b74184f7f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.106742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb64ba-9fa4-4594-9e6b-8d1b74184f7f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.116797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.117416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.117711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.117980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.118634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:23.118930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.119962 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:50:23.119962 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 20:50:23.119962 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 20:50:23.120845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.137219 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 761/1535] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:23.145669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98040850-53c5-4740-9285-40ceec31f637 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.158874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98040850-53c5-4740-9285-40ceec31f637 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.166387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.167286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.167780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.168312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.169149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.170443 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:50:23.170443 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:50:23.170443 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:50:23.172032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.228595 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 775/1536] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:23.233279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.249172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a65f55-0d2a-42ee-8e69-f64668f0759b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.344400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.345022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.345296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.345593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.346358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.347370 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:50:23.347370 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 20:50:23.347370 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 20:50:23.348578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.383443 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 762/1537] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:23.388130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.402628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06160e-b72a-46bf-b207-74bbcd36c414 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.412726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e5cd7-8627-48db-81e1-ec11f6802fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.413693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e5cd7-8627-48db-81e1-ec11f6802fca admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.414194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e5cd7-8627-48db-81e1-ec11f6802fca admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.414683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e5cd7-8627-48db-81e1-ec11f6802fca admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.415534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e5cd7-8627-48db-81e1-ec11f6802fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.417441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e5cd7-8627-48db-81e1-ec11f6802fca admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.434546 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 776/1538] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:50:23.439243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.452501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5933-84fb-4331-b587-96ba0c0ef38f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.462261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.463133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.463243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.463543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.464195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.465569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cde5c-f833-4ace-8086-0fb17d9623c6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.465946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.466308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cde5c-f833-4ace-8086-0fb17d9623c6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.466777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cde5c-f833-4ace-8086-0fb17d9623c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.480766 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 763/1539] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:23.493795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.499525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-564cde5c-f833-4ace-8086-0fb17d9623c6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.501173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-564cde5c-f833-4ace-8086-0fb17d9623c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.509506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca80ac-0a55-478b-a59a-fdef3a512c39 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.514072 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 777/1540] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:23.514269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef213b-0a2a-418d-a0ec-512cc3c3f3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.514998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef213b-0a2a-418d-a0ec-512cc3c3f3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.515277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef213b-0a2a-418d-a0ec-512cc3c3f3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.515547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef213b-0a2a-418d-a0ec-512cc3c3f3f7 admin admin] PATH_INFO: `/v3/projects/aa4ffd27d7414061b67e053657044400/users/f05aab669547487d8876a70e338d3e4a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.525010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153ed6-d743-4f3f-bdcd-0bbb0264f47e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.525694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153ed6-d743-4f3f-bdcd-0bbb0264f47e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.526195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153ed6-d743-4f3f-bdcd-0bbb0264f47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.541516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef213b-0a2a-418d-a0ec-512cc3c3f3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4ffd27d7414061b67e053657044400, user_id=f05aab669547487d8876a70e338d3e4a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.541727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef213b-0a2a-418d-a0ec-512cc3c3f3f7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.552418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0e153ed6-d743-4f3f-bdcd-0bbb0264f47e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.558396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0e153ed6-d743-4f3f-bdcd-0bbb0264f47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.571469 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 778/1541] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:23.576201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e153ed6-d743-4f3f-bdcd-0bbb0264f47e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.577607 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 764/1542] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:23 2022] PUT /identity/v3/projects/aa4ffd27d7414061b67e053657044400/users/f05aab669547487d8876a70e338d3e4a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:23.591415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e153ed6-d743-4f3f-bdcd-0bbb0264f47e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.634723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce81d-4800-419e-b62f-c1a5d5ddfccf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.635086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce81d-4800-419e-b62f-c1a5d5ddfccf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.635320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce81d-4800-419e-b62f-c1a5d5ddfccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.684322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e98b5d-fed3-4916-aaf8-b7626decb266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.685372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e98b5d-fed3-4916-aaf8-b7626decb266 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.685922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e98b5d-fed3-4916-aaf8-b7626decb266 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.690382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e98b5d-fed3-4916-aaf8-b7626decb266 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.691316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e98b5d-fed3-4916-aaf8-b7626decb266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:23.696232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e98b5d-fed3-4916-aaf8-b7626decb266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.696984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-213ce81d-4800-419e-b62f-c1a5d5ddfccf None None] MFA Rules not processed for user `f05aab669547487d8876a70e338d3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.697866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-213ce81d-4800-419e-b62f-c1a5d5ddfccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.699768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e98b5d-fed3-4916-aaf8-b7626decb266 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.717666 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 779/1543] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:23.719390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59321e7d-3c0c-49a2-be60-bc3b75fffec6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.719879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59321e7d-3c0c-49a2-be60-bc3b75fffec6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.720351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59321e7d-3c0c-49a2-be60-bc3b75fffec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.751847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-59321e7d-3c0c-49a2-be60-bc3b75fffec6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.753422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-59321e7d-3c0c-49a2-be60-bc3b75fffec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.771548 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 780/1544] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:23.772714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59321e7d-3c0c-49a2-be60-bc3b75fffec6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.787378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59321e7d-3c0c-49a2-be60-bc3b75fffec6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.797937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788cbfb-df44-40cf-93dc-9e2338f3bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.798893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788cbfb-df44-40cf-93dc-9e2338f3bba0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.799378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788cbfb-df44-40cf-93dc-9e2338f3bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.799856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788cbfb-df44-40cf-93dc-9e2338f3bba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.800700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788cbfb-df44-40cf-93dc-9e2338f3bba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.805655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788cbfb-df44-40cf-93dc-9e2338f3bba0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.860082 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 765/1545] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:23.861357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d6eb-f1a3-491e-8636-c3f3328a7d17 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.861679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d6eb-f1a3-491e-8636-c3f3328a7d17 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.861952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d6eb-f1a3-491e-8636-c3f3328a7d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.868067 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 781/1546] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:23.869818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953c8ab-5af5-41ee-af7c-ea8d426f583e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.870318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953c8ab-5af5-41ee-af7c-ea8d426f583e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.870792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953c8ab-5af5-41ee-af7c-ea8d426f583e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.884687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4200d6eb-f1a3-491e-8636-c3f3328a7d17 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.885621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4200d6eb-f1a3-491e-8636-c3f3328a7d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.906609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:23.914443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e953c8ab-5af5-41ee-af7c-ea8d426f583e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.915780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e953c8ab-5af5-41ee-af7c-ea8d426f583e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.956826 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 766/1547] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:23.960969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449d592-c414-48f4-9575-e0f04b2cad6d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.961433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449d592-c414-48f4-9575-e0f04b2cad6d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.965687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449d592-c414-48f4-9575-e0f04b2cad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.988510 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 782/1548] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:23.991789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1449d592-c414-48f4-9575-e0f04b2cad6d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.993052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1449d592-c414-48f4-9575-e0f04b2cad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.993251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818623-dcb4-4283-96e6-13c961e4abd6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.993805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818623-dcb4-4283-96e6-13c961e4abd6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.994317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818623-dcb4-4283-96e6-13c961e4abd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.002363 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 767/1549] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.003152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449d592-c414-48f4-9575-e0f04b2cad6d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.013697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449d592-c414-48f4-9575-e0f04b2cad6d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.015952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-36818623-dcb4-4283-96e6-13c961e4abd6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.017314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36818623-dcb4-4283-96e6-13c961e4abd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.018978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3400612-9526-4f42-bde0-834c2354478b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.019588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400612-9526-4f42-bde0-834c2354478b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.019867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400612-9526-4f42-bde0-834c2354478b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.020376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400612-9526-4f42-bde0-834c2354478b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.020765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3400612-9526-4f42-bde0-834c2354478b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.022716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3400612-9526-4f42-bde0-834c2354478b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.026786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 783/1550] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.028039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36818623-dcb4-4283-96e6-13c961e4abd6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.038527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36818623-dcb4-4283-96e6-13c961e4abd6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.044013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0432179-8d5d-4115-9580-84ab444c3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.044785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0432179-8d5d-4115-9580-84ab444c3959 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.045185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0432179-8d5d-4115-9580-84ab444c3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.045450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0432179-8d5d-4115-9580-84ab444c3959 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.046117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0432179-8d5d-4115-9580-84ab444c3959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.048247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0432179-8d5d-4115-9580-84ab444c3959 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.052935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 768/1551] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.054321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffced02f-e333-4c88-b688-bb6db81ef6d8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.054606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffced02f-e333-4c88-b688-bb6db81ef6d8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.054872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffced02f-e333-4c88-b688-bb6db81ef6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.056617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.074317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 784/1552] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.074317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c837c-ce76-4786-b494-f27776ddde94 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.074317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c837c-ce76-4786-b494-f27776ddde94 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.074317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c837c-ce76-4786-b494-f27776ddde94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.075817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ffced02f-e333-4c88-b688-bb6db81ef6d8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.077022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ffced02f-e333-4c88-b688-bb6db81ef6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.092152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-df9c837c-ce76-4786-b494-f27776ddde94 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.093345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-df9c837c-ce76-4786-b494-f27776ddde94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.135166 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 769/1553] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.136030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffced02f-e333-4c88-b688-bb6db81ef6d8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.144937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffced02f-e333-4c88-b688-bb6db81ef6d8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.150625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e514927-b042-4b2c-9918-bb03a6b57f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.151243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e514927-b042-4b2c-9918-bb03a6b57f3b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.151515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e514927-b042-4b2c-9918-bb03a6b57f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.151798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e514927-b042-4b2c-9918-bb03a6b57f3b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.152553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e514927-b042-4b2c-9918-bb03a6b57f3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:24.152848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e514927-b042-4b2c-9918-bb03a6b57f3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.153888 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 785/1554] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.154828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e514927-b042-4b2c-9918-bb03a6b57f3b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.155137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee9cd-2b25-4c17-bcc1-b04a2782ba76 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.155425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee9cd-2b25-4c17-bcc1-b04a2782ba76 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.155755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee9cd-2b25-4c17-bcc1-b04a2782ba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.171929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 770/1555] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.173491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23253c-2dac-4068-82a1-1b18475241e5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.173839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23253c-2dac-4068-82a1-1b18475241e5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.174163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23253c-2dac-4068-82a1-1b18475241e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.176272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-763ee9cd-2b25-4c17-bcc1-b04a2782ba76 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.177451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-763ee9cd-2b25-4c17-bcc1-b04a2782ba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.186024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 786/1556] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.187225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203139e7-efd9-48e5-a8a4-b4918569a475 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.187494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203139e7-efd9-48e5-a8a4-b4918569a475 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.187772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203139e7-efd9-48e5-a8a4-b4918569a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.199296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0d23253c-2dac-4068-82a1-1b18475241e5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.200566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0d23253c-2dac-4068-82a1-1b18475241e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.206945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-203139e7-efd9-48e5-a8a4-b4918569a475 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.208168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-203139e7-efd9-48e5-a8a4-b4918569a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.208444 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 771/1557] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.209347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23253c-2dac-4068-82a1-1b18475241e5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.218389 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 787/1558] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.218777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23253c-2dac-4068-82a1-1b18475241e5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.219193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203139e7-efd9-48e5-a8a4-b4918569a475 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.223790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132fca3-ed74-4c55-904a-14492c54dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.224391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fca3-ed74-4c55-904a-14492c54dfbd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.224665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fca3-ed74-4c55-904a-14492c54dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.224931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fca3-ed74-4c55-904a-14492c54dfbd admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.225589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132fca3-ed74-4c55-904a-14492c54dfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.227514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132fca3-ed74-4c55-904a-14492c54dfbd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.228613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203139e7-efd9-48e5-a8a4-b4918569a475 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.233354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.233962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.234237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.234501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.235144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.236793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.250767 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 788/1559] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:24.250767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.252423 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 772/1560] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.253796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196aa12-0938-471c-a871-4aea8536c4e9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.254082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196aa12-0938-471c-a871-4aea8536c4e9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.254353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196aa12-0938-471c-a871-4aea8536c4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.260420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0727189-b234-429a-9817-6db97b938342 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.265358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.265982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.266257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.266524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.267209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:24.267490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.269081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.273392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b196aa12-0938-471c-a871-4aea8536c4e9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.274619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b196aa12-0938-471c-a871-4aea8536c4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.289149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 789/1561] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.289314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 773/1562] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.289406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.289525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e0721-580e-4113-b2eb-bf5780ad8413 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.289525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e0721-580e-4113-b2eb-bf5780ad8413 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.289525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e0721-580e-4113-b2eb-bf5780ad8413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.299194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fcafa-8cf5-4649-ac51-b731e6571ff6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.304528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.305236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.305611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.305941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.306312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-128e0721-580e-4113-b2eb-bf5780ad8413 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:24.306652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.307503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-128e0721-580e-4113-b2eb-bf5780ad8413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:24.308722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.315738 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 774/1563] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:24.316682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e0721-580e-4113-b2eb-bf5780ad8413 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.326645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e0721-580e-4113-b2eb-bf5780ad8413 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.331612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.332213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.332522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.332789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.333504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:24.333820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.335400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.336420 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 790/1564] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.339918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.355061 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 775/1565] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.355061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.364867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127880d-fd98-4c16-98a6-db9b54b49c03 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.395457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.402624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce45c16-b4b5-4c3d-a11c-5f03f1e2f0aa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.407683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.408290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.408569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.408840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.409505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:24.409811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.411431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.423577 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 791/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.424521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.426180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.426988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.427238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.427531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.428212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.429866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.434179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac218ed-6d5c-4f28-9cc9-3d822e01d96e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.439098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.439687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.440188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.440288 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 776/1567] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:24.440370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.440969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.441067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.442607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.450482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ba0b4-4876-40c1-a5d8-edf67ce265fb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.453758 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 792/1568] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:24.454589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.457412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.458044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.458319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.458588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.459239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:24.459572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.461163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.465088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b5bc2-f5c5-4bc8-b87f-5ac86486fb3e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.471127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.471714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.471990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.472260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.472879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.473555 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 777/1569] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.474177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.475525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.484307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9075bdf-df92-46f0-a397-dc60f8bb1a72 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.487457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.489267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.489890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.490169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.490433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.491061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:24.491360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.492946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.506236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 793/1570] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.506473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 778/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.507395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.510375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.517647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486600f-6f0e-44a3-a231-aaaa536f21b2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.570796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838aff1-6773-4165-b828-836ef287a78f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.573790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.574497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.574772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.575040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.575519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.575844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.576194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.576476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.576747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.577382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.577841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.579323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.600133 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 779/1572] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.600952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.610517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827938c-2d15-4bde-aefb-c1b2f2677a00 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.610671 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 794/1573] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.611535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.674644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831a7c5-11a5-47b7-adff-2bd753cab541 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.675700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.676313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.676586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.676871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.677744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.679649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.679767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.680317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.680597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.680868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] PATH_INFO: `/v3/projects/e5375b35e2884395beebeb16434f4f16/users/8c17fdfd9c954b6b8f1e89fc663f1a5e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.698239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5375b35e2884395beebeb16434f4f16, user_id=8c17fdfd9c954b6b8f1e89fc663f1a5e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.699361 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:50:24.699361 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:50:24.699361 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:50:24.701310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.712429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 780/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.712429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.718343 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 795/1575] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:24 2022] PUT /identity/v3/projects/e5375b35e2884395beebeb16434f4f16/users/8c17fdfd9c954b6b8f1e89fc663f1a5e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:24.719134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.739717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.789263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc71a5-d919-46ce-9bd5-3d0bb45f0c13 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.789434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8690d41-5c48-44a1-91e2-b5c7be14705b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.793940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.794410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.794559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.794837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.795106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.795205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.795393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.795664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] PATH_INFO: `/v3/projects/13bb5c9671214f449194584247b12e4d/users/553cf96465f34dbcb6cc26d9fe1bd72e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.795766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.797679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.802204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.813734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bb5c9671214f449194584247b12e4d, user_id=553cf96465f34dbcb6cc26d9fe1bd72e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.816629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.827424 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 796/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/projects => generated 395 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.828396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.833213 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 781/1577] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:24 2022] PUT /identity/v3/projects/13bb5c9671214f449194584247b12e4d/users/553cf96465f34dbcb6cc26d9fe1bd72e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:24.836161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.891825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1448cc8d-046d-4695-8236-3fd9f11f5467 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.894314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62933670-c0c1-431d-bdac-70538105273a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.896533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.897140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.897419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.897725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.898355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.898836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.899436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.899708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.899973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.900293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.900587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.902580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.923152 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 797/1578] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.924156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.931161 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 782/1579] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.934147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e2bc-4e94-415b-9018-226b18b99597 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.934725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.990723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.991335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.991606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.991872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.992504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.994143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.998549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a748b1-fe48-41f0-a91d-5ff7377de35f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.003355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518f634-055f-4ec7-a1e5-70e973307fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.003950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 798/1580] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.004037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518f634-055f-4ec7-a1e5-70e973307fed admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.004306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518f634-055f-4ec7-a1e5-70e973307fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.004552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.004709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518f634-055f-4ec7-a1e5-70e973307fed admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.005440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518f634-055f-4ec7-a1e5-70e973307fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.007506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518f634-055f-4ec7-a1e5-70e973307fed admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.014449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1414b07-2f03-44cf-9563-23e234b528d7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.019359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.019962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.020248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.020523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.021149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.023090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.033317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 783/1581] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:25.033317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c0553-a387-471f-ac53-4b16ed8e3086 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.033317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c0553-a387-471f-ac53-4b16ed8e3086 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.033317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c0553-a387-471f-ac53-4b16ed8e3086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.051285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 799/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:25.051285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.060191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a0116-8d56-4615-baa7-3681d20d9fe3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.060516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e10c0553-a387-471f-ac53-4b16ed8e3086 None None] MFA Rules not processed for user `8c17fdfd9c954b6b8f1e89fc663f1a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:25.061843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e10c0553-a387-471f-ac53-4b16ed8e3086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:25.064770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.065367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.065678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.066110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.066298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.066735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.069185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.092952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 800/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/users => generated 298 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:25.093809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.103828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb8f6c-9e1b-4439-b64e-a1164d5278c5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.109208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.109847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.110244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.110510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.111194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.112819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.123313 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 801/1584] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.124101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.129182 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 784/1585] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:25.130227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c0553-a387-471f-ac53-4b16ed8e3086 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.134339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3f1f-5a7a-429c-868e-bcd636ff87e7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.139567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.140174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.140428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.140723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.141366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.142336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c0553-a387-471f-ac53-4b16ed8e3086 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.142974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.147112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.147719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.147988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.148276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.148920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.150890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.153069 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 802/1586] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.154087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.163337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aec6f2-4b56-47c8-8479-dd64f902623e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.168417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.169103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.169393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.169684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.170402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.171984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.179094 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 785/1587] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/users => generated 291 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:25.179094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.182870 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 803/1588] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.183694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.188410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.191795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a25ba-dbe9-4aa0-9e86-5a56b49c5ca6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.193587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9abea-4b18-46ae-8c2e-2d135823ca25 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.196599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.197200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.197499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.197792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.198424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.199403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.200000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.200136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.200231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.200495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.201107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.202707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.216649 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 786/1589] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.216649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.216909 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 804/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.216909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.221808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5dfb4-e03d-4f33-9c21-65fe05afe02c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.223217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee168d3-ff34-4723-ac4d-75d5c943c8c3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.226539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.227130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.227404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.227780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.227879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.228394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.228480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.228667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.228936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] PATH_INFO: `/v3/projects/13bb5c9671214f449194584247b12e4d/users/553cf96465f34dbcb6cc26d9fe1bd72e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.230007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.230905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bb5c9671214f449194584247b12e4d, user_id=553cf96465f34dbcb6cc26d9fe1bd72e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.233536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.243694 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 787/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.243694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.247189 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 805/1592] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:25 2022] PUT /identity/v3/projects/13bb5c9671214f449194584247b12e4d/users/553cf96465f34dbcb6cc26d9fe1bd72e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:25.247918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.262592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17815027-1ec0-4130-b2c9-ddd406d9d9d8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.321249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0582304-7b61-47e4-96a5-baf9ec663798 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.323245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.323867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.324227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.324511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.325143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.326108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.326721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.326998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.327116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.327268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.327944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.329582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.340525 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 806/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.341431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.352151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8a7ae-526e-4d6f-ae6f-492f7573504c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.357362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.358011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.358310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.358563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.359203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.362546 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 788/1594] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:25.362546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.362765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.376556 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 807/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.376556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.388530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f52599-be5b-4650-bf74-89e9c769211f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.416854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.429388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93feda4-243b-406c-b261-e1d2ad068913 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.434834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.435509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.435788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.436064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] PATH_INFO: `/v3/projects/84675c7d7632442ead7879973c5a87ed/users/65c1fa6cebe54ac2898f8a615802b4ea/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.450693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=84675c7d7632442ead7879973c5a87ed, user_id=65c1fa6cebe54ac2898f8a615802b4ea, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.453454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.454700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.455319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.455602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.455874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.456518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.458271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.467261 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 789/1596] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:25 2022] PUT /identity/v3/projects/84675c7d7632442ead7879973c5a87ed/users/65c1fa6cebe54ac2898f8a615802b4ea/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:25.472537 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 808/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.472537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.474512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.480705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289a27c-ecd1-466c-ade5-437c1f59141f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.537696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f41ff-f348-471e-81ef-b3331d170efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.537884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecf817-fa3a-4807-b9bd-b0db16bf2ebe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.538606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f41ff-f348-471e-81ef-b3331d170efb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.539528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f41ff-f348-471e-81ef-b3331d170efb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.539810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f41ff-f348-471e-81ef-b3331d170efb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.540453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f41ff-f348-471e-81ef-b3331d170efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.542114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f41ff-f348-471e-81ef-b3331d170efb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.542889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.543394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.543622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.544166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.544166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.545198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.546966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.554062 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 809/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.555519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca8d5-1196-48e5-867b-7bef736ef378 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.555806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca8d5-1196-48e5-867b-7bef736ef378 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.556084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca8d5-1196-48e5-867b-7bef736ef378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.557845 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 790/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.558719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.569095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f6f06-0ee1-457c-b183-a365bfad3cc3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.574181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.574745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.575152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.575433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] PATH_INFO: `/v3/projects/2378d365d8774876b60c6f00a61ffae4/users/23cc6d06eba24928a2e50e1d1e6c4c8a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.584737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5a4ca8d5-1196-48e5-867b-7bef736ef378 None None] MFA Rules not processed for user `553cf96465f34dbcb6cc26d9fe1bd72e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:25.585985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5a4ca8d5-1196-48e5-867b-7bef736ef378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:25.591291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2378d365d8774876b60c6f00a61ffae4, user_id=23cc6d06eba24928a2e50e1d1e6c4c8a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.594016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.607647 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 791/1600] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:25 2022] PUT /identity/v3/projects/2378d365d8774876b60c6f00a61ffae4/users/23cc6d06eba24928a2e50e1d1e6c4c8a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:25.608427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.654971 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 810/1601] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:25.655918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca8d5-1196-48e5-867b-7bef736ef378 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.666663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca8d5-1196-48e5-867b-7bef736ef378 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.683093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bd50-c94b-4d7c-9273-06ba25b91744 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.688601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.689222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.689576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.689862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.692441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.694699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.720176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 792/1602] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:25.720176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.727678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47c6af-24c3-40e7-9287-02c6753d1782 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.732519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.733165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.733441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.733741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] PATH_INFO: `/v3/projects/e3353bdd6b0b48c8baff5752e19f1d27/users/adede130089948afa19189b9522f7820/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.738536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cfc5e-15c4-431b-84e8-93a249618701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.739141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cfc5e-15c4-431b-84e8-93a249618701 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.739419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cfc5e-15c4-431b-84e8-93a249618701 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.739690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cfc5e-15c4-431b-84e8-93a249618701 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.740619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cfc5e-15c4-431b-84e8-93a249618701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.742726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cfc5e-15c4-431b-84e8-93a249618701 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.748563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3353bdd6b0b48c8baff5752e19f1d27, user_id=adede130089948afa19189b9522f7820, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.751268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.755997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 811/1603] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:25.755997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355f550-8969-493e-aa0f-5ba92a89bcc4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.755997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355f550-8969-493e-aa0f-5ba92a89bcc4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.755997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355f550-8969-493e-aa0f-5ba92a89bcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.767052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 793/1604] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:25 2022] PUT /identity/v3/projects/e3353bdd6b0b48c8baff5752e19f1d27/users/adede130089948afa19189b9522f7820/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:25.768709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.791541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.792066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7355f550-8969-493e-aa0f-5ba92a89bcc4 None None] MFA Rules not processed for user `65c1fa6cebe54ac2898f8a615802b4ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:25.793402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7355f550-8969-493e-aa0f-5ba92a89bcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:25.849380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0faa84-b701-4d4c-8058-83730e6d9394 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.854406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b56b9-001c-4133-bdd3-8107edd27f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.855055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b56b9-001c-4133-bdd3-8107edd27f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.855328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b56b9-001c-4133-bdd3-8107edd27f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.855593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b56b9-001c-4133-bdd3-8107edd27f97 admin admin] PATH_INFO: `/v3/projects/0a55519ffc224c849e9be490acb4596e/users/1e865b793b834802a2bfa7146987d8f2/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.860110 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 812/1605] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:25.863843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355f550-8969-493e-aa0f-5ba92a89bcc4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.871825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b56b9-001c-4133-bdd3-8107edd27f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a55519ffc224c849e9be490acb4596e, user_id=1e865b793b834802a2bfa7146987d8f2, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.872048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355f550-8969-493e-aa0f-5ba92a89bcc4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.876118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b56b9-001c-4133-bdd3-8107edd27f97 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.876681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cfcb5-855f-4666-a79d-33bd04684d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.877343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cfcb5-855f-4666-a79d-33bd04684d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.877655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cfcb5-855f-4666-a79d-33bd04684d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.877932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cfcb5-855f-4666-a79d-33bd04684d54 admin admin] PATH_INFO: `/v3/projects/4e1c57a31bd8425f927f361aaeffec9e/users/a6f71b6b15c442e2b683e337da7a35fc/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.880397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.891313 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 794/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:25 2022] PUT /identity/v3/projects/0a55519ffc224c849e9be490acb4596e/users/1e865b793b834802a2bfa7146987d8f2/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:25.895888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cfcb5-855f-4666-a79d-33bd04684d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1c57a31bd8425f927f361aaeffec9e, user_id=a6f71b6b15c442e2b683e337da7a35fc, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.898627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cfcb5-855f-4666-a79d-33bd04684d54 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.899030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584f7eb-b7d5-4635-9745-22563c9d33f6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.899399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584f7eb-b7d5-4635-9745-22563c9d33f6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.899766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584f7eb-b7d5-4635-9745-22563c9d33f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.913703 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 813/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:25 2022] PUT /identity/v3/projects/4e1c57a31bd8425f927f361aaeffec9e/users/a6f71b6b15c442e2b683e337da7a35fc/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:25.915201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40cf6-ef17-4596-87ef-6fba468f692d None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.915490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40cf6-ef17-4596-87ef-6fba468f692d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.915766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f40cf6-ef17-4596-87ef-6fba468f692d None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.917156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 814/1608] 10.222.0.26 () {54 vars in 965 bytes} [Sat Sep 17 20:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:25.922337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f40cf6-ef17-4596-87ef-6fba468f692d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.930250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6584f7eb-b7d5-4635-9745-22563c9d33f6 None None] MFA Rules not processed for user `23cc6d06eba24928a2e50e1d1e6c4c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:25.931450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6584f7eb-b7d5-4635-9745-22563c9d33f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:25.990300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f40cf6-ef17-4596-87ef-6fba468f692d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.994789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65798166-1326-4e72-ab86-e8081afb94a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.995400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798166-1326-4e72-ab86-e8081afb94a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.995702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798166-1326-4e72-ab86-e8081afb94a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.996006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798166-1326-4e72-ab86-e8081afb94a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.996248 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 795/1609] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:25.997097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584f7eb-b7d5-4635-9745-22563c9d33f6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.997908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65798166-1326-4e72-ab86-e8081afb94a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.999515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65798166-1326-4e72-ab86-e8081afb94a8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.009647 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 815/1610] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:26.010924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeefda-7adb-4503-8f6b-7a709cbcad6c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.011213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeefda-7adb-4503-8f6b-7a709cbcad6c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.011489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafeefda-7adb-4503-8f6b-7a709cbcad6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.013793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584f7eb-b7d5-4635-9745-22563c9d33f6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.018461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.019083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.019355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.019619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] PATH_INFO: `/v3/projects/332a658e83ff48de8b6c50ff0a113463/users/8c6db72795fc47bd8c820ac966cbd731/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.035779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332a658e83ff48de8b6c50ff0a113463, user_id=8c6db72795fc47bd8c820ac966cbd731, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.038469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.039418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fafeefda-7adb-4503-8f6b-7a709cbcad6c None None] MFA Rules not processed for user `adede130089948afa19189b9522f7820`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:26.040659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fafeefda-7adb-4503-8f6b-7a709cbcad6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:26.053417 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 796/1611] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:25 2022] PUT /identity/v3/projects/332a658e83ff48de8b6c50ff0a113463/users/8c6db72795fc47bd8c820ac966cbd731/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:26.054332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.115292 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 816/1612] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:26.116669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887cc25-5b83-4cf0-b994-619d2cdd8755 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.116953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887cc25-5b83-4cf0-b994-619d2cdd8755 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.117224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887cc25-5b83-4cf0-b994-619d2cdd8755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.119805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d56a-4a5d-4223-8658-d08824ebffe0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.125107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45731b36-9199-409c-ba37-5452d555bc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.126809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45731b36-9199-409c-ba37-5452d555bc2b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.127097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45731b36-9199-409c-ba37-5452d555bc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.127373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45731b36-9199-409c-ba37-5452d555bc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.128065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45731b36-9199-409c-ba37-5452d555bc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.130103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45731b36-9199-409c-ba37-5452d555bc2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.134911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:26.147337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6887cc25-5b83-4cf0-b994-619d2cdd8755 None None] MFA Rules not processed for user `1e865b793b834802a2bfa7146987d8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:26.148571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6887cc25-5b83-4cf0-b994-619d2cdd8755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:26.160482 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 797/1613] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:26.161887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53161a7-d16c-448f-9949-2d9e0e87b06c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.162172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53161a7-d16c-448f-9949-2d9e0e87b06c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.162464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53161a7-d16c-448f-9949-2d9e0e87b06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.203131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e53161a7-d16c-448f-9949-2d9e0e87b06c None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:26.204347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e53161a7-d16c-448f-9949-2d9e0e87b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:26.214049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:26.220891 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 817/1614] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:26.223055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c0854-20f4-4053-951f-ed48bbc0f29e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.223658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c0854-20f4-4053-951f-ed48bbc0f29e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.224140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c0854-20f4-4053-951f-ed48bbc0f29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.252837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-283c0854-20f4-4053-951f-ed48bbc0f29e None None] MFA Rules not processed for user `a6f71b6b15c442e2b683e337da7a35fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:26.254074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-283c0854-20f4-4053-951f-ed48bbc0f29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:26.271415 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 798/1615] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:26.272336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53161a7-d16c-448f-9949-2d9e0e87b06c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.318491 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 818/1616] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:26.319558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c0854-20f4-4053-951f-ed48bbc0f29e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.332082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53161a7-d16c-448f-9949-2d9e0e87b06c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.336546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77628d-de22-414f-90c1-a055502a3ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.337155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77628d-de22-414f-90c1-a055502a3ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.337424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77628d-de22-414f-90c1-a055502a3ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.337719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77628d-de22-414f-90c1-a055502a3ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.338341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77628d-de22-414f-90c1-a055502a3ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.340423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77628d-de22-414f-90c1-a055502a3ef1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.370872 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 799/1617] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:26.372150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d46f6c-a329-49b5-b3ff-08b26156b1c5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.372435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d46f6c-a329-49b5-b3ff-08b26156b1c5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.372710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d46f6c-a329-49b5-b3ff-08b26156b1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.386247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c0854-20f4-4053-951f-ed48bbc0f29e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.401798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-25d46f6c-a329-49b5-b3ff-08b26156b1c5 None None] MFA Rules not processed for user `8c6db72795fc47bd8c820ac966cbd731`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:26.403096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-25d46f6c-a329-49b5-b3ff-08b26156b1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:26.441078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:26.446521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.447675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.448028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.448298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] PATH_INFO: `/v3/projects/49303d354e4442f9aaea429c760892db/users/d93585c867aa484b82b32a8bd4c00038/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.464482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49303d354e4442f9aaea429c760892db, user_id=d93585c867aa484b82b32a8bd4c00038, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.467402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.467869 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 800/1618] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:26.468712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d46f6c-a329-49b5-b3ff-08b26156b1c5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.484823 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 819/1619] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:26 2022] PUT /identity/v3/projects/49303d354e4442f9aaea429c760892db/users/d93585c867aa484b82b32a8bd4c00038/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:26.485773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.505945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:26.524690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d46f6c-a329-49b5-b3ff-08b26156b1c5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.548722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5c9a4-19d4-4f87-b52e-51120482a2a0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.553247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.553891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.554163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.554431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.555049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.556985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.573991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.574630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.574910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.575185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.581454 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 820/1620] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:26.581454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.590892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea764-77c9-491e-bbaf-25f789c7fe7b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.595686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.596310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.596587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.596858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.597498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.599417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.631962 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 821/1621] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:26.631962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.635880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.637439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.646702 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 801/1622] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:26.647628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.658197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa0c16-1623-4266-9896-af0e880eed98 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.712610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:26.720411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192063d-1265-4927-8387-1d4c21857269 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.726744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.726899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.727493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.727599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.727886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.727993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.728269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.728357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.729016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.731141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.764018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 802/1623] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:26.764018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.799945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.801733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.812786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 822/1624] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:26.813651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.823984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86767f0b-7a55-4398-a6fa-b86caaa0a492 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.837653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:26.841278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21c965-6ab0-4451-ba0f-3ed0b576973f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.847776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.848396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.848676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.848950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.850300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.852478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.879113 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 803/1625] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:26.880146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.889813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.890706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.891245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.891613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.892439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.894836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.896188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a22fbe-e423-4aa0-8a07-7f80a6ebb022 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.901104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.901907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.902521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.902927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.904088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.905901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.917810 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 804/1626] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:26.919181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:26.931129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c8a55-8fb7-4688-8f17-b318f6c7fa8f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.933628 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 823/1627] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:26.934605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.003636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.004453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.004566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.004787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.005452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.007189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.017829 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 805/1628] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:27.018673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.024940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdfe56-b73c-4ceb-81a5-f1d1cb172513 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.029260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa2eda-4fdf-4968-894a-8cc9f2996083 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.030872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.031558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.031895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.032222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.032919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.034007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.034600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.034887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.035147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.035247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.035799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.037862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.061802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 824/1629] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:26 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:27.061802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.062033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 806/1630] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:27 2022] POST /identity/v3/users => generated 291 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:27.062033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.073526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fc552-0c67-408f-8e31-86e7e228d475 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.073678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6222bd2-e0d5-4799-a664-03f6583de6a0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.078150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.078855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.079194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.079347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.079583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.079998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.080275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.080369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.080546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] PATH_INFO: `/v3/projects/49303d354e4442f9aaea429c760892db/users/d93585c867aa484b82b32a8bd4c00038/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.082113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.082501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49303d354e4442f9aaea429c760892db, user_id=d93585c867aa484b82b32a8bd4c00038, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.085856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.096873 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 825/1631] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:27.097583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.100095 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 807/1632] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:27 2022] PUT /identity/v3/projects/49303d354e4442f9aaea429c760892db/users/d93585c867aa484b82b32a8bd4c00038/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.101051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.107740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c279fc-8049-4925-8429-4d1b199d51bb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.130973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:27.169227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.169860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.170054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.170327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.170959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.172873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.182508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2810e-3180-40e0-946e-238219135863 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.188921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.189749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.190016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.190349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] PATH_INFO: `/v3/users/adede130089948afa19189b9522f7820` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.191486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adede130089948afa19189b9522f7820)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.193424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.197157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 826/1633] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:27 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:27.198901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.212035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee665d-bd42-4d6f-9839-3a41a6babd47 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.217352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.218673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.219876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.220167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] PATH_INFO: `/v3/projects/f5b644fe62414b3ab02fd4fa12e2a4e5/users/3535c9a33ab544688ff3f35f9f139748/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.231991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:27.240309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b644fe62414b3ab02fd4fa12e2a4e5, user_id=3535c9a33ab544688ff3f35f9f139748, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.243200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.246298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adede130089948afa19189b9522f7820'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:27.247560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adede130089948afa19189b9522f7820'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:27.256499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adede130089948afa19189b9522f7820'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:27.257494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adede130089948afa19189b9522f7820'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:27.260899 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 827/1634] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:27 2022] PUT /identity/v3/projects/f5b644fe62414b3ab02fd4fa12e2a4e5/users/3535c9a33ab544688ff3f35f9f139748/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.261767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.263350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.276490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adede130089948afa19189b9522f7820'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:27.291320 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 808/1635] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:27 2022] DELETE /identity/v3/users/adede130089948afa19189b9522f7820 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.292209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.294869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.330925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ad321-dd85-4ffd-99da-fee11c85be37 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.364288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aab89-5d8e-43fb-be31-91dcd369d4b4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.369338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.370044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.370395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.371424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.372169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.374006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.388995 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 809/1636] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:27.388995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.391548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.402171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b1220-0085-4c47-87ae-32001ba843c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.402890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b1220-0085-4c47-87ae-32001ba843c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.403657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b1220-0085-4c47-87ae-32001ba843c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.404026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b1220-0085-4c47-87ae-32001ba843c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.404786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b1220-0085-4c47-87ae-32001ba843c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.406646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b1220-0085-4c47-87ae-32001ba843c0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.411730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e461d3-5e61-49cd-9834-7c8101f0bf1b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.421977 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 828/1637] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:27.421977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.421977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.421977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.425166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.426099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.426477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.426845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] PATH_INFO: `/v3/projects/575dbc45397b43c4a14811f2ef69616c/users/7fbe8c0f8b184630abba1cb6bf8673f1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.445901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] RBAC: Authorizing `identity:create_grant(project_id=575dbc45397b43c4a14811f2ef69616c, user_id=7fbe8c0f8b184630abba1cb6bf8673f1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.449269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.451595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] MFA Rules not processed for user `d93585c867aa484b82b32a8bd4c00038`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:27.453105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.473581 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 810/1638] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:27 2022] PUT /identity/v3/projects/575dbc45397b43c4a14811f2ef69616c/users/7fbe8c0f8b184630abba1cb6bf8673f1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.475861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.477812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.489484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:27.539607 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 829/1639] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:27.540327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.541825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.555424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d98eb2-34c8-451e-8014-ce3d866c2935 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.560615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdfaaa-1187-4e21-bb44-a34c9f6b05ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.561343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdfaaa-1187-4e21-bb44-a34c9f6b05ca admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.561763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdfaaa-1187-4e21-bb44-a34c9f6b05ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.561880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa40a80-d331-4f23-aef4-ed6423d283da admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.562259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdfaaa-1187-4e21-bb44-a34c9f6b05ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.562997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdfaaa-1187-4e21-bb44-a34c9f6b05ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.565023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdfaaa-1187-4e21-bb44-a34c9f6b05ca admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.567394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.568486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.568768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.569094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.569788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.571902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.576775 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 830/1640] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:27.578279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e66ef-cf58-448d-b969-c81ceea85fb4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.578660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e66ef-cf58-448d-b969-c81ceea85fb4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.579020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e66ef-cf58-448d-b969-c81ceea85fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.585644 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 811/1641] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:27.585644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.591229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:27.601749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ffde-aa37-4097-8903-cf044f034372 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.606935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a83e66ef-cf58-448d-b969-c81ceea85fb4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:27.607453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.608150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.608269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a83e66ef-cf58-448d-b969-c81ceea85fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.608653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.608962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] PATH_INFO: `/v3/projects/a6b28cee0cb44e5d8e52af9bf22e7e11/users/525e8cc77ab744f5837b3912e9d754a3/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.615928 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 831/1642] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:27.616492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e66ef-cf58-448d-b969-c81ceea85fb4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.626297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e66ef-cf58-448d-b969-c81ceea85fb4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.630992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b28cee0cb44e5d8e52af9bf22e7e11, user_id=525e8cc77ab744f5837b3912e9d754a3, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.631187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.631862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.632142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.632413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] PATH_INFO: `/v3/projects/d0af2d58b8994eeeb64cec26101d4816/users/62d984d1cf7c41f783ddbe05bc545e4a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.633919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.647652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0af2d58b8994eeeb64cec26101d4816, user_id=62d984d1cf7c41f783ddbe05bc545e4a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.647890 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 812/1643] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:27 2022] PUT /identity/v3/projects/a6b28cee0cb44e5d8e52af9bf22e7e11/users/525e8cc77ab744f5837b3912e9d754a3/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.648718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.650486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.666314 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 832/1644] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:27 2022] PUT /identity/v3/projects/d0af2d58b8994eeeb64cec26101d4816/users/62d984d1cf7c41f783ddbe05bc545e4a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.667359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.669520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.717970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d97e15-1af8-4c42-a00e-6aada86549b5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.744467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a4bd2-5b43-466b-8683-2c2690b85ed2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.750982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.751709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.752058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.752400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.772996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.773635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.773893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.774160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.774832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.776434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.787411 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 813/1645] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:27.788236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.798312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d9c91-e14d-44e9-82a9-33cceb860e7b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.803077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.803732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.804035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.804284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] PATH_INFO: `/v3/projects/f5b644fe62414b3ab02fd4fa12e2a4e5/users/3535c9a33ab544688ff3f35f9f139748/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.806164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b644fe62414b3ab02fd4fa12e2a4e5, user_id=3535c9a33ab544688ff3f35f9f139748, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.808986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.818835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.819748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 814/1646] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:27 2022] PUT /identity/v3/projects/f5b644fe62414b3ab02fd4fa12e2a4e5/users/3535c9a33ab544688ff3f35f9f139748/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.828969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.829113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.833763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:27.834859 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 833/1647] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:27.835719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.837226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.896352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc584fc-1ac4-4ca1-8bb8-44d1d3bdfeba service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.908641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a886534-1a8c-431f-a9e7-69b136d82031 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:27.908641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182c822-70e1-47bc-8489-50242e2bf3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.908641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182c822-70e1-47bc-8489-50242e2bf3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.908641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182c822-70e1-47bc-8489-50242e2bf3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.908928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182c822-70e1-47bc-8489-50242e2bf3a0 admin admin] PATH_INFO: `/v3/projects/504f9c5f7884489084c9a00bf846ed98/users/764d9954a6ef48c0bb28d6dc38603265/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.921913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182c822-70e1-47bc-8489-50242e2bf3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504f9c5f7884489084c9a00bf846ed98, user_id=764d9954a6ef48c0bb28d6dc38603265, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.924551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:27.924661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182c822-70e1-47bc-8489-50242e2bf3a0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.939632 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 815/1648] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:27 2022] PUT /identity/v3/projects/504f9c5f7884489084c9a00bf846ed98/users/764d9954a6ef48c0bb28d6dc38603265/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:27.941500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03944-696f-4821-b800-6b815b65113b None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.941500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03944-696f-4821-b800-6b815b65113b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.941695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03944-696f-4821-b800-6b815b65113b None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.943058 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 816/1649] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:27.943700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a03944-696f-4821-b800-6b815b65113b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.958966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:27.959582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.959861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.960132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.969715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:27.971280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:27.980848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 834/1650] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:27.981763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:27.990957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99ce22-8583-4f7e-9c34-07f528873af9 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.010432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a03944-696f-4821-b800-6b815b65113b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.015690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.016426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.016705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.016972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.017756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.019358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.031173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 817/1651] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:28.032004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.041806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4ac9-421a-4fc4-b5f1-b8d4be485991 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.046772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db802e9-7046-4a96-a13a-3a0788507305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.047424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db802e9-7046-4a96-a13a-3a0788507305 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.047691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db802e9-7046-4a96-a13a-3a0788507305 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.047962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db802e9-7046-4a96-a13a-3a0788507305 admin admin] PATH_INFO: `/v3/projects/575dbc45397b43c4a14811f2ef69616c/users/7fbe8c0f8b184630abba1cb6bf8673f1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.050164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db802e9-7046-4a96-a13a-3a0788507305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575dbc45397b43c4a14811f2ef69616c, user_id=7fbe8c0f8b184630abba1cb6bf8673f1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.052848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db802e9-7046-4a96-a13a-3a0788507305 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.054712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.055435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.055832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.056311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.057066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.059463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.064939 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 818/1652] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:28 2022] PUT /identity/v3/projects/575dbc45397b43c4a14811f2ef69616c/users/7fbe8c0f8b184630abba1cb6bf8673f1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:28.073712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257efce9-c32a-4532-ba5b-8cfb50e56b97 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.073712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257efce9-c32a-4532-ba5b-8cfb50e56b97 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.073712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257efce9-c32a-4532-ba5b-8cfb50e56b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.080299 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 835/1653] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:28.080299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.101986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-257efce9-c32a-4532-ba5b-8cfb50e56b97 None None] MFA Rules not processed for user `3535c9a33ab544688ff3f35f9f139748`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.103453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-257efce9-c32a-4532-ba5b-8cfb50e56b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.160410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32e7e1-6da9-4309-be5d-3111b5509bf7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.181924 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 819/1654] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:28.184548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257efce9-c32a-4532-ba5b-8cfb50e56b97 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.191719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.231304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.244876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.245692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.246035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.246313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.248102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.256641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257efce9-c32a-4532-ba5b-8cfb50e56b97 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.261899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f90e5-cf45-4960-9929-f03f08798058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.262519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f90e5-cf45-4960-9929-f03f08798058 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.262802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f90e5-cf45-4960-9929-f03f08798058 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.263098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f90e5-cf45-4960-9929-f03f08798058 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.263722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f90e5-cf45-4960-9929-f03f08798058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.265491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f90e5-cf45-4960-9929-f03f08798058 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.275883 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 820/1655] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:28.277121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2878-5abc-4d30-b478-97ea6bc7de25 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.277453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2878-5abc-4d30-b478-97ea6bc7de25 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.277745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2878-5abc-4d30-b478-97ea6bc7de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.299431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-43fc2878-5abc-4d30-b478-97ea6bc7de25 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.300778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43fc2878-5abc-4d30-b478-97ea6bc7de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.326734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.328359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.341632 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 836/1656] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:28.342531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.353279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e6cf3-fc40-4a4e-b47a-643bb175b5cc service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.358282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077bc7d-0217-4fb4-81cb-54aa60565fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.359009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bc7d-0217-4fb4-81cb-54aa60565fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.359440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bc7d-0217-4fb4-81cb-54aa60565fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.359722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bc7d-0217-4fb4-81cb-54aa60565fe7 admin admin] PATH_INFO: `/v3/projects/a6b28cee0cb44e5d8e52af9bf22e7e11/users/525e8cc77ab744f5837b3912e9d754a3/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.361875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077bc7d-0217-4fb4-81cb-54aa60565fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b28cee0cb44e5d8e52af9bf22e7e11, user_id=525e8cc77ab744f5837b3912e9d754a3, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.364565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077bc7d-0217-4fb4-81cb-54aa60565fe7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.376586 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 837/1657] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:28 2022] PUT /identity/v3/projects/a6b28cee0cb44e5d8e52af9bf22e7e11/users/525e8cc77ab744f5837b3912e9d754a3/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:28.377919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.378211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.378482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.392863 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 821/1658] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:28.399469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc2878-5abc-4d30-b478-97ea6bc7de25 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.410434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc2878-5abc-4d30-b478-97ea6bc7de25 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.421478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] MFA Rules not processed for user `7fbe8c0f8b184630abba1cb6bf8673f1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.422784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.468847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.469547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.469822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.470101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] PATH_INFO: `/v3/projects/d0af2d58b8994eeeb64cec26101d4816/users/62d984d1cf7c41f783ddbe05bc545e4a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.472103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0af2d58b8994eeeb64cec26101d4816, user_id=62d984d1cf7c41f783ddbe05bc545e4a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.474967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.486128 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 822/1659] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:28 2022] PUT /identity/v3/projects/d0af2d58b8994eeeb64cec26101d4816/users/62d984d1cf7c41f783ddbe05bc545e4a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:28.486854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.499661 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 838/1660] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:28.501182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.502688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.517369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888965a-ee8d-4f38-bf3b-31b397b9dfe1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.518072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.559617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cbef7-4629-40b4-93c4-f69da46b12d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.565626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6924b44-5290-4a22-a039-509e8827e67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.566318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6924b44-5290-4a22-a039-509e8827e67d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.566591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6924b44-5290-4a22-a039-509e8827e67d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.566861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6924b44-5290-4a22-a039-509e8827e67d admin admin] PATH_INFO: `/v3/projects/504f9c5f7884489084c9a00bf846ed98/users/764d9954a6ef48c0bb28d6dc38603265/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.568933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6924b44-5290-4a22-a039-509e8827e67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=504f9c5f7884489084c9a00bf846ed98, user_id=764d9954a6ef48c0bb28d6dc38603265, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.571652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6924b44-5290-4a22-a039-509e8827e67d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.580572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.581247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.581583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.581857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] PATH_INFO: `/v3/projects/e3353bdd6b0b48c8baff5752e19f1d27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.582289 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 823/1661] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:28 2022] PUT /identity/v3/projects/504f9c5f7884489084c9a00bf846ed98/users/764d9954a6ef48c0bb28d6dc38603265/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:28.583324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3353bdd6b0b48c8baff5752e19f1d27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.583569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.583774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.584034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.585647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.597186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.614851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] MFA Rules not processed for user `525e8cc77ab744f5837b3912e9d754a3`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.616201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.635549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3353bdd6b0b48c8baff5752e19f1d27', 'request_id': 'req-40f721a9-f213-4ae8-8102-3639e03c30fd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:28.644181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3353bdd6b0b48c8baff5752e19f1d27', 'request_id': 'req-40f721a9-f213-4ae8-8102-3639e03c30fd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:28.650559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3353bdd6b0b48c8baff5752e19f1d27', 'request_id': 'req-40f721a9-f213-4ae8-8102-3639e03c30fd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:28.662022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] The token cache is being invalidate because project e3353bdd6b0b48c8baff5752e19f1d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:28.662307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:28.662976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:28.664488 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 839/1662] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:28 2022] DELETE /identity/v3/projects/e3353bdd6b0b48c8baff5752e19f1d27 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:28.665247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.667670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.693840 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 824/1663] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:28.693840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.693840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.745431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f721a9-f213-4ae8-8102-3639e03c30fd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.751033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e61ffc-bab5-4752-bd24-1d4df07e18b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.751673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61ffc-bab5-4752-bd24-1d4df07e18b1 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.751952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61ffc-bab5-4752-bd24-1d4df07e18b1 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.752218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61ffc-bab5-4752-bd24-1d4df07e18b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.754014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a1e61ffc-bab5-4752-bd24-1d4df07e18b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.761386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b90e1-2448-41b8-a28e-d58f3f4ee636 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.809490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.815258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e61ffc-bab5-4752-bd24-1d4df07e18b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.816876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e61ffc-bab5-4752-bd24-1d4df07e18b1 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.825838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f0801-b72b-45b6-b6e7-25df8d52a5f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.826410 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 840/1664] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:28.826503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0801-b72b-45b6-b6e7-25df8d52a5f3 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.826801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0801-b72b-45b6-b6e7-25df8d52a5f3 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.827079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f0801-b72b-45b6-b6e7-25df8d52a5f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.827668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98015ac-87c9-4cda-a255-817a0c60f63a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.827904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98015ac-87c9-4cda-a255-817a0c60f63a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.828172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98015ac-87c9-4cda-a255-817a0c60f63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.829215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-166f0801-b72b-45b6-b6e7-25df8d52a5f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.840990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f0801-b72b-45b6-b6e7-25df8d52a5f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.842625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f0801-b72b-45b6-b6e7-25df8d52a5f3 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.853445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 825/1665] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:28.865788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b877fdd-1aac-4eba-96a2-03048966c1ad None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.865788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b877fdd-1aac-4eba-96a2-03048966c1ad None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.865788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b877fdd-1aac-4eba-96a2-03048966c1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.866421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e98015ac-87c9-4cda-a255-817a0c60f63a None None] MFA Rules not processed for user `62d984d1cf7c41f783ddbe05bc545e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.867634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e98015ac-87c9-4cda-a255-817a0c60f63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.894376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.898873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3b877fdd-1aac-4eba-96a2-03048966c1ad None None] MFA Rules not processed for user `764d9954a6ef48c0bb28d6dc38603265`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.900317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3b877fdd-1aac-4eba-96a2-03048966c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.960299 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 841/1666] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:28.961534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.961881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.962221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.982657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.983855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.990885 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 826/1667] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:28.990885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b877fdd-1aac-4eba-96a2-03048966c1ad None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.005451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b877fdd-1aac-4eba-96a2-03048966c1ad None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.011905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881bc50-79ba-4ae1-9a35-2f989d5c85cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.012711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881bc50-79ba-4ae1-9a35-2f989d5c85cd service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.013059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881bc50-79ba-4ae1-9a35-2f989d5c85cd service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.013534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881bc50-79ba-4ae1-9a35-2f989d5c85cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.015408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3881bc50-79ba-4ae1-9a35-2f989d5c85cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.046621 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 842/1668] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:29.047457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.048806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.060446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2996440-4f6c-43cb-9069-1b584264f28d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.065352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051dec0-aa2f-4058-8c98-03fe263e4b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.066085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dec0-aa2f-4058-8c98-03fe263e4b1d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.066361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dec0-aa2f-4058-8c98-03fe263e4b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.066625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051dec0-aa2f-4058-8c98-03fe263e4b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.067252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051dec0-aa2f-4058-8c98-03fe263e4b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.069170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051dec0-aa2f-4058-8c98-03fe263e4b1d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.082494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881bc50-79ba-4ae1-9a35-2f989d5c85cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.084035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881bc50-79ba-4ae1-9a35-2f989d5c85cd service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.094363 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 827/1669] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:29.095606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02a53-94b7-47d2-8b46-ce3b4b4a439a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.095891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02a53-94b7-47d2-8b46-ce3b4b4a439a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.096166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02a53-94b7-47d2-8b46-ce3b4b4a439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.101780 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 843/1670] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/projects => generated 395 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:29.101780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d5699-5a60-4897-a4f8-084069781add None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.101780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d5699-5a60-4897-a4f8-084069781add None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.101780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d5699-5a60-4897-a4f8-084069781add None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.101780 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 844/1671] 10.222.0.26 () {54 vars in 965 bytes} [Sat Sep 17 20:50:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:29.102783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ce41-9a5d-4d44-97df-e5dff9cec2b3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.103052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ce41-9a5d-4d44-97df-e5dff9cec2b3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.103316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ce41-9a5d-4d44-97df-e5dff9cec2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.118480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-53d02a53-94b7-47d2-8b46-ce3b4b4a439a None None] MFA Rules not processed for user `f05aab669547487d8876a70e338d3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:29.119684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-53d02a53-94b7-47d2-8b46-ce3b4b4a439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.125656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2d13ce41-9a5d-4d44-97df-e5dff9cec2b3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:29.126838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d13ce41-9a5d-4d44-97df-e5dff9cec2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.184696 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 828/1672] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:29.185479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02a53-94b7-47d2-8b46-ce3b4b4a439a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.190793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.198965 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 845/1673] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:29.201537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae229b-e27f-4b5b-933c-f99e7ab6de44 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.202106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae229b-e27f-4b5b-933c-f99e7ab6de44 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.202839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae229b-e27f-4b5b-933c-f99e7ab6de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.227066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-41ae229b-e27f-4b5b-933c-f99e7ab6de44 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:29.228326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-41ae229b-e27f-4b5b-933c-f99e7ab6de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.264874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02a53-94b7-47d2-8b46-ce3b4b4a439a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.269386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.270235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.270457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.270837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.271482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.271599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.273443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.296147 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 829/1674] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:29.297048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.306690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450c01d-e4f5-48d7-bf9b-6df8be466a1e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.311057 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 846/1675] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:29.311541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.311775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae229b-e27f-4b5b-933c-f99e7ab6de44 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.312136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.312417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.312691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.313402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:29.313740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.315331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.322589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae229b-e27f-4b5b-933c-f99e7ab6de44 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.328188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.328871 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 830/1676] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:29.328986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.329074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.329317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.329891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.331094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.339720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65e9b0-d19b-4c75-bb75-a843c57b353a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.344296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.344949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.345319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.345614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.346247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.347870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.357748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 831/1677] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:29.358545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.370487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faac0e-7b05-4156-9f27-d20f9ea4bbd2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.376372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.377033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.377385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.377816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.391525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.393262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.402678 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 847/1678] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:29.404208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.417074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509a03e-ea5f-432a-9e3a-fc7e202922c8 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.420208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.420819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.421096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.421621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.422285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.424309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.450669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.452275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.457811 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 848/1679] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:29.457811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.463093 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 832/1680] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:29.463967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.465598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.481592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327939b6-0fbc-4953-9832-b7d4b0bec053 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.531788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509532f7-8e19-420f-af6f-32297eae68a9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.536528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.537353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.537780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.538145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.538856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.540540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.553506 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 849/1681] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:29.553506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.555308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.556059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.556395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.556758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] PATH_INFO: `/v3/users/8c6db72795fc47bd8c820ac966cbd731` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.558343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6db72795fc47bd8c820ac966cbd731)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.561191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.562592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295aeed-5c81-4b01-b8d5-b5d6d4a0dbef admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.566376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.581400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.582303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.582785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.583183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.583788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.585980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.620554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 850/1682] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/users => generated 288 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:29.620554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.627194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6db72795fc47bd8c820ac966cbd731'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:29.628270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6db72795fc47bd8c820ac966cbd731'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:29.639574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6db72795fc47bd8c820ac966cbd731'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:29.640235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6db72795fc47bd8c820ac966cbd731'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:29.657001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.657488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6db72795fc47bd8c820ac966cbd731'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:29.683503 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 833/1683] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:29 2022] DELETE /identity/v3/users/8c6db72795fc47bd8c820ac966cbd731 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:29.687014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.688361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.721831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fc6b1-c48d-4d76-8f2c-803fec4b7f00 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.780605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bbeaf-cd96-44b7-8928-7a439814afe9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.785111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.785940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.786216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.786483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] PATH_INFO: `/v3/projects/ceac6247852c48a99cc782fb135c6a28/users/755c745bca614903bd8958d18e98afc0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.804055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceac6247852c48a99cc782fb135c6a28, user_id=755c745bca614903bd8958d18e98afc0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.806947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.811421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.812591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.812932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.813381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.815432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.826340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 834/1684] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:29 2022] PUT /identity/v3/projects/ceac6247852c48a99cc782fb135c6a28/users/755c745bca614903bd8958d18e98afc0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:29.837056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.839185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.881585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.883204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.893188 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 851/1685] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:29.898881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.900654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.919663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e356c4-d30c-4bde-bf4c-68d8be755ef0 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.925151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.926002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.926546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.926959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] PATH_INFO: `/v3/users/a6f71b6b15c442e2b683e337da7a35fc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.928702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f71b6b15c442e2b683e337da7a35fc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.928925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31ebba-834f-4084-b49b-077eef4e70eb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.930240 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:50:29.930240 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 20:50:29.930240 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 20:50:29.931701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.934810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983337f6-b626-4a66-8521-a6154f81764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.936122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983337f6-b626-4a66-8521-a6154f81764d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.936985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983337f6-b626-4a66-8521-a6154f81764d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.936985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983337f6-b626-4a66-8521-a6154f81764d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.938500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983337f6-b626-4a66-8521-a6154f81764d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.941590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983337f6-b626-4a66-8521-a6154f81764d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.962581 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 835/1686] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:29.962581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.962581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.962581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.968682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.982798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:29.985087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.997405 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 836/1687] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:29.997405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.998726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.013071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e62fc-91e7-4c11-8b15-980a84366ac6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.018644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.019387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.019726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.020005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] PATH_INFO: `/v3/users/8c17fdfd9c954b6b8f1e89fc663f1a5e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.021293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c17fdfd9c954b6b8f1e89fc663f1a5e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.023479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.036873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f71b6b15c442e2b683e337da7a35fc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.045093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.059491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f71b6b15c442e2b683e337da7a35fc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.067308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f71b6b15c442e2b683e337da7a35fc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.068081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f71b6b15c442e2b683e337da7a35fc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.068835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-08fcab9b-5ed4-41cf-ba26-c43b4cbde3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f71b6b15c442e2b683e337da7a35fc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.079169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c17fdfd9c954b6b8f1e89fc663f1a5e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.079878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c17fdfd9c954b6b8f1e89fc663f1a5e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.089167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c17fdfd9c954b6b8f1e89fc663f1a5e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.089885 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 852/1688] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:29 2022] DELETE /identity/v3/users/a6f71b6b15c442e2b683e337da7a35fc => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:30.090653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd6fb0-33b7-4675-8c75-8b5be2a6052d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.090940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd6fb0-33b7-4675-8c75-8b5be2a6052d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.091213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd6fb0-33b7-4675-8c75-8b5be2a6052d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.093027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c17fdfd9c954b6b8f1e89fc663f1a5e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.109647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c17fdfd9c954b6b8f1e89fc663f1a5e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:30.120248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8bdd6fb0-33b7-4675-8c75-8b5be2a6052d None None] MFA Rules not processed for user `755c745bca614903bd8958d18e98afc0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:30.121574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8bdd6fb0-33b7-4675-8c75-8b5be2a6052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.124496 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 837/1689] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:29 2022] DELETE /identity/v3/users/8c17fdfd9c954b6b8f1e89fc663f1a5e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:30.127870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.128959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.199443 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 853/1690] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:30.201033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec23a6f-cec6-4066-b312-dae56de06c20 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.201642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec23a6f-cec6-4066-b312-dae56de06c20 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.202586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec23a6f-cec6-4066-b312-dae56de06c20 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.204532 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 854/1691] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:30.205570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec23a6f-cec6-4066-b312-dae56de06c20 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.206081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df615-2fe1-4c07-9bc1-291aa41c18de admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.207731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eec23a6f-cec6-4066-b312-dae56de06c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.211065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94470127-4d29-4707-bf5f-d418ca7e9973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.211745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94470127-4d29-4707-bf5f-d418ca7e9973 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.212027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94470127-4d29-4707-bf5f-d418ca7e9973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.212290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94470127-4d29-4707-bf5f-d418ca7e9973 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.212986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94470127-4d29-4707-bf5f-d418ca7e9973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.214656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94470127-4d29-4707-bf5f-d418ca7e9973 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.225485 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 838/1692] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:30.226590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363c02-f9aa-459e-93af-51db539a9878 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.226871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363c02-f9aa-459e-93af-51db539a9878 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.227140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363c02-f9aa-459e-93af-51db539a9878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.245790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-49363c02-f9aa-459e-93af-51db539a9878 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:30.247055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49363c02-f9aa-459e-93af-51db539a9878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.254580 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 839/1693] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:30.255754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.256032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.256302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.275671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec23a6f-cec6-4066-b312-dae56de06c20 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.276209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:30.276851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.279960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2ee96-3f6e-4112-bd30-87c2b5b472ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.280665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ee96-3f6e-4112-bd30-87c2b5b472ef service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.281002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ee96-3f6e-4112-bd30-87c2b5b472ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.281357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2ee96-3f6e-4112-bd30-87c2b5b472ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.283211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ebf2ee96-3f6e-4112-bd30-87c2b5b472ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.285027 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 840/1694] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:30.285922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.287321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.296703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2ee96-3f6e-4112-bd30-87c2b5b472ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.298385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2ee96-3f6e-4112-bd30-87c2b5b472ef service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.309856 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 855/1695] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:30.311204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38ca4b-d3dd-483b-a405-0cf0fc086475 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.311342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38ca4b-d3dd-483b-a405-0cf0fc086475 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.311512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a38ca4b-d3dd-483b-a405-0cf0fc086475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.332938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6a38ca4b-d3dd-483b-a405-0cf0fc086475 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:30.334198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6a38ca4b-d3dd-483b-a405-0cf0fc086475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.348778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.396238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245f1f2-fe94-49ab-99d4-27213bf046d9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.409863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.413086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.414145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.417866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.418565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.421484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.433234 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 856/1696] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:30.435175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38ca4b-d3dd-483b-a405-0cf0fc086475 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.447333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a38ca4b-d3dd-483b-a405-0cf0fc086475 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.453427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.454215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.454497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.454786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] PATH_INFO: `/v3/projects/07ce9fd746374b478a743261eb014e5c/users/226a87a1f393420b8d19ec2390e9d063/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.474551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ce9fd746374b478a743261eb014e5c, user_id=226a87a1f393420b8d19ec2390e9d063, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.478346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.498214 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 857/1697] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:30 2022] PUT /identity/v3/projects/07ce9fd746374b478a743261eb014e5c/users/226a87a1f393420b8d19ec2390e9d063/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:30.499827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.501497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.514336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.516110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.527183 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 841/1698] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:30.527992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.529800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.541837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7797d0-6410-4d7d-ad0f-6057e6720665 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.575777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e50034-3df1-4cef-9e22-2559e282cfed admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.582894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.583272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.583561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.583823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.601765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c565e-323c-4edb-949c-f92e2323851d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.602521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c565e-323c-4edb-949c-f92e2323851d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.602903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c565e-323c-4edb-949c-f92e2323851d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.603368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c565e-323c-4edb-949c-f92e2323851d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.660032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.662029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.676288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c565e-323c-4edb-949c-f92e2323851d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.678032 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 858/1699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:30.678154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c565e-323c-4edb-949c-f92e2323851d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.678904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.680200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.697313 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 842/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:30.698990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f538b41-e1bd-43b2-b394-52f1e9901673 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.699356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f538b41-e1bd-43b2-b394-52f1e9901673 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.699687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f538b41-e1bd-43b2-b394-52f1e9901673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.734785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3f538b41-e1bd-43b2-b394-52f1e9901673 None None] MFA Rules not processed for user `226a87a1f393420b8d19ec2390e9d063`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:30.736569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3f538b41-e1bd-43b2-b394-52f1e9901673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.763921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.776212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84eee9b-5614-4638-9906-19db9b64a58e service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.780842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.781636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.781946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.782222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.797265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.837434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 843/1701] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:30.838394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f538b41-e1bd-43b2-b394-52f1e9901673 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.865931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.867965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.889810 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 859/1702] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:30.891321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.894984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.939729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f538b41-e1bd-43b2-b394-52f1e9901673 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.945604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.946425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.946784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.947196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.949155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:30.952436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576b195-f4b7-4dd7-a75c-b6305600091b service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.967314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.968354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.968730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.969355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] PATH_INFO: `/v3/projects/332a658e83ff48de8b6c50ff0a113463` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.971901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=332a658e83ff48de8b6c50ff0a113463)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.974944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.025982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.029199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.043077 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 844/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:31.044108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.049878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332a658e83ff48de8b6c50ff0a113463', 'request_id': 'req-2e4f0c97-052b-483c-939c-7215c4f22d2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.056300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2cef8-2505-4ebb-bae9-468e2c0a26a5 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.060678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332a658e83ff48de8b6c50ff0a113463', 'request_id': 'req-2e4f0c97-052b-483c-939c-7215c4f22d2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.070720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332a658e83ff48de8b6c50ff0a113463', 'request_id': 'req-2e4f0c97-052b-483c-939c-7215c4f22d2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.088958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] The token cache is being invalidate because project 332a658e83ff48de8b6c50ff0a113463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:31.089543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.090780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.092837 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 860/1704] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:30 2022] DELETE /identity/v3/projects/332a658e83ff48de8b6c50ff0a113463 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:31.097644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.099045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.149055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05fbea-b1d6-4275-9075-62733b299da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.150137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fbea-b1d6-4275-9075-62733b299da5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.150137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fbea-b1d6-4275-9075-62733b299da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.150341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fbea-b1d6-4275-9075-62733b299da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.150868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05fbea-b1d6-4275-9075-62733b299da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.155235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05fbea-b1d6-4275-9075-62733b299da5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.162258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.192166 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 845/1705] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:31 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:31.193638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc04e2-d83b-4174-bf11-7059da0b4fc3 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.197222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc04e2-d83b-4174-bf11-7059da0b4fc3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.197222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc04e2-d83b-4174-bf11-7059da0b4fc3 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.197222 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 846/1706] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:31.197222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc04e2-d83b-4174-bf11-7059da0b4fc3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.200218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c7cc04e2-d83b-4174-bf11-7059da0b4fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.213005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f0c97-052b-483c-939c-7215c4f22d2d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.247479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.294414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.295985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.296435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.297103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] PATH_INFO: `/v3/users/764d9954a6ef48c0bb28d6dc38603265` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.298468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] RBAC: Authorizing `identity:delete_user(user_id=764d9954a6ef48c0bb28d6dc38603265)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.300394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc04e2-d83b-4174-bf11-7059da0b4fc3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.301018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.305854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.306991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.306991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.307201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.307740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.309742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.346856 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 847/1707] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:31 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:31.346856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:31.348678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.368724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764d9954a6ef48c0bb28d6dc38603265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.382530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764d9954a6ef48c0bb28d6dc38603265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.396047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764d9954a6ef48c0bb28d6dc38603265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.396750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764d9954a6ef48c0bb28d6dc38603265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.397453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764d9954a6ef48c0bb28d6dc38603265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.417522 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 861/1708] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:31 2022] DELETE /identity/v3/users/764d9954a6ef48c0bb28d6dc38603265 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:31.417522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.418453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.500892 ncdv-master-1 devstack@keystone.service[99167]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 17 20:50:31.501414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.502828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.540367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2013f-103c-4175-ae0b-e5670a82d95c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.546874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.547772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.548127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.548404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] PATH_INFO: `/v3/projects/e5375b35e2884395beebeb16434f4f16` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.549761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5375b35e2884395beebeb16434f4f16)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.551977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.584379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2123f-fed5-43cd-a984-6cad28084d70 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.603514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.617807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5375b35e2884395beebeb16434f4f16', 'request_id': 'req-e48f18ee-f7a1-4183-a754-67326ca658e8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.622115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.647392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5375b35e2884395beebeb16434f4f16', 'request_id': 'req-e48f18ee-f7a1-4183-a754-67326ca658e8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.656730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5375b35e2884395beebeb16434f4f16', 'request_id': 'req-e48f18ee-f7a1-4183-a754-67326ca658e8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.678844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62d984d1cf7c41f783ddbe05bc545e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0af2d58b8994eeeb64cec26101d4816', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.679719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.680606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.680606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] PATH_INFO: `/v3/projects/0a55519ffc224c849e9be490acb4596e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.681984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorizing `identity:get_project(project_id=0a55519ffc224c849e9be490acb4596e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.682820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] The token cache is being invalidate because project e5375b35e2884395beebeb16434f4f16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:31.683198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.684049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.684937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.686188 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 862/1709] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:31 2022] DELETE /identity/v3/projects/e5375b35e2884395beebeb16434f4f16 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:31.687069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.693216 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 848/1710] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:31 2022] GET /identity/v3/projects/0a55519ffc224c849e9be490acb4596e => generated 395 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:31.693216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.695815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.706514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.714118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b27f4-c0c7-445c-8790-9ce01effab21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.797877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc2cea-20fc-47b2-b5b2-f38a2cb7602d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.798716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc2cea-20fc-47b2-b5b2-f38a2cb7602d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.799178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc2cea-20fc-47b2-b5b2-f38a2cb7602d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.799621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc2cea-20fc-47b2-b5b2-f38a2cb7602d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.800455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc2cea-20fc-47b2-b5b2-f38a2cb7602d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.802740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc2cea-20fc-47b2-b5b2-f38a2cb7602d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.814368 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 849/1711] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:31.815680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f22cf-f55a-4e3e-a19a-e71a9ee04f80 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.816179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f22cf-f55a-4e3e-a19a-e71a9ee04f80 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.816563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f22cf-f55a-4e3e-a19a-e71a9ee04f80 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.818142 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 850/1712] 10.222.0.26 () {54 vars in 961 bytes} [Sat Sep 17 20:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:31.818868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f22cf-f55a-4e3e-a19a-e71a9ee04f80 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.821493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8c9f22cf-f55a-4e3e-a19a-e71a9ee04f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.823208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f18ee-f7a1-4183-a754-67326ca658e8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.834168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.834795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.835083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.835365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] PATH_INFO: `/v3/projects/4e1c57a31bd8425f927f361aaeffec9e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.836898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1c57a31bd8425f927f361aaeffec9e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.838775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f22cf-f55a-4e3e-a19a-e71a9ee04f80 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.839283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.845035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.845976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.846548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.848869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] PATH_INFO: `/v3/users/d93585c867aa484b82b32a8bd4c00038` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.848869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93585c867aa484b82b32a8bd4c00038)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.850198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.911729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1c57a31bd8425f927f361aaeffec9e', 'request_id': 'req-b29f4512-9730-46bf-91da-e56e91608fcf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.922757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93585c867aa484b82b32a8bd4c00038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.923584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93585c867aa484b82b32a8bd4c00038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.927060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1c57a31bd8425f927f361aaeffec9e', 'request_id': 'req-b29f4512-9730-46bf-91da-e56e91608fcf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.932676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93585c867aa484b82b32a8bd4c00038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.933409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93585c867aa484b82b32a8bd4c00038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.934347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1c57a31bd8425f927f361aaeffec9e', 'request_id': 'req-b29f4512-9730-46bf-91da-e56e91608fcf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.948019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] The token cache is being invalidate because project 4e1c57a31bd8425f927f361aaeffec9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:31.948318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.949010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b29f4512-9730-46bf-91da-e56e91608fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.949446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0471e827-4233-42e6-b1c3-5d51cbca8563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93585c867aa484b82b32a8bd4c00038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:31.951070 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 863/1713] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:31 2022] DELETE /identity/v3/projects/4e1c57a31bd8425f927f361aaeffec9e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:31.953356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4438e3-5cde-4aff-8202-250408dae6d0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.954079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4438e3-5cde-4aff-8202-250408dae6d0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.954429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4438e3-5cde-4aff-8202-250408dae6d0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.955924 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 864/1714] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:31.956631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4438e3-5cde-4aff-8202-250408dae6d0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.958951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5c4438e3-5cde-4aff-8202-250408dae6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:31.966810 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 851/1715] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:31 2022] DELETE /identity/v3/users/d93585c867aa484b82b32a8bd4c00038 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:31.966810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.966810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.966810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.991512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:31.994824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.023164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:32.052128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4438e3-5cde-4aff-8202-250408dae6d0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.058675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.059557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.061237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.061701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] PATH_INFO: `/v3/projects/cbe06cc27b4f403faac35ee3521893ea/users/27c681039283443c8dd284f22277bc8f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.080041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe06cc27b4f403faac35ee3521893ea, user_id=27c681039283443c8dd284f22277bc8f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.083198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.085781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:32.088511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 852/1716] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Sep 17 20:50:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:32.089448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:32.090984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.104938 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 865/1717] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:31 2022] PUT /identity/v3/projects/cbe06cc27b4f403faac35ee3521893ea/users/27c681039283443c8dd284f22277bc8f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:32.105988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930317c1-c5b1-4ae7-8cbe-eae18f610777 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.106430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:32.108207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.195924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.198913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.199505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.200045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] PATH_INFO: `/v3/projects/504f9c5f7884489084c9a00bf846ed98` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.201480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] RBAC: Authorizing `identity:delete_project(project_id=504f9c5f7884489084c9a00bf846ed98)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.202837 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:50:32.202837 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 20:50:32.202837 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 20:50:32.204713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.211959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56c86-bbbe-4ea4-a86a-0d2c16b18699 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.217977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.218627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.218902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.219166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] PATH_INFO: `/v3/users/525e8cc77ab744f5837b3912e9d754a3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.220641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525e8cc77ab744f5837b3912e9d754a3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.222649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.287045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504f9c5f7884489084c9a00bf846ed98', 'request_id': 'req-363f920c-a531-4db0-b0be-19957fde085a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.289666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525e8cc77ab744f5837b3912e9d754a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.303355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504f9c5f7884489084c9a00bf846ed98', 'request_id': 'req-363f920c-a531-4db0-b0be-19957fde085a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.313717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525e8cc77ab744f5837b3912e9d754a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.326404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504f9c5f7884489084c9a00bf846ed98', 'request_id': 'req-363f920c-a531-4db0-b0be-19957fde085a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.334774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525e8cc77ab744f5837b3912e9d754a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.335825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525e8cc77ab744f5837b3912e9d754a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.336756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525e8cc77ab744f5837b3912e9d754a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.342851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] The token cache is being invalidate because project 504f9c5f7884489084c9a00bf846ed98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:32.343398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.345141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.349567 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 853/1718] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:32 2022] DELETE /identity/v3/projects/504f9c5f7884489084c9a00bf846ed98 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:32.350860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:32.352484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.366376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 866/1719] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:32 2022] DELETE /identity/v3/users/525e8cc77ab744f5837b3912e9d754a3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:32.374687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:32.376225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.429964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:32.462316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:32.463415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.467306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade143-ef47-4d4c-8020-02cc496a9ac9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.473260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.474479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.475015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.475261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.477761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.504930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.504930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.508549 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 867/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:32.511275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:32.512831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.519432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:32.556796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f920c-a531-4db0-b0be-19957fde085a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.622494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76f1be-3408-4775-bb9e-200d59b6cc98 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.622746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62d984d1cf7c41f783ddbe05bc545e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0af2d58b8994eeeb64cec26101d4816', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.623413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.623665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.623934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] PATH_INFO: `/v3/projects/0a55519ffc224c849e9be490acb4596e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.625071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorizing `identity:get_project(project_id=0a55519ffc224c849e9be490acb4596e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.628273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.631079 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 854/1721] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:32 2022] GET /identity/v3/projects/0a55519ffc224c849e9be490acb4596e => generated 395 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:32.631939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:32.634060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.662247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fef99-c25f-4e8e-84f5-3388bf830569 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.664934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6fb38-61ff-4d95-9e69-2918be501691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.667161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6fb38-61ff-4d95-9e69-2918be501691 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.667348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6fb38-61ff-4d95-9e69-2918be501691 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.667581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6fb38-61ff-4d95-9e69-2918be501691 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.668339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6fb38-61ff-4d95-9e69-2918be501691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.670025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6fb38-61ff-4d95-9e69-2918be501691 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.677273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.677545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.677627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.678244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] PATH_INFO: `/v3/projects/49303d354e4442f9aaea429c760892db` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.679924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] RBAC: Authorizing `identity:delete_project(project_id=49303d354e4442f9aaea429c760892db)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.684537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.700598 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 868/1722] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:32.704141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907576f-a7aa-4650-b8f3-837dabcef7db None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.704141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907576f-a7aa-4650-b8f3-837dabcef7db None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.704141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907576f-a7aa-4650-b8f3-837dabcef7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.742876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9907576f-a7aa-4650-b8f3-837dabcef7db None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:32.744118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9907576f-a7aa-4650-b8f3-837dabcef7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.757280 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 869/1723] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:32.758223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907576f-a7aa-4650-b8f3-837dabcef7db None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:32.783454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907576f-a7aa-4650-b8f3-837dabcef7db None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:32.801686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49303d354e4442f9aaea429c760892db', 'request_id': 'req-5536e88d-19fc-4ad7-ac1d-1b78a878768e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.809523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49303d354e4442f9aaea429c760892db', 'request_id': 'req-5536e88d-19fc-4ad7-ac1d-1b78a878768e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.821939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49303d354e4442f9aaea429c760892db', 'request_id': 'req-5536e88d-19fc-4ad7-ac1d-1b78a878768e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.850688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] The token cache is being invalidate because project 49303d354e4442f9aaea429c760892db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:32.850688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.851723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5536e88d-19fc-4ad7-ac1d-1b78a878768e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:32.854236 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 855/1724] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:32 2022] DELETE /identity/v3/projects/49303d354e4442f9aaea429c760892db => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:32.855869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.856160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.856773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.880747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861ec69-9cc0-493e-ad97-4372b5d91aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:32.881560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ec69-9cc0-493e-ad97-4372b5d91aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.881900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ec69-9cc0-493e-ad97-4372b5d91aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.882251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ec69-9cc0-493e-ad97-4372b5d91aa9 admin admin] PATH_INFO: `/v3/projects/cbe06cc27b4f403faac35ee3521893ea/users/27c681039283443c8dd284f22277bc8f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.884537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861ec69-9cc0-493e-ad97-4372b5d91aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe06cc27b4f403faac35ee3521893ea, user_id=27c681039283443c8dd284f22277bc8f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:32.887658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861ec69-9cc0-493e-ad97-4372b5d91aa9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:32.900205 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 870/1725] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:32 2022] PUT /identity/v3/projects/cbe06cc27b4f403faac35ee3521893ea/users/27c681039283443c8dd284f22277bc8f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:32.900885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:32.901686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76d05-7eb7-4ced-80bf-33df695f5621 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.902135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76d05-7eb7-4ced-80bf-33df695f5621 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.902285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.902603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76d05-7eb7-4ced-80bf-33df695f5621 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.904499 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 871/1726] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:32.905418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ae60-77f9-4f7e-ab00-b751a5f7fa54 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.905783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ae60-77f9-4f7e-ab00-b751a5f7fa54 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.906115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4ae60-77f9-4f7e-ab00-b751a5f7fa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.930990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:32.940847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-42d4ae60-77f9-4f7e-ab00-b751a5f7fa54 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:32.942267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-42d4ae60-77f9-4f7e-ab00-b751a5f7fa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.997961 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 856/1727] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:32.999066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.000928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.015327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef9411-6989-4680-a74b-4f5fad86ef30 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.017867 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 872/1728] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:33.019703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.020586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.020863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.027844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.028950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.029227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.029670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] PATH_INFO: `/v3/projects/a6b28cee0cb44e5d8e52af9bf22e7e11` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.031785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b28cee0cb44e5d8e52af9bf22e7e11)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.033444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.057730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] MFA Rules not processed for user `27c681039283443c8dd284f22277bc8f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:33.059037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.073539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:33.092338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b28cee0cb44e5d8e52af9bf22e7e11', 'request_id': 'req-8fff804a-49b8-47d6-a7b9-3539c4f1f283'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.099909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b28cee0cb44e5d8e52af9bf22e7e11', 'request_id': 'req-8fff804a-49b8-47d6-a7b9-3539c4f1f283'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.109545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b28cee0cb44e5d8e52af9bf22e7e11', 'request_id': 'req-8fff804a-49b8-47d6-a7b9-3539c4f1f283'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.121321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] The token cache is being invalidate because project a6b28cee0cb44e5d8e52af9bf22e7e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:33.121745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.122478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.124331 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 857/1729] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:32 2022] DELETE /identity/v3/projects/a6b28cee0cb44e5d8e52af9bf22e7e11 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:33.125229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:33.126797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.139068 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 873/1730] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:33.141525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.150269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.166734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a407416-9d7b-4789-b721-97755b58bf4b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.209039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.210604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.234320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.234971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.235267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.235527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] PATH_INFO: `/v3/users/65c1fa6cebe54ac2898f8a615802b4ea` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.236978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c1fa6cebe54ac2898f8a615802b4ea)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.239103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.259532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:33.292751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff804a-49b8-47d6-a7b9-3539c4f1f283 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.298330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62d984d1cf7c41f783ddbe05bc545e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0af2d58b8994eeeb64cec26101d4816', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.298970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.299254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.299530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] PATH_INFO: `/v3/projects/0a55519ffc224c849e9be490acb4596e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.300176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c1fa6cebe54ac2898f8a615802b4ea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.301016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorizing `identity:get_project(project_id=0a55519ffc224c849e9be490acb4596e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.302834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.304740 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 858/1731] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/projects/0a55519ffc224c849e9be490acb4596e => generated 395 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:33.305298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.307075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.316256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c1fa6cebe54ac2898f8a615802b4ea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.325152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4df00-98ed-49a0-bd45-7d9223113e21 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.328077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c1fa6cebe54ac2898f8a615802b4ea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.328759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c1fa6cebe54ac2898f8a615802b4ea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.329521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4a9ac30-d0c6-4edd-b5e8-32076e206b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c1fa6cebe54ac2898f8a615802b4ea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:33.333884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3df07e-37cd-405a-8142-3ca8402535cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.334509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df07e-37cd-405a-8142-3ca8402535cf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.335085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df07e-37cd-405a-8142-3ca8402535cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.335446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df07e-37cd-405a-8142-3ca8402535cf admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.336112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3df07e-37cd-405a-8142-3ca8402535cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:33.336409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3df07e-37cd-405a-8142-3ca8402535cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.338392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3df07e-37cd-405a-8142-3ca8402535cf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.348277 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 874/1732] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:33 2022] DELETE /identity/v3/users/65c1fa6cebe54ac2898f8a615802b4ea => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:33.350170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83be5b-9da3-4e39-8a2f-7f4bfda86147 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.350468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83be5b-9da3-4e39-8a2f-7f4bfda86147 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.350763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83be5b-9da3-4e39-8a2f-7f4bfda86147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.356647 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 859/1733] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:33.358317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf839b-6be5-4c72-9254-cd5710f6973c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.358570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf839b-6be5-4c72-9254-cd5710f6973c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.358819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf839b-6be5-4c72-9254-cd5710f6973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.381395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7e83be5b-9da3-4e39-8a2f-7f4bfda86147 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:33.382978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7e83be5b-9da3-4e39-8a2f-7f4bfda86147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.388128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0bf839b-6be5-4c72-9254-cd5710f6973c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:33.389343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0bf839b-6be5-4c72-9254-cd5710f6973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.423916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:33.473566 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 875/1734] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:33.477574 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 860/1735] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:33.477723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988b95-a194-46e3-9122-bdddc2bbd36c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.477723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988b95-a194-46e3-9122-bdddc2bbd36c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.477723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988b95-a194-46e3-9122-bdddc2bbd36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.477934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.477934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.477934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.498154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:33.499398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.507957 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 861/1736] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:33.508933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.510211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.516543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ed988b95-a194-46e3-9122-bdddc2bbd36c None None] MFA Rules not processed for user `226a87a1f393420b8d19ec2390e9d063`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:33.518083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ed988b95-a194-46e3-9122-bdddc2bbd36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.524017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd2ebd-8b00-491a-9ce2-1919a08bb6f6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.532931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.533728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.534015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.534320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.535022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.536946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.587775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 862/1737] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:33.588729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.603679 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 876/1738] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:33.605500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e980-60eb-4dd5-ac04-f6630b8ce586 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.605804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e980-60eb-4dd5-ac04-f6630b8ce586 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.606260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8e980-60eb-4dd5-ac04-f6630b8ce586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.636822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-32f8e980-60eb-4dd5-ac04-f6630b8ce586 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:33.638116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-32f8e980-60eb-4dd5-ac04-f6630b8ce586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.668126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69453-ab19-4d65-bb78-eb523ba6cf21 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.673739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.675677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.676531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.676953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.677882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:33.678327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.680288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.695979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 863/1739] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:33.697049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.698991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.707958 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 877/1740] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:33.707958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8e980-60eb-4dd5-ac04-f6630b8ce586 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.717685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8e980-60eb-4dd5-ac04-f6630b8ce586 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.722370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.722985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.723291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.723569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.724210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.726188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.736396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:33.757615 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 878/1741] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:33.757615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.757615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.763943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dd687-863f-48db-94bd-cc008d55ec67 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.768601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.769308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.769574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.769793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.780928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.783135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.794843 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 864/1742] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:33.795683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.797782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.831049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:33.860411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da28fe-3624-4e5d-9738-e6e1bb2fc025 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.868927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.870284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.870428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.870674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.873419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.915246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bd41e-2ac6-4ba7-b88c-567cde56f403 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.924499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.925341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.925713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.926055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.929452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.974677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.977193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.989414 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 879/1743] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:33.991083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.002098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dfdf8-5da6-4f96-979e-40bb6fbe2858 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.004253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.005961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.006586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.007253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.007546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.007819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.008454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.010389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.016923 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 865/1744] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:34.017913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.035169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b4ad3-d76c-4970-9c97-d23996770290 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.046344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955f2a4-bf25-4102-95ab-5956d6862953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.047105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955f2a4-bf25-4102-95ab-5956d6862953 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.047405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955f2a4-bf25-4102-95ab-5956d6862953 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.048017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955f2a4-bf25-4102-95ab-5956d6862953 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.048692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955f2a4-bf25-4102-95ab-5956d6862953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:34.049049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955f2a4-bf25-4102-95ab-5956d6862953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.051987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955f2a4-bf25-4102-95ab-5956d6862953 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.066003 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 880/1745] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/projects => generated 411 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:34.067093 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 866/1746] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:34.067204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.068548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d64e4-be30-49fd-858c-d2a65cb53892 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.068839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d64e4-be30-49fd-858c-d2a65cb53892 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.069113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d64e4-be30-49fd-858c-d2a65cb53892 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.070800 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 867/1747] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:34.071570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d64e4-be30-49fd-858c-d2a65cb53892 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.149518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a97c1-1d9b-425b-b769-7efe8958375f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.155403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.156246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.156606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.157004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.157959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.159742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.168303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d64e4-be30-49fd-858c-d2a65cb53892 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.179914 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 881/1748] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:34.181138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.184915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3222a1-adb1-4bc1-ab61-fb2777c526e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.185943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3222a1-adb1-4bc1-ab61-fb2777c526e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.186423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3222a1-adb1-4bc1-ab61-fb2777c526e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.191021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3222a1-adb1-4bc1-ab61-fb2777c526e5 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.191784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3222a1-adb1-4bc1-ab61-fb2777c526e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.195488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3222a1-adb1-4bc1-ab61-fb2777c526e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.199127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b80f02-a6f5-4925-a94e-d0b75658cf99 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.205085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.205948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.206111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.206438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.206720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.207429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.209932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.236712 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 868/1749] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/users => generated 306 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:34.236712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e88ba-6f49-4f42-8fde-5e69b79f2293 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.236712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e88ba-6f49-4f42-8fde-5e69b79f2293 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.236712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e88ba-6f49-4f42-8fde-5e69b79f2293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.252234 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 882/1750] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:34.253305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.262307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-363e88ba-6f49-4f42-8fde-5e69b79f2293 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:34.263585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-363e88ba-6f49-4f42-8fde-5e69b79f2293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.316347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.334424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888188c5-33be-4a12-b60c-ac1978fff7fd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.341691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.342331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.342603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.342875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.343537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.345297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.356718 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 883/1751] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:34.357738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.359515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.374718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef0883-339f-4505-bbcf-8d19c08d5671 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.380722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.381279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.381760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.382006 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 869/1752] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:34.383189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e88ba-6f49-4f42-8fde-5e69b79f2293 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.385783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] PATH_INFO: `/v3/projects/84675c7d7632442ead7879973c5a87ed` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.388045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=84675c7d7632442ead7879973c5a87ed)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.391338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.405156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e88ba-6f49-4f42-8fde-5e69b79f2293 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.414310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.415119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.415958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.417109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.418309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.420129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.440915 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 870/1753] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:34.440915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.455697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3655df-72a4-4df0-8a70-32f672861323 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.466893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.467796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.468306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.468669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.469588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.471754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.492959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84675c7d7632442ead7879973c5a87ed', 'request_id': 'req-05542df0-58d3-4484-8ec8-af983a1987fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.509247 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 871/1754] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/users => generated 288 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:34.513480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84675c7d7632442ead7879973c5a87ed', 'request_id': 'req-05542df0-58d3-4484-8ec8-af983a1987fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.514379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.522210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84675c7d7632442ead7879973c5a87ed', 'request_id': 'req-05542df0-58d3-4484-8ec8-af983a1987fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.525344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ed4b-589b-4f34-bba6-825851448a60 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.537826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] The token cache is being invalidate because project 84675c7d7632442ead7879973c5a87ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:34.538256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.539057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.541165 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 884/1755] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:34 2022] DELETE /identity/v3/projects/84675c7d7632442ead7879973c5a87ed => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:34.542167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.544649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.602291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.602972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.603252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.603526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] PATH_INFO: `/v3/projects/ea88200545ae4003b42b0d475f482c9d/users/eb8bfe1d590a4500b7e7b5c79a6a4e6b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.611482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05542df0-58d3-4484-8ec8-af983a1987fb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.618114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.618447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.618718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.618985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.619086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea88200545ae4003b42b0d475f482c9d, user_id=eb8bfe1d590a4500b7e7b5c79a6a4e6b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.620723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.621899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.641149 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 872/1756] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:34 2022] PUT /identity/v3/projects/ea88200545ae4003b42b0d475f482c9d/users/eb8bfe1d590a4500b7e7b5c79a6a4e6b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:34.646523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.650341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.706008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.708928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.710420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.721188 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 885/1757] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:34.722032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.723296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.732633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98468974-b76f-42d2-add0-ba910336a0e6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.736651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3e2a-0f56-4192-a560-15f40c6ea49a service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.737588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8c25e-6c75-4ba6-9546-ac04a11e4bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.738204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8c25e-6c75-4ba6-9546-ac04a11e4bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.738485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8c25e-6c75-4ba6-9546-ac04a11e4bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.738786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8c25e-6c75-4ba6-9546-ac04a11e4bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.739402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8c25e-6c75-4ba6-9546-ac04a11e4bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.741201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8c25e-6c75-4ba6-9546-ac04a11e4bd8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.741740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2ae09-d1d2-4ebc-9bd0-590c1277af23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.742338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2ae09-d1d2-4ebc-9bd0-590c1277af23 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.742610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2ae09-d1d2-4ebc-9bd0-590c1277af23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.742885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2ae09-d1d2-4ebc-9bd0-590c1277af23 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.743510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2ae09-d1d2-4ebc-9bd0-590c1277af23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.745212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2ae09-d1d2-4ebc-9bd0-590c1277af23 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.747503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.753253 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 873/1758] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:34.754664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1cbcf-e2c8-4fcc-b242-0efa9c9f852c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.754978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1cbcf-e2c8-4fcc-b242-0efa9c9f852c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.755216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1cbcf-e2c8-4fcc-b242-0efa9c9f852c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.756737 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 874/1759] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:34.757575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1cbcf-e2c8-4fcc-b242-0efa9c9f852c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.759384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bba1cbcf-e2c8-4fcc-b242-0efa9c9f852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.763370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 886/1760] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:34.764813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0214-17f5-4090-981b-f5010b94edf5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.765339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0214-17f5-4090-981b-f5010b94edf5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.765707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0214-17f5-4090-981b-f5010b94edf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.779089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1cbcf-e2c8-4fcc-b242-0efa9c9f852c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.783540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.784178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.784451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.784718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] PATH_INFO: `/v3/users/3535c9a33ab544688ff3f35f9f139748` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.785881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3535c9a33ab544688ff3f35f9f139748)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.787798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.802303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-17ef0214-17f5-4090-981b-f5010b94edf5 None None] MFA Rules not processed for user `eb8bfe1d590a4500b7e7b5c79a6a4e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:34.804436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17ef0214-17f5-4090-981b-f5010b94edf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.861717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3535c9a33ab544688ff3f35f9f139748'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.863764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3535c9a33ab544688ff3f35f9f139748'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.876742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3535c9a33ab544688ff3f35f9f139748'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.877380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3535c9a33ab544688ff3f35f9f139748'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.894288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3535c9a33ab544688ff3f35f9f139748'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:34.895479 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 887/1761] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:34.897071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.897550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.898047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.914135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 875/1762] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:34 2022] DELETE /identity/v3/users/3535c9a33ab544688ff3f35f9f139748 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:34.915055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.916422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.977599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:34.978878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.002270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3f569-39e9-46c0-a75b-da564a406ed8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.010350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.012188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.012710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.013170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] PATH_INFO: `/v3/projects/7251e61bfc7e4fdcb7ede9dc59b61138/users/a9626c56a7af47e292e3c3b61e35cd7b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.037945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7251e61bfc7e4fdcb7ede9dc59b61138, user_id=a9626c56a7af47e292e3c3b61e35cd7b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.040978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.069243 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 876/1763] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:34 2022] PUT /identity/v3/projects/7251e61bfc7e4fdcb7ede9dc59b61138/users/a9626c56a7af47e292e3c3b61e35cd7b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:35.069243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:35.071163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.076935 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 888/1764] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:35.077959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.079834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.095906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88afaad-22d8-4f53-96ca-96719fcad0df None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.137144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.175300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.176026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.176248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.176523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.177527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.179006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.180716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.196107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 889/1765] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:35.196107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.197737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.212666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32874ddb-2300-41cd-b904-21a60f945b78 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.218095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21856456-ba33-4c78-92fc-6dc0d134ac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.218915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21856456-ba33-4c78-92fc-6dc0d134ac71 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.219401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21856456-ba33-4c78-92fc-6dc0d134ac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.219682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21856456-ba33-4c78-92fc-6dc0d134ac71 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.220348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21856456-ba33-4c78-92fc-6dc0d134ac71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.222469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21856456-ba33-4c78-92fc-6dc0d134ac71 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.242338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.247197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.286490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 890/1766] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:35 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:35.303090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f794ea0-655f-426d-9899-96f063ed0530 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.303569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f794ea0-655f-426d-9899-96f063ed0530 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.303692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f794ea0-655f-426d-9899-96f063ed0530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.350132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0f794ea0-655f-426d-9899-96f063ed0530 None None] MFA Rules not processed for user `a9626c56a7af47e292e3c3b61e35cd7b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:35.351466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0f794ea0-655f-426d-9899-96f063ed0530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.453659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc175a46-d914-492d-a1c7-6d8808ed1eed admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.472973 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 891/1767] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:35 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:35.476921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f794ea0-655f-426d-9899-96f063ed0530 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.578998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62d984d1cf7c41f783ddbe05bc545e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0af2d58b8994eeeb64cec26101d4816', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.580260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.580260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.580617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] PATH_INFO: `/v3/projects/0a55519ffc224c849e9be490acb4596e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.585994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorizing `identity:get_project(project_id=0a55519ffc224c849e9be490acb4596e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.588296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.590392 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 877/1768] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/projects/0a55519ffc224c849e9be490acb4596e => generated 395 bytes in 529 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:35.594048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.604232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.684734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f794ea0-655f-426d-9899-96f063ed0530 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.705316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.706004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.706259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.706806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] PATH_INFO: `/v3/projects/a1d03e8b59b7494a909f33fbee0cadfb/users/abbb938447164ec6ab8461731a9f3780/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.737339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d03e8b59b7494a909f33fbee0cadfb, user_id=abbb938447164ec6ab8461731a9f3780, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.740941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.761368 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 892/1769] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:35 2022] PUT /identity/v3/projects/a1d03e8b59b7494a909f33fbee0cadfb/users/abbb938447164ec6ab8461731a9f3780/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:35.762823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.802277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.833959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6292c-fabb-4855-be62-132433c0ba5d tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.853177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.885237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73817e-d511-4c78-be07-45f29f5c6670 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.898897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.899670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.900091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.900454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.901597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.904466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.941346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d41a13-8dd9-4ce0-9940-4f2a9b2cbc19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.942989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41a13-8dd9-4ce0-9940-4f2a9b2cbc19 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.942989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41a13-8dd9-4ce0-9940-4f2a9b2cbc19 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.942989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41a13-8dd9-4ce0-9940-4f2a9b2cbc19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.951011 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 893/1770] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:35 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:35.951011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.967836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36223f9b-1dae-4f38-b74d-9a9e0a1f92fe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.972654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.973265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.973690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.973961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.974692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.976621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.020942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d41a13-8dd9-4ce0-9940-4f2a9b2cbc19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.021656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 894/1771] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:35 2022] POST /identity/v3/projects => generated 409 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:36.022582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d41a13-8dd9-4ce0-9940-4f2a9b2cbc19 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.022965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.025081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:36.037304 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 878/1772] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:50:36.041034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13ddce-fc55-418f-827d-cc36603f750d None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.041600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13ddce-fc55-418f-827d-cc36603f750d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.041809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13ddce-fc55-418f-827d-cc36603f750d None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.048576 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 879/1773] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:36.050260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6961ce3-132f-43c4-8c1f-d67062c4fe30 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.050688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6961ce3-132f-43c4-8c1f-d67062c4fe30 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.051038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6961ce3-132f-43c4-8c1f-d67062c4fe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.096420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a6961ce3-132f-43c4-8c1f-d67062c4fe30 None None] MFA Rules not processed for user `abbb938447164ec6ab8461731a9f3780`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:36.097754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a6961ce3-132f-43c4-8c1f-d67062c4fe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:36.173511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 880/1774] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:36.178554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6961ce3-132f-43c4-8c1f-d67062c4fe30 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.192000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6961ce3-132f-43c4-8c1f-d67062c4fe30 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.195109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531029-65db-463b-b543-b0700a228b59 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.196844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.197932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.198366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.198723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.199526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.201908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.215810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.217253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.218302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.218932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] PATH_INFO: `/v3/projects/f5b644fe62414b3ab02fd4fa12e2a4e5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.221038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b644fe62414b3ab02fd4fa12e2a4e5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.224789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.231990 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 881/1775] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:36.234072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.261525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23fe4b-fd19-495f-8bfe-110e85565ece admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.277386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.278131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.279368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.279368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.279949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.282218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.309138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:36.331012 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 882/1776] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:36 2022] POST /identity/v3/users => generated 311 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:36.332074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b644fe62414b3ab02fd4fa12e2a4e5', 'request_id': 'req-799ee2e4-6aa6-46bf-a70d-58bb061a9871'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:36.332414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:36.350678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b644fe62414b3ab02fd4fa12e2a4e5', 'request_id': 'req-799ee2e4-6aa6-46bf-a70d-58bb061a9871'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:36.370564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b644fe62414b3ab02fd4fa12e2a4e5', 'request_id': 'req-799ee2e4-6aa6-46bf-a70d-58bb061a9871'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:36.411399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] The token cache is being invalidate because project f5b644fe62414b3ab02fd4fa12e2a4e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:36.411908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:36.413452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:36.415627 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 895/1777] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:36 2022] DELETE /identity/v3/projects/f5b644fe62414b3ab02fd4fa12e2a4e5 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:36.421618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.423225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:36.525973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:36.560925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.572883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ee2e4-6aa6-46bf-a70d-58bb061a9871 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.584944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.585505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.587170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.587584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.588590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.592065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.646024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 896/1778] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:36 2022] POST /identity/v3/projects => generated 373 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:36.647007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.655682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:36.755274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90149f2f-713e-45b0-bf4f-a0b6db2166d5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.786238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038444e-aa82-4661-a33e-9c132c6313cd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.794136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.794789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.795083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.795432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] PATH_INFO: `/v3/projects/ee8337c545374fb79e72672eb365bb58/users/2ce5fae031e34a228fd0375b92ba8d2b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.818335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8337c545374fb79e72672eb365bb58, user_id=2ce5fae031e34a228fd0375b92ba8d2b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.821132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.840328 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 897/1779] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:36 2022] PUT /identity/v3/projects/ee8337c545374fb79e72672eb365bb58/users/2ce5fae031e34a228fd0375b92ba8d2b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:36.841976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.844237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:36.877840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f83edb-83c3-42a6-906f-4ba3e89853d1 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62d984d1cf7c41f783ddbe05bc545e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0af2d58b8994eeeb64cec26101d4816', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.878509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f83edb-83c3-42a6-906f-4ba3e89853d1 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.878838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f83edb-83c3-42a6-906f-4ba3e89853d1 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.879112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f83edb-83c3-42a6-906f-4ba3e89853d1 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] PATH_INFO: `/v3/projects/0a55519ffc224c849e9be490acb4596e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.880663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f83edb-83c3-42a6-906f-4ba3e89853d1 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorizing `identity:get_project(project_id=0a55519ffc224c849e9be490acb4596e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.882947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f83edb-83c3-42a6-906f-4ba3e89853d1 tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.885014 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 883/1780] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/projects/0a55519ffc224c849e9be490acb4596e => generated 395 bytes in 557 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:36.886361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b564-6e53-477c-884a-4b8ed8ef4385 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.886708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b564-6e53-477c-884a-4b8ed8ef4385 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.887049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b564-6e53-477c-884a-4b8ed8ef4385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.949069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a3b1b564-6e53-477c-884a-4b8ed8ef4385 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:36.950199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a3b1b564-6e53-477c-884a-4b8ed8ef4385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:36.979466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.019172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a1421-38ac-4f2c-9912-d585055f5fd6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.020004 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 884/1781] 10.222.0.60 () {58 vars in 1015 bytes} [Sat Sep 17 20:50:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:37.021445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68071ac1-730b-46b9-8a0b-a86f52d3560e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.021855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68071ac1-730b-46b9-8a0b-a86f52d3560e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.022279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68071ac1-730b-46b9-8a0b-a86f52d3560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.030280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.031264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.031729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.032131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.035348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.037081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.047154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-68071ac1-730b-46b9-8a0b-a86f52d3560e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:37.048416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-68071ac1-730b-46b9-8a0b-a86f52d3560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:37.061317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 885/1782] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:37.062456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68071ac1-730b-46b9-8a0b-a86f52d3560e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.077493 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 898/1783] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:37.077493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.089236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68071ac1-730b-46b9-8a0b-a86f52d3560e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.100184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c817062-93f5-4527-b648-a6589612f9e2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.104269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a89c77-6444-44f2-92fa-b5b1f82d4202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.107700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89c77-6444-44f2-92fa-b5b1f82d4202 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.107700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89c77-6444-44f2-92fa-b5b1f82d4202 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.107700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89c77-6444-44f2-92fa-b5b1f82d4202 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.107700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89c77-6444-44f2-92fa-b5b1f82d4202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.113765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89c77-6444-44f2-92fa-b5b1f82d4202 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.128992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.130525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.132349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.132713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.138293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.141387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.158781 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 886/1784] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:37 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:37.159386 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 899/1785] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:37.160460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.170580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f209663-84a1-42b9-aa41-98db20fd6327 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.170891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f209663-84a1-42b9-aa41-98db20fd6327 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.171175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f209663-84a1-42b9-aa41-98db20fd6327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.189334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:37.210220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc769e9-98ad-4cf7-8d14-0aabe948d219 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.211734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6f209663-84a1-42b9-aa41-98db20fd6327 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:37.213315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6f209663-84a1-42b9-aa41-98db20fd6327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:37.221623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98bfd8-afe0-426b-ab58-c448dc8bf5a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.222263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98bfd8-afe0-426b-ab58-c448dc8bf5a0 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.222549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98bfd8-afe0-426b-ab58-c448dc8bf5a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.222801 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 887/1786] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:37.223639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f209663-84a1-42b9-aa41-98db20fd6327 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.226288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98bfd8-afe0-426b-ab58-c448dc8bf5a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.241825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f209663-84a1-42b9-aa41-98db20fd6327 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.247260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.250408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.250710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.250992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] PATH_INFO: `/v3/projects/b710ac6f05774b7380c602a99891b955/users/1de2d93fba78404bbe0f2f76317f903c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.256326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98bfd8-afe0-426b-ab58-c448dc8bf5a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.257841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98bfd8-afe0-426b-ab58-c448dc8bf5a0 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.274187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b710ac6f05774b7380c602a99891b955, user_id=1de2d93fba78404bbe0f2f76317f903c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.279665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.301098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 900/1787] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:37.303972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f94fa-a783-4935-bad1-400de64ce189 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.304686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f94fa-a783-4935-bad1-400de64ce189 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.305280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f94fa-a783-4935-bad1-400de64ce189 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.308585 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 901/1788] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:37.310727 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 888/1789] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:37 2022] PUT /identity/v3/projects/b710ac6f05774b7380c602a99891b955/users/1de2d93fba78404bbe0f2f76317f903c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:37.315330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.315468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f94fa-a783-4935-bad1-400de64ce189 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.407031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.498859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500321d-f422-4b54-9dc5-ccf3cd1b6ba8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.506187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.506826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.507109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.507378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.508131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.509419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f94fa-a783-4935-bad1-400de64ce189 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.510088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.521423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.522700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.522979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.523613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] PATH_INFO: `/v3/projects/ee8337c545374fb79e72672eb365bb58/users/2ce5fae031e34a228fd0375b92ba8d2b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.531175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8337c545374fb79e72672eb365bb58, user_id=2ce5fae031e34a228fd0375b92ba8d2b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.538340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.540991 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 889/1790] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:37.545719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.560228 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 902/1791] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:37 2022] PUT /identity/v3/projects/ee8337c545374fb79e72672eb365bb58/users/2ce5fae031e34a228fd0375b92ba8d2b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:37.561620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:37.563938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:37.571313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55992e1d-5761-4c9a-a9e9-84607bf27227 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.704107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.709629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.711001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.712189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.714499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:37.715604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.723005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.758005 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 890/1792] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:37.758642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.772527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317f925-ee07-489e-89a7-2d0f437aef79 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.775602 ncdv-master-1 devstack@keystone.service[99166]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 17 20:50:37.775999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.777419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:37.780727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34578064-22b7-4588-949e-907699017c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.780727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578064-22b7-4588-949e-907699017c08 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.780953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578064-22b7-4588-949e-907699017c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.781047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578064-22b7-4588-949e-907699017c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.782938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34578064-22b7-4588-949e-907699017c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.785476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34578064-22b7-4588-949e-907699017c08 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.793626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.801911 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 891/1793] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:37.803394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34578064-22b7-4588-949e-907699017c08 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.818941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34578064-22b7-4588-949e-907699017c08 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.826756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b5732-2849-484a-b1fe-1dd60b38a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.827819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5732-2849-484a-b1fe-1dd60b38a5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.828839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5732-2849-484a-b1fe-1dd60b38a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.828839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b5732-2849-484a-b1fe-1dd60b38a5ec admin admin] PATH_INFO: `/v3/projects/4790d0272151458794aaa3fdd314950d/users/a64d4b12872e451fae453d792f4e70f3/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.850436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b5732-2849-484a-b1fe-1dd60b38a5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4790d0272151458794aaa3fdd314950d, user_id=a64d4b12872e451fae453d792f4e70f3, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.854085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b5732-2849-484a-b1fe-1dd60b38a5ec admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.897798 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 892/1794] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:37 2022] PUT /identity/v3/projects/4790d0272151458794aaa3fdd314950d/users/a64d4b12872e451fae453d792f4e70f3/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:37.899499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490d41-6eff-485a-ae54-b6ae13a21711 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.899782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490d41-6eff-485a-ae54-b6ae13a21711 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.900050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490d41-6eff-485a-ae54-b6ae13a21711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.957381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a2490d41-6eff-485a-ae54-b6ae13a21711 None None] MFA Rules not processed for user `2ce5fae031e34a228fd0375b92ba8d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:37.958787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a2490d41-6eff-485a-ae54-b6ae13a21711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:37.977697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927f8bb-c26b-4733-a78a-1512977d66c0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.099090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62d984d1cf7c41f783ddbe05bc545e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0af2d58b8994eeeb64cec26101d4816', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.100126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.100856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.100856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] PATH_INFO: `/v3/projects/ceac6247852c48a99cc782fb135c6a28` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.102140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorizing `identity:get_project(project_id=ceac6247852c48a99cc782fb135c6a28)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.104364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.112249 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 903/1795] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/projects/ceac6247852c48a99cc782fb135c6a28 => generated 395 bytes in 549 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:38.117654 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 893/1796] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:38.118299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.118534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2490d41-6eff-485a-ae54-b6ae13a21711 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.135485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.253404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f681ca-4729-4cdf-a735-f52824b89a7e tempest-FlavorsAccessNegativeTestJSON-36790199 tempest-FlavorsAccessNegativeTestJSON-36790199-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.260426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.261349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.262014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.263394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.264426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.266875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.298661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2490d41-6eff-485a-ae54-b6ae13a21711 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.317217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceb625-cbec-4c9d-b700-31622dcc8a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.317861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb625-cbec-4c9d-b700-31622dcc8a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.318178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb625-cbec-4c9d-b700-31622dcc8a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.318503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb625-cbec-4c9d-b700-31622dcc8a0a admin admin] PATH_INFO: `/v3/projects/b710ac6f05774b7380c602a99891b955/users/1de2d93fba78404bbe0f2f76317f903c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.328191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ceb625-cbec-4c9d-b700-31622dcc8a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b710ac6f05774b7380c602a99891b955, user_id=1de2d93fba78404bbe0f2f76317f903c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.331591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ceb625-cbec-4c9d-b700-31622dcc8a0a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.355239 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 904/1797] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:38 2022] POST /identity/v3/projects => generated 393 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:38.357751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.358201 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 894/1798] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:38 2022] PUT /identity/v3/projects/b710ac6f05774b7380c602a99891b955/users/1de2d93fba78404bbe0f2f76317f903c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:38.359599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b0b7-13a5-42a0-a6da-6cbdd637f5f0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.359888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b0b7-13a5-42a0-a6da-6cbdd637f5f0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.360160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b0b7-13a5-42a0-a6da-6cbdd637f5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.415300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d7b1b0b7-13a5-42a0-a6da-6cbdd637f5f0 None None] MFA Rules not processed for user `eb8bfe1d590a4500b7e7b5c79a6a4e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:38.416716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d7b1b0b7-13a5-42a0-a6da-6cbdd637f5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:38.553247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.560090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928683c6-bdef-44e3-bd3b-bb40840c8940 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.575847 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 895/1799] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:38.575847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbcbc6-f491-4c38-a899-6cf4f0aae20d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.575847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbcbc6-f491-4c38-a899-6cf4f0aae20d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.575847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbcbc6-f491-4c38-a899-6cf4f0aae20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.580357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.581119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.581529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.581906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.582687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.585635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.610636 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 905/1800] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:38.612644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.623146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-55bbcbc6-f491-4c38-a899-6cf4f0aae20d None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:38.624626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-55bbcbc6-f491-4c38-a899-6cf4f0aae20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:38.633191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:38.659798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e86cc6-c5bc-43bd-9ffb-7309dfc65384 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.680217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.682233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.683391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.683866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] PATH_INFO: `/v3/users/1e865b793b834802a2bfa7146987d8f2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.687272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e865b793b834802a2bfa7146987d8f2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.690662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.731545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 896/1801] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:50:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:38.732842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbcbc6-f491-4c38-a899-6cf4f0aae20d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.780795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e865b793b834802a2bfa7146987d8f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:38.788722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbcbc6-f491-4c38-a899-6cf4f0aae20d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.804005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e865b793b834802a2bfa7146987d8f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:38.805299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.806564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.806846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.807117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.807924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.810815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.817416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e865b793b834802a2bfa7146987d8f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:38.818987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e865b793b834802a2bfa7146987d8f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:38.820492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7255d06-1d16-456d-8b23-3f275daa5ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e865b793b834802a2bfa7146987d8f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:38.831352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.849548 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 906/1802] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:38 2022] DELETE /identity/v3/users/1e865b793b834802a2bfa7146987d8f2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:38.854388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.856059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.856537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.862210 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 897/1803] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:38 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:38.863236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.877492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:38.908944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] MFA Rules not processed for user `1de2d93fba78404bbe0f2f76317f903c`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:38.913078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.012237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d54779-b3f3-46e4-9803-5c7935afda7c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.090325 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 907/1804] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:38 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:39.092151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.096292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.157631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb7afa-209d-4801-aaa8-b90bba925ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.158377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb7afa-209d-4801-aaa8-b90bba925ff5 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.158757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb7afa-209d-4801-aaa8-b90bba925ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.159028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb7afa-209d-4801-aaa8-b90bba925ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.161756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8fb7afa-209d-4801-aaa8-b90bba925ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.228817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.272145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cef13f-c53d-4a2a-8eb4-056332433269 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.285340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.286130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.286421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.286759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] PATH_INFO: `/v3/projects/4790d0272151458794aaa3fdd314950d/users/a64d4b12872e451fae453d792f4e70f3/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.289310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4790d0272151458794aaa3fdd314950d, user_id=a64d4b12872e451fae453d792f4e70f3, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.292214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.303604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb7afa-209d-4801-aaa8-b90bba925ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.305372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb7afa-209d-4801-aaa8-b90bba925ff5 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.314676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 908/1805] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:39 2022] PUT /identity/v3/projects/4790d0272151458794aaa3fdd314950d/users/a64d4b12872e451fae453d792f4e70f3/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:39.316836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.318785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.340550 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 898/1806] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:39.351432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.351432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.351432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.389386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:39.395183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.483213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d78f1a-49a8-4bc2-b7c6-1a788ca90f3c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.493755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd1a5e-b3cb-4131-b949-44f90be928f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.495019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd1a5e-b3cb-4131-b949-44f90be928f1 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.495366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd1a5e-b3cb-4131-b949-44f90be928f1 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.495637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd1a5e-b3cb-4131-b949-44f90be928f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.497903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f1cd1a5e-b3cb-4131-b949-44f90be928f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.523240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.586697 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 899/1807] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:39.587795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.592430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.608836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78093337-5c7b-47d3-8c04-4ca7ced320d9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.620107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389655e7-3b26-40b3-b255-797b5a008e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.620949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389655e7-3b26-40b3-b255-797b5a008e60 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.621313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389655e7-3b26-40b3-b255-797b5a008e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.621724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389655e7-3b26-40b3-b255-797b5a008e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.622606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389655e7-3b26-40b3-b255-797b5a008e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.624545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389655e7-3b26-40b3-b255-797b5a008e60 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.628664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd1a5e-b3cb-4131-b949-44f90be928f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.630699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd1a5e-b3cb-4131-b949-44f90be928f1 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.638891 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 900/1808] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:39.641574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41d00-f965-436a-890a-c6d38c98b32a None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.641832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41d00-f965-436a-890a-c6d38c98b32a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.642197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41d00-f965-436a-890a-c6d38c98b32a None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.643950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 901/1809] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:50:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:39.644856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c41d00-f965-436a-890a-c6d38c98b32a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.646681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70c41d00-f965-436a-890a-c6d38c98b32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.648156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 909/1810] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:50:39.649763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.650061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.650497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.715020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] MFA Rules not processed for user `a64d4b12872e451fae453d792f4e70f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:39.716276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.870122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c41d00-f965-436a-890a-c6d38c98b32a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.885948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.889695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.890034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.890701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.902729 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 910/1811] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:39.902729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.903993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.957052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.975778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.977766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.009939 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 902/1812] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:40.013168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.019358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7371265-0c86-4db5-be46-5d59a9169673 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.023552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd20dc-75e6-4117-af47-d4d662d5c1d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.025497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd20dc-75e6-4117-af47-d4d662d5c1d2 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.025497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd20dc-75e6-4117-af47-d4d662d5c1d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.025497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd20dc-75e6-4117-af47-d4d662d5c1d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.040782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd20dc-75e6-4117-af47-d4d662d5c1d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.045027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd20dc-75e6-4117-af47-d4d662d5c1d2 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.050638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78d928-0839-42a3-8e51-48af9dc6b312 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.057450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.058790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.059344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.068031 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 911/1813] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:40.069437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297c62-26b2-4daa-8290-8b7fd338ff94 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.069629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.069725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297c62-26b2-4daa-8290-8b7fd338ff94 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.070005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297c62-26b2-4daa-8290-8b7fd338ff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.070450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.072088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.090054 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 903/1814] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:40.094358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.109432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2b297c62-26b2-4daa-8290-8b7fd338ff94 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:40.111932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834796e0-dbe8-45a3-9a5a-8ab0f465f525 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.114417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2b297c62-26b2-4daa-8290-8b7fd338ff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:40.120365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.121447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.121902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.122181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] PATH_INFO: `/v3/projects/c06c143978aa46188dea1abd69444966/users/1d4e78f16e2847b19f79f4068644d902/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.125819 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 912/1815] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:50:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:40.146562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06c143978aa46188dea1abd69444966, user_id=1d4e78f16e2847b19f79f4068644d902, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.149610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.175288 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 904/1816] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:40 2022] PUT /identity/v3/projects/c06c143978aa46188dea1abd69444966/users/1d4e78f16e2847b19f79f4068644d902/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:40.185928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08ab64-f304-4c7b-aa97-c27a25b28552 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.186342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08ab64-f304-4c7b-aa97-c27a25b28552 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.186933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08ab64-f304-4c7b-aa97-c27a25b28552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.234894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4e08ab64-f304-4c7b-aa97-c27a25b28552 None None] MFA Rules not processed for user `1d4e78f16e2847b19f79f4068644d902`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:40.236339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4e08ab64-f304-4c7b-aa97-c27a25b28552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:40.285331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.359766 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 913/1817] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:40.364304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.438033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08ab64-f304-4c7b-aa97-c27a25b28552 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.527018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3593c49-5def-46a7-bdd5-7eb98231fc00 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.555894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.557244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.557705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.560272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.561135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.567092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.635148 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 905/1818] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:40 2022] POST /identity/v3/projects => generated 391 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:40.636053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.647298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08ab64-f304-4c7b-aa97-c27a25b28552 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.789965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacf442-eb9f-423a-a049-21b835328ae2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.792429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.793357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.793815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.794252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.809314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.810714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.811060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.811430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.812366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.814890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.853135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 906/1819] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:40 2022] POST /identity/v3/users => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:40.862930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.865401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:40.911471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371bb71-fb23-4ee8-928a-7924db89105e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.932394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.932966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.933243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.933580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] PATH_INFO: `/v3/projects/0a55519ffc224c849e9be490acb4596e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.934973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a55519ffc224c849e9be490acb4596e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.938734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.969344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.971036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.976868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.992622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 914/1820] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:50:40.993640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.004813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc6db-3641-4162-9040-a7d0e1c800bc service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.009406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:41.012045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.013233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.013796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.014305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.015813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.017745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.036610 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 915/1821] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:41.038615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.071840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27f606-b9bc-4857-83dc-1e9db8feffed admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.094375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a55519ffc224c849e9be490acb4596e', 'request_id': 'req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.112859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a55519ffc224c849e9be490acb4596e', 'request_id': 'req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.138545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a55519ffc224c849e9be490acb4596e', 'request_id': 'req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.157422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] The token cache is being invalidate because project 0a55519ffc224c849e9be490acb4596e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:41.158022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.159362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.162066 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 907/1822] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:40 2022] DELETE /identity/v3/projects/0a55519ffc224c849e9be490acb4596e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:41.163565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.166429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:41.210506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.211552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.211969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.212420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] PATH_INFO: `/v3/projects/b169bca404fd4bd4a079fc781c6854e1/users/94fce00fe5944c6ea940de3bab30b058/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.239278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b169bca404fd4bd4a079fc781c6854e1, user_id=94fce00fe5944c6ea940de3bab30b058, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.243169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.267139 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 916/1823] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:41 2022] PUT /identity/v3/projects/b169bca404fd4bd4a079fc781c6854e1/users/94fce00fe5944c6ea940de3bab30b058/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:41.268255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.270338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:41.352646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9f5c0-af19-40ca-bba2-9f25aa02d8d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.395587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580454c9-c28b-4433-ac4f-4637769d0b5a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.416373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.417156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.417570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.417777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] PATH_INFO: `/v3/users/62d984d1cf7c41f783ddbe05bc545e4a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.426973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d984d1cf7c41f783ddbe05bc545e4a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.429091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.469225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.469854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.470152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.470348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.472428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:41.533164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:41.563640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.565314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.582745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d984d1cf7c41f783ddbe05bc545e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.594019 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 908/1824] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:41 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:41.594908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.596325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:41.614409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247de4-c1ff-4c57-9ed4-416b4fcebaa4 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.620633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.621335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.626735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.626735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.626735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.628469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.630727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d984d1cf7c41f783ddbe05bc545e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.649088 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 909/1825] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:41.649088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.651682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d984d1cf7c41f783ddbe05bc545e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.653031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d984d1cf7c41f783ddbe05bc545e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.654260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d984d1cf7c41f783ddbe05bc545e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:41.656487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:41.673970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf9e7-1d8d-473f-91ad-efb544db6ca7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.681111 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 917/1826] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:41 2022] DELETE /identity/v3/users/62d984d1cf7c41f783ddbe05bc545e4a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:41.681276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94a876-7787-4e87-9e35-fb7a305d09a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.682142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94a876-7787-4e87-9e35-fb7a305d09a6 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.682467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94a876-7787-4e87-9e35-fb7a305d09a6 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.682975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.683347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94a876-7787-4e87-9e35-fb7a305d09a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.690720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:41.694328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b94a876-7787-4e87-9e35-fb7a305d09a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:41.709727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94a876-7787-4e87-9e35-fb7a305d09a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.711344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94a876-7787-4e87-9e35-fb7a305d09a6 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.726933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 910/1827] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:41.828757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19fc6-f252-4072-a85a-e8578834492f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.844376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85910c-fdfd-4ce3-bde8-99e45c68f5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.845521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85910c-fdfd-4ce3-bde8-99e45c68f5c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.845716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85910c-fdfd-4ce3-bde8-99e45c68f5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.845982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85910c-fdfd-4ce3-bde8-99e45c68f5c7 admin admin] PATH_INFO: `/v3/projects/b169bca404fd4bd4a079fc781c6854e1/users/94fce00fe5944c6ea940de3bab30b058/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.851305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85910c-fdfd-4ce3-bde8-99e45c68f5c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b169bca404fd4bd4a079fc781c6854e1, user_id=94fce00fe5944c6ea940de3bab30b058, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.855189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85910c-fdfd-4ce3-bde8-99e45c68f5c7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.868473 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 918/1828] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:41 2022] PUT /identity/v3/projects/b169bca404fd4bd4a079fc781c6854e1/users/94fce00fe5944c6ea940de3bab30b058/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:41.886439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.886734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.887572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.964543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] MFA Rules not processed for user `94fce00fe5944c6ea940de3bab30b058`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:41.968122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.096017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.096224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.096423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.123219 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 911/1829] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:41 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:42.132057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:42.133424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.252739 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 919/1830] 10.222.0.50 () {58 vars in 1015 bytes} [Sat Sep 17 20:50:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:50:42.264261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:42.265886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.284976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:42.288601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57325309-4ea9-42c1-a1d9-4a2e608baae6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:42.299775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.300419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:42.301417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.301917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.302790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.322231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:42.324371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:42.364391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 912/1831] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:42.365493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:42.369098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.514930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f12037-8ea4-4ddc-b206-8c858d2f632e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:42.532384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:42.533054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.533337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.533681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] PATH_INFO: `/v3/projects/d0af2d58b8994eeeb64cec26101d4816` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.535344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0af2d58b8994eeeb64cec26101d4816)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:42.537495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:42.606360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde15faf-8cdc-4f46-9f2e-5ea38e1c6054 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:42.622477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:42.631520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:42.632237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.632460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.632734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.641272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0af2d58b8994eeeb64cec26101d4816', 'request_id': 'req-2c232568-ff82-4853-8a25-98ca712a273f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:42.641613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.655588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0af2d58b8994eeeb64cec26101d4816', 'request_id': 'req-2c232568-ff82-4853-8a25-98ca712a273f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:42.673505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0af2d58b8994eeeb64cec26101d4816', 'request_id': 'req-2c232568-ff82-4853-8a25-98ca712a273f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:42.702797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] The token cache is being invalidate because project d0af2d58b8994eeeb64cec26101d4816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:42.703095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:42.706469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:42.709744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:42.714247 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 920/1832] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:42 2022] DELETE /identity/v3/projects/d0af2d58b8994eeeb64cec26101d4816 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:42.715465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:42.719333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.770379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:42.772409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:42.804871 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 913/1833] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:42.864235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232568-ff82-4853-8a25-98ca712a273f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:42.869494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:42.870268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.870697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.871205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] PATH_INFO: `/v3/users/755c745bca614903bd8958d18e98afc0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.872700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755c745bca614903bd8958d18e98afc0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:42.874779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:42.997330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755c745bca614903bd8958d18e98afc0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.021297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755c745bca614903bd8958d18e98afc0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.033831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755c745bca614903bd8958d18e98afc0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.034982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755c745bca614903bd8958d18e98afc0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.035947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755c745bca614903bd8958d18e98afc0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.065848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 921/1834] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:42 2022] DELETE /identity/v3/users/755c745bca614903bd8958d18e98afc0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:43.145195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.148231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:43.175987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.178284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:43.195928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4bef-bf3e-4f6e-adb6-80eb4b45fea2 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.360921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.430423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.442331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd3a89-7935-4753-bcbb-044c23dce391 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.449646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.451468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.451986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] PATH_INFO: `/v3/users/a9626c56a7af47e292e3c3b61e35cd7b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.460064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9626c56a7af47e292e3c3b61e35cd7b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:43.463812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:43.470242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90cc3b-dbab-4fe8-941f-2221e2af3781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.471016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90cc3b-dbab-4fe8-941f-2221e2af3781 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.471227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90cc3b-dbab-4fe8-941f-2221e2af3781 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.473939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90cc3b-dbab-4fe8-941f-2221e2af3781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.479383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4c90cc3b-dbab-4fe8-941f-2221e2af3781 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:43.577915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9626c56a7af47e292e3c3b61e35cd7b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.578824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9626c56a7af47e292e3c3b61e35cd7b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.582680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90cc3b-dbab-4fe8-941f-2221e2af3781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:43.584384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90cc3b-dbab-4fe8-941f-2221e2af3781 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:43.595611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9626c56a7af47e292e3c3b61e35cd7b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.596415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9626c56a7af47e292e3c3b61e35cd7b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.615223 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 922/1835] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:43 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:50:43.631537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9626c56a7af47e292e3c3b61e35cd7b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:43.669745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.670316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.670316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.676738 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 914/1836] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:43 2022] DELETE /identity/v3/users/a9626c56a7af47e292e3c3b61e35cd7b => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:43.700485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:43.703807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:43.806953 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 923/1837] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:43.821175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.823498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:43.947182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.968525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581e61a-116c-4506-83e7-95484bef8666 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.979507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1db17-786f-4ada-ae8c-42f22f9d8f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.981296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1db17-786f-4ada-ae8c-42f22f9d8f3d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.983673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1db17-786f-4ada-ae8c-42f22f9d8f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.985589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1db17-786f-4ada-ae8c-42f22f9d8f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.001023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.003106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:44.065791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dce29-83e4-4bb3-b3f2-e2140296c21e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.073246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1db17-786f-4ada-ae8c-42f22f9d8f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.077270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1db17-786f-4ada-ae8c-42f22f9d8f3d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.094714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.096288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.096764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.097053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] PATH_INFO: `/v3/projects/ceac6247852c48a99cc782fb135c6a28` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.098400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceac6247852c48a99cc782fb135c6a28)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.100363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.105966 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 915/1838] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:50:44.247502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceac6247852c48a99cc782fb135c6a28', 'request_id': 'req-4beef34d-2454-4df1-bce9-bcc934510c5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:44.265922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceac6247852c48a99cc782fb135c6a28', 'request_id': 'req-4beef34d-2454-4df1-bce9-bcc934510c5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:44.295692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceac6247852c48a99cc782fb135c6a28', 'request_id': 'req-4beef34d-2454-4df1-bce9-bcc934510c5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:44.307628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:44.314240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] The token cache is being invalidate because project ceac6247852c48a99cc782fb135c6a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:44.315779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:44.318429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4beef34d-2454-4df1-bce9-bcc934510c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:44.321138 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 924/1839] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:43 2022] DELETE /identity/v3/projects/ceac6247852c48a99cc782fb135c6a28 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:44.328273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc8138-d900-4b44-bcc5-80eaaa8e9438 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.328693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc8138-d900-4b44-bcc5-80eaaa8e9438 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.329087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc8138-d900-4b44-bcc5-80eaaa8e9438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.367057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-70fc8138-d900-4b44-bcc5-80eaaa8e9438 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:44.369685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70fc8138-d900-4b44-bcc5-80eaaa8e9438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:44.475400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 916/1840] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:44.476199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec305f5-ba71-4441-a9a0-3c99f46ef90d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.476517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec305f5-ba71-4441-a9a0-3c99f46ef90d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.476789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec305f5-ba71-4441-a9a0-3c99f46ef90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.512863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5ec305f5-ba71-4441-a9a0-3c99f46ef90d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:44.514836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5ec305f5-ba71-4441-a9a0-3c99f46ef90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:44.539652 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 925/1841] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:44.545314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc8138-d900-4b44-bcc5-80eaaa8e9438 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.561497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc8138-d900-4b44-bcc5-80eaaa8e9438 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.576795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.577615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.578103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.578235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.579007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:44.579378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.588186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.626403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec305f5-ba71-4441-a9a0-3c99f46ef90d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.631267 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 917/1842] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:44.639990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec305f5-ba71-4441-a9a0-3c99f46ef90d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.677259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.678174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.678457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.680057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.683167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.700045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.804402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.811607 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 926/1843] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:44 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:44.816445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c08f0-f6fb-4071-b361-fd814fc4efbb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.913895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.916137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:44.925390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.942848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.943684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4142c9-0f3e-4c9d-88e7-e551b1c03a5f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.945328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.955306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.956203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.958593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.961733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.967581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.967581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.967581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] PATH_INFO: `/v3/projects/7251e61bfc7e4fdcb7ede9dc59b61138` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.972948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7251e61bfc7e4fdcb7ede9dc59b61138)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.014580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.095325 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 918/1844] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:44 2022] POST /identity/v3/users => generated 291 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:45.095438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.132379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64805a-c2b1-44c4-bbd7-57d9dd194ae0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.158582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.160466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.161932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.163248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.166471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7251e61bfc7e4fdcb7ede9dc59b61138', 'request_id': 'req-cbe7e867-f301-4fdc-9d67-c2f9c192622a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.167218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.171659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.179203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7251e61bfc7e4fdcb7ede9dc59b61138', 'request_id': 'req-cbe7e867-f301-4fdc-9d67-c2f9c192622a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.188654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7251e61bfc7e4fdcb7ede9dc59b61138', 'request_id': 'req-cbe7e867-f301-4fdc-9d67-c2f9c192622a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.209716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 919/1845] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:45.215953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.234013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] The token cache is being invalidate because project 7251e61bfc7e4fdcb7ede9dc59b61138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:45.234396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.241038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cdc96-d668-4036-947a-72d0f183e91a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.242824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.244199 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 927/1846] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:44 2022] DELETE /identity/v3/projects/7251e61bfc7e4fdcb7ede9dc59b61138 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:45.248176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.263953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:45.378953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:45.419034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.419581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.419870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.420138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.420919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.432029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.462183 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 920/1847] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:45.463476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7e867-f301-4fdc-9d67-c2f9c192622a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.467169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.470330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:45.488223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.502828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec26d3-d44c-4575-9748-223db0e6d0f5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.509785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.510546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.512006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] PATH_INFO: `/v3/users/1de2d93fba78404bbe0f2f76317f903c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.529387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.529863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de2d93fba78404bbe0f2f76317f903c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.531305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.532450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.533065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] PATH_INFO: `/v3/projects/c8a32c8816514b6aa7aa27895e8c4dd1/users/1dbc0ba3e29e4fa9a5a60f322502e0a9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.535027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.585333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a32c8816514b6aa7aa27895e8c4dd1, user_id=1dbc0ba3e29e4fa9a5a60f322502e0a9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.588595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.633587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 921/1848] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:45 2022] PUT /identity/v3/projects/c8a32c8816514b6aa7aa27895e8c4dd1/users/1dbc0ba3e29e4fa9a5a60f322502e0a9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:45.635418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.638806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:45.675629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de2d93fba78404bbe0f2f76317f903c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.697689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=15 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:45.743188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de2d93fba78404bbe0f2f76317f903c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.764971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de2d93fba78404bbe0f2f76317f903c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.766001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de2d93fba78404bbe0f2f76317f903c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.767079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25c598c7-7b9f-47dc-8aa3-44f48e06cfc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de2d93fba78404bbe0f2f76317f903c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.782978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49968b-1c19-4099-89bc-c49e60a833bb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.790631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.791821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.792367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.792943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] PATH_INFO: `/v3/users/1d4e78f16e2847b19f79f4068644d902` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.794525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4e78f16e2847b19f79f4068644d902)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.798066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.816034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 928/1849] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:45 2022] DELETE /identity/v3/users/1de2d93fba78404bbe0f2f76317f903c => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:45.852863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f80a8-7600-4b7e-91b3-f1f0b0b14ccf None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.854321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f80a8-7600-4b7e-91b3-f1f0b0b14ccf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.854610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f80a8-7600-4b7e-91b3-f1f0b0b14ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.918643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4e78f16e2847b19f79f4068644d902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.919746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4e78f16e2847b19f79f4068644d902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.945977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4e78f16e2847b19f79f4068644d902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.946965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7f6f80a8-7600-4b7e-91b3-f1f0b0b14ccf None None] MFA Rules not processed for user `1dbc0ba3e29e4fa9a5a60f322502e0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:45.947306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4e78f16e2847b19f79f4068644d902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:45.948699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7f6f80a8-7600-4b7e-91b3-f1f0b0b14ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:45.978650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4e78f16e2847b19f79f4068644d902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.009068 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 922/1850] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:45 2022] DELETE /identity/v3/users/1d4e78f16e2847b19f79f4068644d902 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:46.009068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.015343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.094487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:46.130936 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 929/1851] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:46.134021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.134734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.135459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.136145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70daea-20aa-4e75-a222-80b73f772620 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.152169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.154197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.154575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.154955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] PATH_INFO: `/v3/users/abbb938447164ec6ab8461731a9f3780` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.158962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbb938447164ec6ab8461731a9f3780)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:46.164044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:46.164769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:46.166861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.184156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 930/1852] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:46.185727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.187330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.205575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f250f31-e958-434a-9018-3e06b4ef948e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.212811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.213890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.214902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.214902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.217210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:46.219791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:46.282485 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 931/1853] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:46 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:46.284634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.289340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.303198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbb938447164ec6ab8461731a9f3780'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.305589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbb938447164ec6ab8461731a9f3780'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.339204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbb938447164ec6ab8461731a9f3780'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.343051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbb938447164ec6ab8461731a9f3780'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.415065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbb938447164ec6ab8461731a9f3780'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.442598 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 923/1854] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:46 2022] DELETE /identity/v3/users/abbb938447164ec6ab8461731a9f3780 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:46.442598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.443923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.453415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8e9ab-7980-442c-9165-7d00ee0843e9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.566375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:46.581968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a412c-fd70-4339-b241-701ff7c3638b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.582611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a412c-fd70-4339-b241-701ff7c3638b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.582898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a412c-fd70-4339-b241-701ff7c3638b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.583147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a412c-fd70-4339-b241-701ff7c3638b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.587873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b77a412c-fd70-4339-b241-701ff7c3638b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.606925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12008d1f-8315-4b22-b8e9-09981518e66e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.618756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a412c-fd70-4339-b241-701ff7c3638b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:46.621620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.622639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a412c-fd70-4339-b241-701ff7c3638b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:46.623760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.624048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.625360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.627069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:46.632591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:46.651171 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 932/1855] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:46.653948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f307-02bd-4535-b258-f8a980bad575 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.654238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f307-02bd-4535-b258-f8a980bad575 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.654506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f307-02bd-4535-b258-f8a980bad575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.712794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3d10f307-02bd-4535-b258-f8a980bad575 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:46.713775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 924/1856] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:46 2022] POST /identity/v3/users => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:46.716402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d10f307-02bd-4535-b258-f8a980bad575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.721189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.729111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.746061 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 933/1857] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:46.747141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f307-02bd-4535-b258-f8a980bad575 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.759697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26958e-c85e-417e-abd6-eff379b93b74 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.767235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.770009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.770538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.771098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] PATH_INFO: `/v3/projects/b710ac6f05774b7380c602a99891b955` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.773081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b710ac6f05774b7380c602a99891b955)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:46.774603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f307-02bd-4535-b258-f8a980bad575 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.777179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:46.787992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.789529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.789992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.790465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.791325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:46.793358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:46.816412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 934/1858] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:46.817906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.820528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.854477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e568728-5d5e-44e0-9d31-0c1c524da0c1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.877035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b710ac6f05774b7380c602a99891b955', 'request_id': 'req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.879087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:46.888611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b710ac6f05774b7380c602a99891b955', 'request_id': 'req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.900823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b710ac6f05774b7380c602a99891b955', 'request_id': 'req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.920380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] The token cache is being invalidate because project b710ac6f05774b7380c602a99891b955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:46.921090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.921820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:46.925027 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 925/1859] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:46 2022] DELETE /identity/v3/projects/b710ac6f05774b7380c602a99891b955 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:46.926761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.928486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.990009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf94f02-285c-4777-a54c-cda8b0371d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.990750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf94f02-285c-4777-a54c-cda8b0371d6a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.991089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf94f02-285c-4777-a54c-cda8b0371d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.991421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf94f02-285c-4777-a54c-cda8b0371d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.994377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7cf94f02-285c-4777-a54c-cda8b0371d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.043505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c00c3e-ad4d-4b0f-a21d-eceeac3f1e3c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.051825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.052761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.053077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.053421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] PATH_INFO: `/v3/projects/115f5ac7e1164643a0737a0836d4d8ab/users/5ab8354581a24c5887fc5d9afb72bd08/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.086560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115f5ac7e1164643a0737a0836d4d8ab, user_id=5ab8354581a24c5887fc5d9afb72bd08, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.087904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf94f02-285c-4777-a54c-cda8b0371d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.089628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf94f02-285c-4777-a54c-cda8b0371d6a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.090017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.101976 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 935/1860] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:47.103871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ba52f-831c-4360-afad-38aa5da0446e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.104390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ba52f-831c-4360-afad-38aa5da0446e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.104855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ba52f-831c-4360-afad-38aa5da0446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.112616 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 926/1861] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:46 2022] PUT /identity/v3/projects/115f5ac7e1164643a0737a0836d4d8ab/users/5ab8354581a24c5887fc5d9afb72bd08/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:47.116425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.129826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ff7ba52f-831c-4360-afad-38aa5da0446e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:47.131190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ff7ba52f-831c-4360-afad-38aa5da0446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.212274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b800d7-b2d3-4d80-b7a9-2cf857ea0642 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.222978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.224595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.225283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.225757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.226876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.230400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.246764 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 927/1862] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:47.251680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.268316 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 936/1863] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:47.270809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942657cb-4839-41b6-aece-7162f8345feb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.270809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942657cb-4839-41b6-aece-7162f8345feb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.270809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942657cb-4839-41b6-aece-7162f8345feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.287097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1e1be-eed2-4cad-ae96-e6919adcdf46 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.295020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-942657cb-4839-41b6-aece-7162f8345feb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:47.296547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-942657cb-4839-41b6-aece-7162f8345feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.298018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.299662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.300197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.300647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] PATH_INFO: `/v3/projects/115f5ac7e1164643a0737a0836d4d8ab/users/5ab8354581a24c5887fc5d9afb72bd08/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.304269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] RBAC: Authorizing `identity:create_grant(project_id=115f5ac7e1164643a0737a0836d4d8ab, user_id=5ab8354581a24c5887fc5d9afb72bd08, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.309273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.310745 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 937/1864] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:47.311839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942657cb-4839-41b6-aece-7162f8345feb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.313487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-942657cb-4839-41b6-aece-7162f8345feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.324685 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 928/1865] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:47 2022] PUT /identity/v3/projects/115f5ac7e1164643a0737a0836d4d8ab/users/5ab8354581a24c5887fc5d9afb72bd08/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:47.326023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.362886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942657cb-4839-41b6-aece-7162f8345feb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.382460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:47.453332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:47.462312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.463293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.463643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.463911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] PATH_INFO: `/v3/projects/c06c143978aa46188dea1abd69444966` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.466410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c06c143978aa46188dea1abd69444966)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.478772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.494452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabf3f7-d1e2-4194-85e6-2565206a511b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.512190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69fb85-255f-4c01-b8b1-db9a18a98ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.513846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69fb85-255f-4c01-b8b1-db9a18a98ddb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.514159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69fb85-255f-4c01-b8b1-db9a18a98ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.515583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69fb85-255f-4c01-b8b1-db9a18a98ddb admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.516894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69fb85-255f-4c01-b8b1-db9a18a98ddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:47.518178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69fb85-255f-4c01-b8b1-db9a18a98ddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.523824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69fb85-255f-4c01-b8b1-db9a18a98ddb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.560028 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 929/1866] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:47.560028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.560028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.560028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.612472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c06c143978aa46188dea1abd69444966', 'request_id': 'req-f6cae567-9301-4d0f-94d3-4b6aec65bc52'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:47.624215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c06c143978aa46188dea1abd69444966', 'request_id': 'req-f6cae567-9301-4d0f-94d3-4b6aec65bc52'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:47.637039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c06c143978aa46188dea1abd69444966', 'request_id': 'req-f6cae567-9301-4d0f-94d3-4b6aec65bc52'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:47.644583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] MFA Rules not processed for user `5ab8354581a24c5887fc5d9afb72bd08`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:47.646843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.654308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] The token cache is being invalidate because project c06c143978aa46188dea1abd69444966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:47.654782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:47.656354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:47.658925 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 938/1867] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:47 2022] DELETE /identity/v3/projects/c06c143978aa46188dea1abd69444966 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:47.659979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.662121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.779329 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 930/1868] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:47.780320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.781805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.818535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cae567-9301-4d0f-94d3-4b6aec65bc52 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.823433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.824147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.824405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.824681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.825440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.827458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.866588 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 939/1869] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:47 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:47.868492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.871300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.901225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2347-2b27-49bb-9def-7994a1c6ab36 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.964805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.020207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d01a1-8ced-4994-be80-906758381402 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.031041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.031808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.032085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.032353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] PATH_INFO: `/v3/projects/a1d03e8b59b7494a909f33fbee0cadfb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.033700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d03e8b59b7494a909f33fbee0cadfb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.035293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.035880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.036028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.036432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.036902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] PATH_INFO: `/v3/users/94fce00fe5944c6ea940de3bab30b058` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.038437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fce00fe5944c6ea940de3bab30b058)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.040991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.070891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.108025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d03e8b59b7494a909f33fbee0cadfb', 'request_id': 'req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.121505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d03e8b59b7494a909f33fbee0cadfb', 'request_id': 'req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.133360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d03e8b59b7494a909f33fbee0cadfb', 'request_id': 'req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.139942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fce00fe5944c6ea940de3bab30b058'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.140981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fce00fe5944c6ea940de3bab30b058'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.155790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fce00fe5944c6ea940de3bab30b058'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.156506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fce00fe5944c6ea940de3bab30b058'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.159651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] The token cache is being invalidate because project a1d03e8b59b7494a909f33fbee0cadfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:48.159947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.160609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.167540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 940/1870] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:47 2022] DELETE /identity/v3/projects/a1d03e8b59b7494a909f33fbee0cadfb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:48.168995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.170652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.181926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fce00fe5944c6ea940de3bab30b058'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.204244 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 931/1871] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:47 2022] DELETE /identity/v3/users/94fce00fe5944c6ea940de3bab30b058 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:48.205786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.208112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.279452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25e153-6c6f-4fa6-bdff-51dc6988cdd8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.323053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fac81b-9518-4d3f-8133-59052369b6e5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.333092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.334095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.334356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.334738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.336400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.340531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.370252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.386215 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 932/1872] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:48 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:48.387196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.388289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.389229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.389354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.389510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.389733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.392870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.418216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10be244-0e06-45dd-9b43-1cd0273821a1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.425224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.426278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.426686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.427498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] PATH_INFO: `/v3/users/a64d4b12872e451fae453d792f4e70f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.429333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64d4b12872e451fae453d792f4e70f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.431658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.477697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.480102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.498773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 941/1873] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:48.499726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.501238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.522086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4748f-5293-4da1-bbfe-a8dfbc6d1c34 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.532717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.533508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.533793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.534097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.535060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.537576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.550617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64d4b12872e451fae453d792f4e70f3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.553490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 942/1874] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:48.553651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64d4b12872e451fae453d792f4e70f3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.566409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.587542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64d4b12872e451fae453d792f4e70f3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.588370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64d4b12872e451fae453d792f4e70f3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.593691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5976f0-dc4d-4658-b6d3-7697c3b43cb0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.603325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64d4b12872e451fae453d792f4e70f3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:48.605042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.605915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.607692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.607964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.608666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.610337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.624996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=76,keystone:DELETE=19,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.631931 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 943/1875] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:48.633335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.635468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.652633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198bb82-7986-4ac2-a5fd-5088900fe02b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.673774 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 933/1876] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:48 2022] DELETE /identity/v3/users/a64d4b12872e451fae453d792f4e70f3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:48.675737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cf484-89f8-472d-8f38-7967fe829d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.676737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cf484-89f8-472d-8f38-7967fe829d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.677451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cf484-89f8-472d-8f38-7967fe829d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.677965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cf484-89f8-472d-8f38-7967fe829d89 admin admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077/users/ee3d8ef713bd4844833d9374766452d8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.697793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.699779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.717604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cf484-89f8-472d-8f38-7967fe829d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f1d412d4a44e09b51be3b14530c077, user_id=ee3d8ef713bd4844833d9374766452d8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.720656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cf484-89f8-472d-8f38-7967fe829d89 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.773841 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 944/1877] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:48 2022] PUT /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077/users/ee3d8ef713bd4844833d9374766452d8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:48.785162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eee075-261a-4b63-8e2a-e0f84fe6fe1e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.785937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eee075-261a-4b63-8e2a-e0f84fe6fe1e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.786458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eee075-261a-4b63-8e2a-e0f84fe6fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.815983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1eb23-836e-4639-9522-745a2c43acc2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.867970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f1eee075-261a-4b63-8e2a-e0f84fe6fe1e None None] MFA Rules not processed for user `ee3d8ef713bd4844833d9374766452d8`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:48.870001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f1eee075-261a-4b63-8e2a-e0f84fe6fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.935069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.936550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.936550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.937024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] PATH_INFO: `/v3/projects/b169bca404fd4bd4a079fc781c6854e1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.943298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b169bca404fd4bd4a079fc781c6854e1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.945502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.029389 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 945/1878] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:50:49.040269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eee075-261a-4b63-8e2a-e0f84fe6fe1e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.076048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eee075-261a-4b63-8e2a-e0f84fe6fe1e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.114362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b169bca404fd4bd4a079fc781c6854e1', 'request_id': 'req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:49.133495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b169bca404fd4bd4a079fc781c6854e1', 'request_id': 'req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:49.149497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b169bca404fd4bd4a079fc781c6854e1', 'request_id': 'req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:49.153826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:49.182728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] The token cache is being invalidate because project b169bca404fd4bd4a079fc781c6854e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:49.183026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:49.183878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f8db31cd-5881-4d09-84f1-e9c6e93ef8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:49.187502 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 934/1879] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:48 2022] DELETE /identity/v3/projects/b169bca404fd4bd4a079fc781c6854e1 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:49.193029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df93457-d589-4d1e-b3f7-d251c3bd3924 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.201002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df93457-d589-4d1e-b3f7-d251c3bd3924 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.201002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df93457-d589-4d1e-b3f7-d251c3bd3924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.214821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.215706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.216149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.216568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.217444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.219905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.252043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7df93457-d589-4d1e-b3f7-d251c3bd3924 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:49.253888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7df93457-d589-4d1e-b3f7-d251c3bd3924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:49.268950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 946/1880] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:49 2022] POST /identity/v3/projects => generated 383 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:49.274600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.277416 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 935/1881] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:49.290705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:49.296859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10693-202b-41bf-a206-cef0c9bdae2b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.297418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10693-202b-41bf-a206-cef0c9bdae2b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.297730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10693-202b-41bf-a206-cef0c9bdae2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.344731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-36a10693-202b-41bf-a206-cef0c9bdae2b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:49.346657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-36a10693-202b-41bf-a206-cef0c9bdae2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:49.370473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:49.485911 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 936/1882] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:49.490805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a10693-202b-41bf-a206-cef0c9bdae2b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.517646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cea39-f698-4e00-994b-4078b69adf9a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.525203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a10693-202b-41bf-a206-cef0c9bdae2b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.536858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.537424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.537914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.538234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.539229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.539821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.541077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.543843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.544148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.544736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.546113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:49.546827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.549724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.590349 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 947/1883] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:49 2022] POST /identity/v3/users => generated 298 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:49.594791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.607091 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 937/1884] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:49.610356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.619379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc37e66-4351-4691-8a7b-54c48249f7e9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.628306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.628940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.629214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.629498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.630213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.638103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.651108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e2203-86bb-4e42-8a6f-d511cb69a7b2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.663316 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 948/1885] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:49.667765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.679941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.683324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.684215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.686218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.688026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04b63b-dae8-4bba-b98b-44040d55d683 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.695532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.698083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.698378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.698690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.698985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] PATH_INFO: `/v3/projects/06fddef025ac4c76aba4443a4f467108/users/d11a5517ab2e4991bc00b29ae4d6cd7f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.702862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.745950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06fddef025ac4c76aba4443a4f467108, user_id=d11a5517ab2e4991bc00b29ae4d6cd7f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.747615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.780084 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 938/1886] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:49 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:49.781153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.783577 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 949/1887] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:49 2022] PUT /identity/v3/projects/06fddef025ac4c76aba4443a4f467108/users/d11a5517ab2e4991bc00b29ae4d6cd7f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:49.785222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:49.787543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.884091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717777e9-d3a0-4a6a-9d45-070ee672cd34 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.902417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.903677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.904042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.904395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.905129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.906852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.908974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd691c-ff7c-4e99-8cd9-5719d4881d68 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.931759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.932432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.932708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.932972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] PATH_INFO: `/v3/projects/4790d0272151458794aaa3fdd314950d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.934380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4790d0272151458794aaa3fdd314950d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.936729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.944067 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 950/1888] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:50:49.944968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.959436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13769ffd-eb43-4ddb-aae8-a20e3c42002c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.979417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.980633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.980993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.981342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.982517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.985113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.997499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.013505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4790d0272151458794aaa3fdd314950d', 'request_id': 'req-49e72551-367d-4c26-b9fc-c5bdd1db01f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.021527 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 951/1889] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:49 2022] POST /identity/v3/users => generated 288 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:50.021527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.021527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.025515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4790d0272151458794aaa3fdd314950d', 'request_id': 'req-49e72551-367d-4c26-b9fc-c5bdd1db01f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.038474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4790d0272151458794aaa3fdd314950d', 'request_id': 'req-49e72551-367d-4c26-b9fc-c5bdd1db01f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.042582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42130298-5d3c-471f-9e1d-02a03d03694c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.051561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] The token cache is being invalidate because project 4790d0272151458794aaa3fdd314950d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:50.051855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.052541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.065679 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 939/1890] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:49 2022] DELETE /identity/v3/projects/4790d0272151458794aaa3fdd314950d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:50.067524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.067725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.107056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.131602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.132339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.132537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.132872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] PATH_INFO: `/v3/users/1dbc0ba3e29e4fa9a5a60f322502e0a9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.134045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dbc0ba3e29e4fa9a5a60f322502e0a9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.142356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.201520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dbc0ba3e29e4fa9a5a60f322502e0a9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.210222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e72551-367d-4c26-b9fc-c5bdd1db01f5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.217347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.218394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.218521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dbc0ba3e29e4fa9a5a60f322502e0a9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.218873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.219278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] PATH_INFO: `/v3/projects/06fddef025ac4c76aba4443a4f467108/users/d11a5517ab2e4991bc00b29ae4d6cd7f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.221867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06fddef025ac4c76aba4443a4f467108, user_id=d11a5517ab2e4991bc00b29ae4d6cd7f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.225623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.231584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dbc0ba3e29e4fa9a5a60f322502e0a9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.232298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dbc0ba3e29e4fa9a5a60f322502e0a9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.232962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe5ecf62-49be-489c-9102-b3d101a3c3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dbc0ba3e29e4fa9a5a60f322502e0a9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:50.251947 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 940/1891] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:50 2022] PUT /identity/v3/projects/06fddef025ac4c76aba4443a4f467108/users/d11a5517ab2e4991bc00b29ae4d6cd7f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:50.251947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.251947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.256324 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 952/1892] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:50 2022] DELETE /identity/v3/users/1dbc0ba3e29e4fa9a5a60f322502e0a9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:50.258779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33e78d-2b11-4fb9-a9f1-621d6fe615fc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.259104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33e78d-2b11-4fb9-a9f1-621d6fe615fc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.259402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33e78d-2b11-4fb9-a9f1-621d6fe615fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.320475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ef33e78d-2b11-4fb9-a9f1-621d6fe615fc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:50.321974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ef33e78d-2b11-4fb9-a9f1-621d6fe615fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.368290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7a891-c3c6-431a-bdbe-0dfc44d11652 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.409096 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 953/1893] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:50.410769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.411195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.413187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.449548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] MFA Rules not processed for user `d11a5517ab2e4991bc00b29ae4d6cd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:50.450991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.452318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c6748-11d7-42cd-b779-765b89e65874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.453215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6748-11d7-42cd-b779-765b89e65874 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.453664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6748-11d7-42cd-b779-765b89e65874 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.454297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6748-11d7-42cd-b779-765b89e65874 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.455214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c6748-11d7-42cd-b779-765b89e65874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.457064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c6748-11d7-42cd-b779-765b89e65874 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.458051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.474244 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 941/1894] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:50.475541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732cca6-7a8c-49bf-a039-634c084e6a09 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.475824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732cca6-7a8c-49bf-a039-634c084e6a09 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.476094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732cca6-7a8c-49bf-a039-634c084e6a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.495476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b732cca6-7a8c-49bf-a039-634c084e6a09 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:50.496771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b732cca6-7a8c-49bf-a039-634c084e6a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.504878 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 942/1895] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:50.505947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c139-34ba-47f8-80e5-4a0d7f41ff8d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.506230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c139-34ba-47f8-80e5-4a0d7f41ff8d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.506497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c139-34ba-47f8-80e5-4a0d7f41ff8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.526345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-90c1c139-34ba-47f8-80e5-4a0d7f41ff8d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:50.527613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-90c1c139-34ba-47f8-80e5-4a0d7f41ff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.535799 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 943/1896] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:50.540808 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 954/1897] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:50.540942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c139-34ba-47f8-80e5-4a0d7f41ff8d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.541054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.541054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.550133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c139-34ba-47f8-80e5-4a0d7f41ff8d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.554967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.555578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.555853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.556118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.556762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.558520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.571844 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 944/1898] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:50.571844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.579502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e8f54-a393-4314-801e-ee4f896a9cc9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.584581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.585200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.585509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.585792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.586430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:50.586734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.588343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.594677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcbdcb-398b-4ce4-abb8-86e72d22949a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.598488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.599090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.599362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.599638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.601826 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 945/1899] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:50.602451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.606396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.612119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.613638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.622997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 955/1900] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:50.623762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.627940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.632615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae16936-8c97-4ff8-824b-5ad3b76c6e75 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.637755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.638479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.638751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.639057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] PATH_INFO: `/v3/projects/2f91897b74fe42d5b9e60f347a56ee9b/users/19067517f42f4ac5869e1c3d3b840e41/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.660251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f91897b74fe42d5b9e60f347a56ee9b, user_id=19067517f42f4ac5869e1c3d3b840e41, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.663091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.680309 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 956/1901] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:50 2022] PUT /identity/v3/projects/2f91897b74fe42d5b9e60f347a56ee9b/users/19067517f42f4ac5869e1c3d3b840e41/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:50.681814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.694676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c2d32-9561-457f-a40a-550908e3c73f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.762727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6d97f-7715-46bb-9f42-892cd93715fe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.767979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939d2ac-4328-4235-a009-94b9c35735ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.768705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939d2ac-4328-4235-a009-94b9c35735ef admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.769034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939d2ac-4328-4235-a009-94b9c35735ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.769495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939d2ac-4328-4235-a009-94b9c35735ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.770242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939d2ac-4328-4235-a009-94b9c35735ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.772374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939d2ac-4328-4235-a009-94b9c35735ef admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.787076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.788314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.788759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.789265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.811225 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 957/1902] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:50.813435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdeff2-57c8-4600-a677-8e86a2b3fb51 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.814533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdeff2-57c8-4600-a677-8e86a2b3fb51 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.815256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdeff2-57c8-4600-a677-8e86a2b3fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.825356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.860066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f6fdeff2-57c8-4600-a677-8e86a2b3fb51 None None] MFA Rules not processed for user `19067517f42f4ac5869e1c3d3b840e41`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:50.862244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f6fdeff2-57c8-4600-a677-8e86a2b3fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.875547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.879007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.906007 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 946/1903] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:50.909756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.933145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53567cb6-29d1-4432-9d95-d593f479e7bd service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.954829 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 958/1904] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:50.954829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549cf1-2f33-49d4-b8f6-8f76fa19a795 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.954829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549cf1-2f33-49d4-b8f6-8f76fa19a795 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.954829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549cf1-2f33-49d4-b8f6-8f76fa19a795 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.955997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 959/1905] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:50.956776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549cf1-2f33-49d4-b8f6-8f76fa19a795 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.017693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.031929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.032548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.032829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.033103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.033774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.035697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.040750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549cf1-2f33-49d4-b8f6-8f76fa19a795 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.045411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.046074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.046371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.046688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.047329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.049351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.072603 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 947/1906] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:51.073590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.075247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.093680 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 960/1907] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:51.093680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:51.093880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74d1c-2d23-4ece-afbe-ec75498f4767 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.147103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.147745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.148017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.148286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] PATH_INFO: `/v3/projects/c8a32c8816514b6aa7aa27895e8c4dd1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.149396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a32c8816514b6aa7aa27895e8c4dd1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.151347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.162062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.174581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.200848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a32c8816514b6aa7aa27895e8c4dd1', 'request_id': 'req-7061ee8a-fa49-4a87-8623-1dabdffea9da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.207084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a32c8816514b6aa7aa27895e8c4dd1', 'request_id': 'req-7061ee8a-fa49-4a87-8623-1dabdffea9da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.215694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a32c8816514b6aa7aa27895e8c4dd1', 'request_id': 'req-7061ee8a-fa49-4a87-8623-1dabdffea9da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.226820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] The token cache is being invalidate because project c8a32c8816514b6aa7aa27895e8c4dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:51.227107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.227747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.229292 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 948/1908] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:51 2022] DELETE /identity/v3/projects/c8a32c8816514b6aa7aa27895e8c4dd1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:51.230088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.231424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.233325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e59f2-e9cd-41ae-9824-7f5a68f9cb14 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.295496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.296137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.296418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.296711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.297877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.300008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.300674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061ee8a-fa49-4a87-8623-1dabdffea9da admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.301864 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 961/1909] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:51.302621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.304178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.308733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.309385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.309709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.309981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.310631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.312515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.325685 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 949/1910] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:51.325839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8c6a7-681f-46c4-ad69-519d57d2198b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.325948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.327223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.336438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.337610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.338622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.339603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.340256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.341270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.343585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031b5ec-9def-46ab-a838-2a2668a26feb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.344070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.349610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.350450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.350843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.351254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] PATH_INFO: `/v3/users/5ab8354581a24c5887fc5d9afb72bd08` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.352804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab8354581a24c5887fc5d9afb72bd08)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.354967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.370732 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 962/1911] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:51 2022] POST /identity/v3/users => generated 295 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:51.371937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.382129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04d855-3f20-4015-9e02-55ee37e42c7c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.389100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bea2dc-c881-44a5-b0c8-48cc46d59e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.389765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bea2dc-c881-44a5-b0c8-48cc46d59e4f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.390041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bea2dc-c881-44a5-b0c8-48cc46d59e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.390307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bea2dc-c881-44a5-b0c8-48cc46d59e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.391189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bea2dc-c881-44a5-b0c8-48cc46d59e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.393252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bea2dc-c881-44a5-b0c8-48cc46d59e4f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.408942 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 963/1912] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:51.408942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42c67-40ea-4046-ae22-f520e48f26b0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.408942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42c67-40ea-4046-ae22-f520e48f26b0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.408942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f42c67-40ea-4046-ae22-f520e48f26b0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.409382 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 964/1913] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:51.410146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42c67-40ea-4046-ae22-f520e48f26b0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.426778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f42c67-40ea-4046-ae22-f520e48f26b0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.428785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab8354581a24c5887fc5d9afb72bd08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.429666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab8354581a24c5887fc5d9afb72bd08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.432910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.434225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.434637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.434968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.435686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.437747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.438984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab8354581a24c5887fc5d9afb72bd08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.439825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab8354581a24c5887fc5d9afb72bd08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.453675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab8354581a24c5887fc5d9afb72bd08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.456938 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 965/1914] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:51.456938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.456938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.473060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fd583-d87f-46b3-9158-eb56cb6f7464 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.473533 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 950/1915] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:51 2022] DELETE /identity/v3/users/5ab8354581a24c5887fc5d9afb72bd08 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:51.474540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:51.476379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.553591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.554442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.554719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.555022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] PATH_INFO: `/v3/projects/e0daa58ea49e4b2dbf7a2081940572cc/users/f9354d166ad84d42b03e222288d5a717/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.569158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.571051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.574172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0daa58ea49e4b2dbf7a2081940572cc, user_id=f9354d166ad84d42b03e222288d5a717, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.577965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.602930 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 966/1916] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:51 2022] PUT /identity/v3/projects/e0daa58ea49e4b2dbf7a2081940572cc/users/f9354d166ad84d42b03e222288d5a717/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:51.612468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.612468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.617861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.666783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de41162-f38a-4b07-8515-2a0570486919 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.696149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.719814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13e28c-be6e-43d9-9c07-138da9dc9c40 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.726620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.727525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.727964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.728349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] PATH_INFO: `/v3/projects/1f2f1aec1aea493d957becfaa2ac27d0/users/ad0c03ce425d4e4caf000253c0bb526e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.744768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2f1aec1aea493d957becfaa2ac27d0, user_id=ad0c03ce425d4e4caf000253c0bb526e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.747859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.756581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee8017-88f4-4897-a8f2-50b9edeb3c1a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.757352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8017-88f4-4897-a8f2-50b9edeb3c1a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.757757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8017-88f4-4897-a8f2-50b9edeb3c1a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.758101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8017-88f4-4897-a8f2-50b9edeb3c1a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.759494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee8017-88f4-4897-a8f2-50b9edeb3c1a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.761842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee8017-88f4-4897-a8f2-50b9edeb3c1a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.764484 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 951/1917] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:51.765513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dda02e-1bdb-49cf-874f-f33f6fbed97a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.765866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dda02e-1bdb-49cf-874f-f33f6fbed97a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.766206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dda02e-1bdb-49cf-874f-f33f6fbed97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.781882 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 967/1918] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:51 2022] PUT /identity/v3/projects/1f2f1aec1aea493d957becfaa2ac27d0/users/ad0c03ce425d4e4caf000253c0bb526e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:51.785149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.827411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-00dda02e-1bdb-49cf-874f-f33f6fbed97a None None] MFA Rules not processed for user `f9354d166ad84d42b03e222288d5a717`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:51.829212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-00dda02e-1bdb-49cf-874f-f33f6fbed97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.884882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8248d-54ed-42d4-b1c3-465025defcc4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.891236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.891979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.892816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.893193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.894029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.895896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.906960 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 968/1919] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:51.908532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.912437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.917179 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 952/1920] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:51.918906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d565f-e7c7-4a44-ba26-c80d40dd6d22 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.919225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d565f-e7c7-4a44-ba26-c80d40dd6d22 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.919535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d565f-e7c7-4a44-ba26-c80d40dd6d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.928922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2ae86-7c22-42ba-a176-786c0014526f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.933997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.935123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.935403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.935678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] PATH_INFO: `/v3/projects/115f5ac7e1164643a0737a0836d4d8ab` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.936841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=115f5ac7e1164643a0737a0836d4d8ab)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.938809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.939923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-571d565f-e7c7-4a44-ba26-c80d40dd6d22 None None] MFA Rules not processed for user `ee3d8ef713bd4844833d9374766452d8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:51.941111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-571d565f-e7c7-4a44-ba26-c80d40dd6d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.990172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115f5ac7e1164643a0737a0836d4d8ab', 'request_id': 'req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:51.999234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 953/1921] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:51.999403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115f5ac7e1164643a0737a0836d4d8ab', 'request_id': 'req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:52.000073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d565f-e7c7-4a44-ba26-c80d40dd6d22 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.012158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115f5ac7e1164643a0737a0836d4d8ab', 'request_id': 'req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:52.021114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d565f-e7c7-4a44-ba26-c80d40dd6d22 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.028896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] The token cache is being invalidate because project 115f5ac7e1164643a0737a0836d4d8ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:52.029262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:52.030344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:52.031797 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 969/1922] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:51 2022] DELETE /identity/v3/projects/115f5ac7e1164643a0737a0836d4d8ab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:52.034347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.035825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.048303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.088971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.098549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.099557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.100018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.100432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] PATH_INFO: `/v3/projects/1f2f1aec1aea493d957becfaa2ac27d0/users/ad0c03ce425d4e4caf000253c0bb526e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.102655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2f1aec1aea493d957becfaa2ac27d0, user_id=ad0c03ce425d4e4caf000253c0bb526e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.105361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.110850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a3f0f-61f1-44e3-89c1-8bb2459eb011 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.116510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 954/1923] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:51 2022] PUT /identity/v3/projects/1f2f1aec1aea493d957becfaa2ac27d0/users/ad0c03ce425d4e4caf000253c0bb526e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:52.117563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.119234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.185770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d352-3abc-46e6-a071-4f46a98e1052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.186400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d352-3abc-46e6-a071-4f46a98e1052 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.186665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d352-3abc-46e6-a071-4f46a98e1052 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.186932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d352-3abc-46e6-a071-4f46a98e1052 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.187612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d352-3abc-46e6-a071-4f46a98e1052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.189502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d352-3abc-46e6-a071-4f46a98e1052 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.202512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46636f56-bc1a-41b5-99f4-64b65baa8459 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.209111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.209823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.210102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.210377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.212241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.221643 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 970/1924] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/projects => generated 389 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:52.221643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35d34-ed65-4617-8f74-93cad4e60172 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.221643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35d34-ed65-4617-8f74-93cad4e60172 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.221643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35d34-ed65-4617-8f74-93cad4e60172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.245840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1ec35d34-ed65-4617-8f74-93cad4e60172 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:52.247584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1ec35d34-ed65-4617-8f74-93cad4e60172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.316810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.318452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.327449 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 971/1925] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:52.328776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df38c-49d9-4ec7-a99c-f0e717341edd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.329104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df38c-49d9-4ec7-a99c-f0e717341edd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.329380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df38c-49d9-4ec7-a99c-f0e717341edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.331795 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 955/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:52.332871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.353092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646cb7f-b5ae-45d7-840f-fe69ef89751c service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.359765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb9c7-f6e4-47b9-a549-8668967c15e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.360395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb9c7-f6e4-47b9-a549-8668967c15e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.360678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb9c7-f6e4-47b9-a549-8668967c15e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.360945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb9c7-f6e4-47b9-a549-8668967c15e3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.361676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb9c7-f6e4-47b9-a549-8668967c15e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.363722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb9c7-f6e4-47b9-a549-8668967c15e3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.379573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9a5df38c-49d9-4ec7-a99c-f0e717341edd None None] MFA Rules not processed for user `ad0c03ce425d4e4caf000253c0bb526e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:52.381182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9a5df38c-49d9-4ec7-a99c-f0e717341edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.412353 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 956/1927] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/users => generated 301 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:52.412825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c8fd2-10e9-405b-acee-eb3d0c5074c3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.413112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c8fd2-10e9-405b-acee-eb3d0c5074c3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.413399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c8fd2-10e9-405b-acee-eb3d0c5074c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.430449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.465751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8b3c8fd2-10e9-405b-acee-eb3d0c5074c3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:52.467054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8b3c8fd2-10e9-405b-acee-eb3d0c5074c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.485353 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 957/1928] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:52.499941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c8fd2-10e9-405b-acee-eb3d0c5074c3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.513918 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 972/1929] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:52.523556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c8fd2-10e9-405b-acee-eb3d0c5074c3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.528596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df38c-49d9-4ec7-a99c-f0e717341edd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.538356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.539263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.539545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.540120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.541085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.542789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5df38c-49d9-4ec7-a99c-f0e717341edd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.543073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.548235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.548867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.550457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.550457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.550457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:52.550457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.552199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.566295 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 973/1930] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:52.567126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.571160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.576421 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 958/1931] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:52.576421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.586015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bd5bf-df8f-42c2-9d51-1c7e61e86523 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.595063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.595710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.595994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.596287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.596937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.598997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.648974 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 959/1932] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:52.649862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.664911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67084d3c-9acd-4594-9648-f9a0a652af24 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.714032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.748114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.749013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.749428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.749726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.755906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a04b77-b40c-40a8-a544-99634de9206e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.762336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.763022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.763363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.763643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] PATH_INFO: `/v3/projects/8aafc4a713d94652bd4ee89c1448fc39/users/05052d8e3de14ff19f64a71c8608c064/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.782390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aafc4a713d94652bd4ee89c1448fc39, user_id=05052d8e3de14ff19f64a71c8608c064, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.785951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.808063 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 960/1933] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:52 2022] PUT /identity/v3/projects/8aafc4a713d94652bd4ee89c1448fc39/users/05052d8e3de14ff19f64a71c8608c064/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:52.809183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.813798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.830597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.832236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.849900 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 974/1934] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:52.851074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.864900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2dd80-9988-4734-a20d-b763ad22d3cf service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.905282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143b19a-79bb-4e5f-b73a-b87b78c4c6f1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.910904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.911535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.911813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.912083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.912724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.914703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.953353 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 961/1935] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:52.961623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.980229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201c728-fc5d-4616-b232-b2764adef5fb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.980393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75838a29-1379-40d6-a499-aa54c118cf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.981048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75838a29-1379-40d6-a499-aa54c118cf86 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.981332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75838a29-1379-40d6-a499-aa54c118cf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.981641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75838a29-1379-40d6-a499-aa54c118cf86 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.982299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75838a29-1379-40d6-a499-aa54c118cf86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.983952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75838a29-1379-40d6-a499-aa54c118cf86 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.997060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.998178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 975/1936] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:52.998625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.998625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.998903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.999490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.999595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c907c3-9ac3-4699-b3d1-9bf6384206f1 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.999850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c907c3-9ac3-4699-b3d1-9bf6384206f1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.000136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c907c3-9ac3-4699-b3d1-9bf6384206f1 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.001164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.005915 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 976/1937] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:53.006847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c907c3-9ac3-4699-b3d1-9bf6384206f1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.018900 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 962/1938] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:53.019835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:53.021023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c907c3-9ac3-4699-b3d1-9bf6384206f1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.028478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.029384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.030193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.031340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] PATH_INFO: `/v3/projects/8aafc4a713d94652bd4ee89c1448fc39/users/05052d8e3de14ff19f64a71c8608c064/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.034704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aafc4a713d94652bd4ee89c1448fc39, user_id=05052d8e3de14ff19f64a71c8608c064, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.039914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.052535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 977/1939] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:53 2022] PUT /identity/v3/projects/8aafc4a713d94652bd4ee89c1448fc39/users/05052d8e3de14ff19f64a71c8608c064/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:53.064228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.170325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8611-9a92-4f05-8208-e5881dfc98e0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.177179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdfdc5-826f-425a-aa61-8f7a5200bcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.179152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdfdc5-826f-425a-aa61-8f7a5200bcc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.180653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdfdc5-826f-425a-aa61-8f7a5200bcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.181179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdfdc5-826f-425a-aa61-8f7a5200bcc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.182754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdfdc5-826f-425a-aa61-8f7a5200bcc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.185488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdfdc5-826f-425a-aa61-8f7a5200bcc5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.196942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.208375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:53.214072 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 978/1940] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:53.216187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c6f8-1117-428a-8d43-4403919908c7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.216656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c6f8-1117-428a-8d43-4403919908c7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.217108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c6f8-1117-428a-8d43-4403919908c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.258083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c0e5c6f8-1117-428a-8d43-4403919908c7 None None] MFA Rules not processed for user `05052d8e3de14ff19f64a71c8608c064`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:53.260205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c0e5c6f8-1117-428a-8d43-4403919908c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:53.267021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:53.334559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:53.345652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cb630-2dfd-4b7b-a912-a0027a776dad admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.360673 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 979/1941] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:53 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:53.363168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5c6f8-1117-428a-8d43-4403919908c7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.388736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5c6f8-1117-428a-8d43-4403919908c7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.404211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0c986-5ba3-47e9-b31e-ef4c1a97b601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.404918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0c986-5ba3-47e9-b31e-ef4c1a97b601 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.405432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0c986-5ba3-47e9-b31e-ef4c1a97b601 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.405578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0c986-5ba3-47e9-b31e-ef4c1a97b601 admin admin] PATH_INFO: `/v3/projects/e605ef8a63f64d6a8d1d22f00b3d4a63/users/c646ca44d1874d07b25c806691d2f1b8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.426846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0c986-5ba3-47e9-b31e-ef4c1a97b601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e605ef8a63f64d6a8d1d22f00b3d4a63, user_id=c646ca44d1874d07b25c806691d2f1b8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.432683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0c986-5ba3-47e9-b31e-ef4c1a97b601 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.454730 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 980/1942] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:53 2022] PUT /identity/v3/projects/e605ef8a63f64d6a8d1d22f00b3d4a63/users/c646ca44d1874d07b25c806691d2f1b8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:53.457733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.458527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.458816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.459504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.460300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.462719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.465228 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 963/1943] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:53.466388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab90f4a-8814-4543-99ed-aaaca85b60db None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.466681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab90f4a-8814-4543-99ed-aaaca85b60db None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.466959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab90f4a-8814-4543-99ed-aaaca85b60db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.500003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2ab90f4a-8814-4543-99ed-aaaca85b60db None None] MFA Rules not processed for user `c646ca44d1874d07b25c806691d2f1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:53.501509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2ab90f4a-8814-4543-99ed-aaaca85b60db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:53.597213 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 981/1944] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:50:53.602569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.616745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ee326-24f3-45f5-b890-3c320851716b tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.679414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab90f4a-8814-4543-99ed-aaaca85b60db None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.783838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.787287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.787624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.787901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.788589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.791405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.841861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab90f4a-8814-4543-99ed-aaaca85b60db None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.852906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aab1c-1104-491c-8bf2-940f9cdf03c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.856478 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 964/1945] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:53 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:53.857578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aab1c-1104-491c-8bf2-940f9cdf03c1 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.857716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.858123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aab1c-1104-491c-8bf2-940f9cdf03c1 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.858562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aab1c-1104-491c-8bf2-940f9cdf03c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.936825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:53.956495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a8441-2c1a-4ef8-8530-bce5395f355f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.956707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aab1c-1104-491c-8bf2-940f9cdf03c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.958490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aab1c-1104-491c-8bf2-940f9cdf03c1 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.963288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.965129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.966507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.967151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.968266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.969617 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 982/1946] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:50:53.970915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3697c04-a8d5-4f85-9ea3-fb6a42c42d56 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.971205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3697c04-a8d5-4f85-9ea3-fb6a42c42d56 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.971482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3697c04-a8d5-4f85-9ea3-fb6a42c42d56 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.971999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.972990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 983/1947] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:53.978408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3697c04-a8d5-4f85-9ea3-fb6a42c42d56 None None] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:53.983061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:54.007399 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 965/1948] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:53 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:50:54.010626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.023025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c23c9-76c3-4bcc-b1d6-85039933c6cd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.030210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.031291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.031668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.032201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.033145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.035082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.048464 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 966/1949] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:50:54.052883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.068461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc75173-9174-491b-95b2-5bf5196702e7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.075917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.077361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.077671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.077945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] PATH_INFO: `/v3/projects/6c6ebac696a940c0946c16dd7d8ca45d/users/84d9881677454ff38ad03df3af31fc83/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.098782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6ebac696a940c0946c16dd7d8ca45d, user_id=84d9881677454ff38ad03df3af31fc83, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.101510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.121650 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 967/1950] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:54 2022] PUT /identity/v3/projects/6c6ebac696a940c0946c16dd7d8ca45d/users/84d9881677454ff38ad03df3af31fc83/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:54.123059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3697c04-a8d5-4f85-9ea3-fb6a42c42d56 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.123779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.214872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71964e00-3762-49a1-a0fb-79f3b2782669 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.217911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3697c04-a8d5-4f85-9ea3-fb6a42c42d56 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.221225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.222445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.222805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.223271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.226559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:54.295466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.296215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.296564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.296911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.298302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.300483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.302428 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 984/1951] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:54.303182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.308958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.311079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.337320 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 968/1952] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:54.344111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:54.375855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80cd7d-78a2-4b49-aef4-2a5d834c504a tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.380724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e0f93-9c8d-4c4c-9eb6-d5cccece93f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.381506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e0f93-9c8d-4c4c-9eb6-d5cccece93f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.381865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e0f93-9c8d-4c4c-9eb6-d5cccece93f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.382205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e0f93-9c8d-4c4c-9eb6-d5cccece93f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.382923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e0f93-9c8d-4c4c-9eb6-d5cccece93f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.384708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e0f93-9c8d-4c4c-9eb6-d5cccece93f3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.396727 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 985/1953] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:54.400739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.412266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7a736-309c-4221-8edc-28ffd4f0bb5e None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.412558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7a736-309c-4221-8edc-28ffd4f0bb5e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.412977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7a736-309c-4221-8edc-28ffd4f0bb5e None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.414794 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 986/1954] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:50:54.419485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c9158-d780-430d-899e-0ff29ef753b1 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.421311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7a736-309c-4221-8edc-28ffd4f0bb5e None None] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:54.432588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9684a-17e6-442c-be9b-0a486bc5a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.433391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9684a-17e6-442c-be9b-0a486bc5a144 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.433737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9684a-17e6-442c-be9b-0a486bc5a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.434142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9684a-17e6-442c-be9b-0a486bc5a144 admin admin] PATH_INFO: `/v3/projects/6c6ebac696a940c0946c16dd7d8ca45d/users/84d9881677454ff38ad03df3af31fc83/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.439094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7a736-309c-4221-8edc-28ffd4f0bb5e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.444431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9684a-17e6-442c-be9b-0a486bc5a144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6ebac696a940c0946c16dd7d8ca45d, user_id=84d9881677454ff38ad03df3af31fc83, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.451935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9684a-17e6-442c-be9b-0a486bc5a144 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.458133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7a736-309c-4221-8edc-28ffd4f0bb5e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.471579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb3bd3-3a21-4e06-ab35-918c5e8d28db tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.472261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb3bd3-3a21-4e06-ab35-918c5e8d28db tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.472527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb3bd3-3a21-4e06-ab35-918c5e8d28db tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.472798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb3bd3-3a21-4e06-ab35-918c5e8d28db tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.475756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb3bd3-3a21-4e06-ab35-918c5e8d28db tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.477908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb3bd3-3a21-4e06-ab35-918c5e8d28db tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.479804 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 987/1955] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:54 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:54.496131 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 969/1956] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:54 2022] PUT /identity/v3/projects/6c6ebac696a940c0946c16dd7d8ca45d/users/84d9881677454ff38ad03df3af31fc83/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:54.496811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.497127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.497386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.542135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] MFA Rules not processed for user `84d9881677454ff38ad03df3af31fc83`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:54.544337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:54.633275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395359b-fca3-4719-9403-8b8590cd32aa None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.633275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395359b-fca3-4719-9403-8b8590cd32aa None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.633275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395359b-fca3-4719-9403-8b8590cd32aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.663570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2395359b-fca3-4719-9403-8b8590cd32aa None None] MFA Rules not processed for user `ee3d8ef713bd4844833d9374766452d8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:54.664997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2395359b-fca3-4719-9403-8b8590cd32aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:54.710570 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 988/1957] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:54.712231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.715128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:54.741256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:54.789956 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 970/1958] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:54.800023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395359b-fca3-4719-9403-8b8590cd32aa None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.823558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f48f1-54f1-459a-90aa-1faaf44becca None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.829133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.829295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.829859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.830142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] PATH_INFO: `/v3/users/19067517f42f4ac5869e1c3d3b840e41` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.831327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19067517f42f4ac5869e1c3d3b840e41)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.833285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.897870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:54.900491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19067517f42f4ac5869e1c3d3b840e41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:54.920116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395359b-fca3-4719-9403-8b8590cd32aa None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.922469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19067517f42f4ac5869e1c3d3b840e41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:54.930796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.931843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.932928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.933244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.933627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19067517f42f4ac5869e1c3d3b840e41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:54.934473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19067517f42f4ac5869e1c3d3b840e41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:54.935659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c30fb7-298c-410c-b57c-84f5b2f003a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19067517f42f4ac5869e1c3d3b840e41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:54.943928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.945834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.951348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 989/1959] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:54 2022] DELETE /identity/v3/users/19067517f42f4ac5869e1c3d3b840e41 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:54.955807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab4e4-77ea-432c-bfe3-fce958b23852 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.956104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab4e4-77ea-432c-bfe3-fce958b23852 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.956381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab4e4-77ea-432c-bfe3-fce958b23852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.962036 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 971/1960] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:50:54.983018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a7dab4e4-77ea-432c-bfe3-fce958b23852 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:54.984332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7dab4e4-77ea-432c-bfe3-fce958b23852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.058751 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 990/1961] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:50:55.115286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.117911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.193353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce1594-8dd3-4619-adde-f41754d12c9d service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.262193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e4cba-8911-4fcd-82ee-1078d0bb5db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.262834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e4cba-8911-4fcd-82ee-1078d0bb5db9 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.263115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e4cba-8911-4fcd-82ee-1078d0bb5db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.263395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e4cba-8911-4fcd-82ee-1078d0bb5db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.278560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e4cba-8911-4fcd-82ee-1078d0bb5db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.280714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e4cba-8911-4fcd-82ee-1078d0bb5db9 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.294752 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 972/1962] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:50:55.299152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdfffb-b9f0-4a70-bdd7-f9882234ebc5 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.299624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdfffb-b9f0-4a70-bdd7-f9882234ebc5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.299703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdfffb-b9f0-4a70-bdd7-f9882234ebc5 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.301504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 991/1963] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:55.313969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1389c-16db-4105-a11b-2caaa002ae82 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.314260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1389c-16db-4105-a11b-2caaa002ae82 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.314576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1389c-16db-4105-a11b-2caaa002ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.346853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b9a1389c-16db-4105-a11b-2caaa002ae82 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:55.349898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b9a1389c-16db-4105-a11b-2caaa002ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.376847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdfffb-b9f0-4a70-bdd7-f9882234ebc5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.379590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8fdfffb-b9f0-4a70-bdd7-f9882234ebc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.408261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.498008 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 973/1964] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:50:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:55.499825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.500115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.500385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.502318 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 974/1965] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:55.509667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:55.511574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.528663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fdfffb-b9f0-4a70-bdd7-f9882234ebc5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.538574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.542436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.543500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.544111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.544393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.545797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.549920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.636689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a883bc-8fa9-4011-a4a6-37e8efbb3b80 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.637715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.639589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.646173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.647213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.648470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.649003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.649564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.651884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.655784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.658343 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 975/1966] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:55.665695 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 992/1967] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:55.701383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.701954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.708271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.718410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a40c19-fcc9-4927-9c6a-dd54580783d8 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.721538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8141b-c6bb-487f-acf6-f873119b3879 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.723985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.724627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.724966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.725202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] PATH_INFO: `/v3/projects/2f91897b74fe42d5b9e60f347a56ee9b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.726377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f91897b74fe42d5b9e60f347a56ee9b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.728777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.729689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.730502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.730926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.731341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.742583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.744592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.757538 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 993/1968] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:55.787216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f91897b74fe42d5b9e60f347a56ee9b', 'request_id': 'req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:55.795037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f91897b74fe42d5b9e60f347a56ee9b', 'request_id': 'req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:55.804675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f91897b74fe42d5b9e60f347a56ee9b', 'request_id': 'req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:55.819215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] The token cache is being invalidate because project 2f91897b74fe42d5b9e60f347a56ee9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:55.820191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:55.821013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3b0c003-8232-4f8f-b0ab-a338ce0b76b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:55.826777 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 976/1969] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:55 2022] DELETE /identity/v3/projects/2f91897b74fe42d5b9e60f347a56ee9b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:55.827649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.843156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a0cbb-60dc-4cc3-a2b2-a385824ee92d service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.897638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.897930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.898221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.899616 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 977/1970] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:50:55.906134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:55.909230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.940900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.941717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.942117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.942816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] PATH_INFO: `/v3/users/ad0c03ce425d4e4caf000253c0bb526e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.944138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0c03ce425d4e4caf000253c0bb526e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.947141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.964833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:56.011590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.013043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.032041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0c03ce425d4e4caf000253c0bb526e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.044290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0c03ce425d4e4caf000253c0bb526e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.060669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0c03ce425d4e4caf000253c0bb526e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.061402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0c03ce425d4e4caf000253c0bb526e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.062431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-029afe74-261a-4973-8070-3131bb56805a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0c03ce425d4e4caf000253c0bb526e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.081113 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 994/1971] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:55 2022] DELETE /identity/v3/users/ad0c03ce425d4e4caf000253c0bb526e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:56.113878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9571b-af52-4c36-8635-8452c2923824 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.187067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.188223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.188522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.188966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.190628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.193558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.201716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 978/1972] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:56.293530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1fae-0063-439c-85d1-527ec3c0a91c None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.294287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1fae-0063-439c-85d1-527ec3c0a91c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.294287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1fae-0063-439c-85d1-527ec3c0a91c None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.295434 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 995/1973] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:56.300619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:56.302618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.406931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.409029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.427498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0f5e2-a44b-420c-9aaf-3a9d51b44ff1 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.440623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.443194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.446113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.446551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.448095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.450982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.453528 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 979/1974] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:56 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:56.556304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.556940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.557019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.578894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] MFA Rules not processed for user `ee3d8ef713bd4844833d9374766452d8`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:56.580225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.608991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.611263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.635813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:56.682997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 996/1975] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:56.755047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b644866-b630-4120-9720-53468b483bb9 tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.766179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.766365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.767173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.767515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.767657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.767903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] PATH_INFO: `/v3/projects/1f2f1aec1aea493d957becfaa2ac27d0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.769968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2f1aec1aea493d957becfaa2ac27d0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.780039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.792610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347bdaa-bce0-45a2-975b-56503b37a7ae None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.801817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.802476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.802757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.803032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] PATH_INFO: `/v3/users/f9354d166ad84d42b03e222288d5a717` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.808562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9354d166ad84d42b03e222288d5a717)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.810691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.824547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:56.858516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2f1aec1aea493d957becfaa2ac27d0', 'request_id': 'req-68c6d484-afad-4320-9e0f-f63baecd7a29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.866804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2f1aec1aea493d957becfaa2ac27d0', 'request_id': 'req-68c6d484-afad-4320-9e0f-f63baecd7a29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.879000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2f1aec1aea493d957becfaa2ac27d0', 'request_id': 'req-68c6d484-afad-4320-9e0f-f63baecd7a29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.894902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] The token cache is being invalidate because project 1f2f1aec1aea493d957becfaa2ac27d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:56.895398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.896194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68c6d484-afad-4320-9e0f-f63baecd7a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.898229 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 980/1976] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:56 2022] DELETE /identity/v3/projects/1f2f1aec1aea493d957becfaa2ac27d0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:56.904833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.905109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.905565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.913426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9354d166ad84d42b03e222288d5a717'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.927455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:56.929865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.942368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9354d166ad84d42b03e222288d5a717'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.956548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9354d166ad84d42b03e222288d5a717'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.957535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9354d166ad84d42b03e222288d5a717'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.959621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d747b901-fec4-447e-91f3-48799e34f10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9354d166ad84d42b03e222288d5a717'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:56.981088 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 997/1977] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:56 2022] DELETE /identity/v3/users/f9354d166ad84d42b03e222288d5a717 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:56.982861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311b0d-1cdc-4e96-9b35-022bf97e4016 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.983760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311b0d-1cdc-4e96-9b35-022bf97e4016 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.984223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311b0d-1cdc-4e96-9b35-022bf97e4016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.020780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-93311b0d-1cdc-4e96-9b35-022bf97e4016 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:57.022627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-93311b0d-1cdc-4e96-9b35-022bf97e4016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.046166 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 981/1978] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:57.046964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.048254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.061193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426cf1-c945-4983-ad7d-46ca0ea4e90e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.108552 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 998/1979] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:57.109996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.110305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.110566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.122663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:57.146569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:57.147384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.148029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.148147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.148303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.148574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] PATH_INFO: `/v3/users/c646ca44d1874d07b25c806691d2f1b8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.149738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c646ca44d1874d07b25c806691d2f1b8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.151696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.157502 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 999/1980] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:57.159248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.160769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.208126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c646ca44d1874d07b25c806691d2f1b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.210055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c646ca44d1874d07b25c806691d2f1b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.219278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c646ca44d1874d07b25c806691d2f1b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.220026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c646ca44d1874d07b25c806691d2f1b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.227718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b539c-914f-41d4-ab15-8ac0b7386402 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.232337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30025-8bea-4802-8039-9386816535e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.233124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30025-8bea-4802-8039-9386816535e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.233397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30025-8bea-4802-8039-9386816535e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.233717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30025-8bea-4802-8039-9386816535e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.235466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c646ca44d1874d07b25c806691d2f1b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.235597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d0c30025-8bea-4802-8039-9386816535e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.247810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30025-8bea-4802-8039-9386816535e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.249251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30025-8bea-4802-8039-9386816535e1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.258977 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1000/1981] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:57.260426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9243c7-5543-44b8-842e-304121ae8ce1 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.260698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9243c7-5543-44b8-842e-304121ae8ce1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.261093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9243c7-5543-44b8-842e-304121ae8ce1 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.261207 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 982/1982] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:57 2022] DELETE /identity/v3/users/c646ca44d1874d07b25c806691d2f1b8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:57.262117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.263488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.266307 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1001/1983] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:50:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:57.267403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea9ee-f0e4-4d47-9e8c-e78faa4ae29d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.267695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea9ee-f0e4-4d47-9e8c-e78faa4ae29d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.267970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea9ee-f0e4-4d47-9e8c-e78faa4ae29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.288524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-10cea9ee-f0e4-4d47-9e8c-e78faa4ae29d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:57.289776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10cea9ee-f0e4-4d47-9e8c-e78faa4ae29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.306397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:57.351068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b5704-729a-41a2-9234-a8cb55a4a5e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.356227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.356858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.357153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.357419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.358110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:57.358417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.360072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.373187 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 983/1984] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:57.374001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:50:57.377997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.378562 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1002/1985] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:57.379687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cea9ee-f0e4-4d47-9e8c-e78faa4ae29d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.409142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cea9ee-f0e4-4d47-9e8c-e78faa4ae29d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.417079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.417834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.418120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.418400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.420588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.423189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.459641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:57.468451 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1003/1986] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:57 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:57.469673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.471683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.486189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.489237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.559575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e6120-477c-4d6a-b4f4-f2f3e2c9a3f0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.567107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.567726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.568029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.568267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.606314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02095ad-3c78-4053-8723-2990a7c33923 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.660975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.663290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.675536 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1004/1987] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:57.676441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.689686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62aa69-31c8-4be0-8075-8e615b5d6753 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.696210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.696834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.697112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.697383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.698073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.700100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.709590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11a5517ab2e4991bc00b29ae4d6cd7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06fddef025ac4c76aba4443a4f467108', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.710824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.711484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.711758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.713900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorizing `identity:get_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.716667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.718419 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 984/1988] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 381 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:57.719159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.721123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.728250 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1005/1989] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:57 2022] POST /identity/v3/users => generated 289 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:57.732464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.745164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453d284-f93d-4ec9-a481-9deed1e3e9da admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.750227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.750847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.751175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.751427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.752396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.754932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405333c0-0043-4fd0-b1b7-1ed3d2f1f6cb tempest-QuotasAdminNegativeTest-1218234253 tempest-QuotasAdminNegativeTest-1218234253-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.764745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.766749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.768297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.768595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.770022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] PATH_INFO: `/v3/projects/e0daa58ea49e4b2dbf7a2081940572cc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.773158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0daa58ea49e4b2dbf7a2081940572cc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.778591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.782648 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1006/1990] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:57.784557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.800441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7ad03-fb29-4147-90ff-f18a32712cdc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.813785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.814675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.815067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.815448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.816218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.822734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.841051 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1007/1991] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:50:57.842137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.854612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10a383-01a7-446d-8c5c-9d6a03bd2f67 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.859502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0daa58ea49e4b2dbf7a2081940572cc', 'request_id': 'req-ae3f7525-8e79-4360-8d8b-0b567454dade'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.860994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:57.872475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0daa58ea49e4b2dbf7a2081940572cc', 'request_id': 'req-ae3f7525-8e79-4360-8d8b-0b567454dade'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.891086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0daa58ea49e4b2dbf7a2081940572cc', 'request_id': 'req-ae3f7525-8e79-4360-8d8b-0b567454dade'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.918394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] The token cache is being invalidate because project e0daa58ea49e4b2dbf7a2081940572cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:57.918829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.919519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:57.922413 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 985/1992] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:57 2022] DELETE /identity/v3/projects/e0daa58ea49e4b2dbf7a2081940572cc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:57.926295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.928696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:57.991363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.001344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.002156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.002434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.002755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] PATH_INFO: `/v3/projects/f273bfff7ced4f2e885d1ead94a9a486/users/ec6d47d564174aceb110b3354ff7db2a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.070844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f273bfff7ced4f2e885d1ead94a9a486, user_id=ec6d47d564174aceb110b3354ff7db2a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.073993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.111295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f7525-8e79-4360-8d8b-0b567454dade admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.116185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1008/1993] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:57 2022] PUT /identity/v3/projects/f273bfff7ced4f2e885d1ead94a9a486/users/ec6d47d564174aceb110b3354ff7db2a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:58.121524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.123353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.202687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.203740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.204194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.204350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] PATH_INFO: `/v3/users/05052d8e3de14ff19f64a71c8608c064` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.205567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=05052d8e3de14ff19f64a71c8608c064)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.207534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.224937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283c7b8-15b1-445f-a01d-01302bcc8822 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.229879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.230637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.230914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.231322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] PATH_INFO: `/v3/users/ee3d8ef713bd4844833d9374766452d8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.233274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3d8ef713bd4844833d9374766452d8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.235637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.278559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05052d8e3de14ff19f64a71c8608c064'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.279305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05052d8e3de14ff19f64a71c8608c064'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.290208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05052d8e3de14ff19f64a71c8608c064'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.291015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05052d8e3de14ff19f64a71c8608c064'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.301586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3d8ef713bd4844833d9374766452d8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.310103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-edad1d9d-935c-4f8e-b758-40ad8583efbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05052d8e3de14ff19f64a71c8608c064'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.315741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3d8ef713bd4844833d9374766452d8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.325164 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 986/1994] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:57 2022] DELETE /identity/v3/users/05052d8e3de14ff19f64a71c8608c064 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:58.325318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3d8ef713bd4844833d9374766452d8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.326098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3d8ef713bd4844833d9374766452d8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.326270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a62e2-3c46-4f79-8155-7d9f2419850c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.326512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a62e2-3c46-4f79-8155-7d9f2419850c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.326788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a62e2-3c46-4f79-8155-7d9f2419850c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.326885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3d8ef713bd4844833d9374766452d8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.345706 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1009/1995] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:58 2022] DELETE /identity/v3/users/ee3d8ef713bd4844833d9374766452d8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:58.346571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.349355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.364878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ef7a62e2-3c46-4f79-8155-7d9f2419850c None None] MFA Rules not processed for user `ec6d47d564174aceb110b3354ff7db2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:58.365996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.366204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ef7a62e2-3c46-4f79-8155-7d9f2419850c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.455744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.485992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406948b1-230a-4583-b12a-6ad4d0b19448 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.493799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.494798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.495169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.496281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] PATH_INFO: `/v3/projects/e605ef8a63f64d6a8d1d22f00b3d4a63` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.499488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e605ef8a63f64d6a8d1d22f00b3d4a63)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.502167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.511919 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 987/1996] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:58.513858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.514216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.514579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.542129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:58.544932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.572746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e605ef8a63f64d6a8d1d22f00b3d4a63', 'request_id': 'req-c58e3d08-c54e-4e70-b747-2057fb177d49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.587999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e605ef8a63f64d6a8d1d22f00b3d4a63', 'request_id': 'req-c58e3d08-c54e-4e70-b747-2057fb177d49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.611625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e605ef8a63f64d6a8d1d22f00b3d4a63', 'request_id': 'req-c58e3d08-c54e-4e70-b747-2057fb177d49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.634288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] The token cache is being invalidate because project e605ef8a63f64d6a8d1d22f00b3d4a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:58.634893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.635994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.638550 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1010/1997] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:58 2022] DELETE /identity/v3/projects/e605ef8a63f64d6a8d1d22f00b3d4a63 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:58.639510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.641680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.668564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 988/1998] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:58.669765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.671308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.684681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f3374-3832-455b-8391-0eb0a8c1ab51 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.698991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.700045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.700562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.701163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] PATH_INFO: `/v3/users/84d9881677454ff38ad03df3af31fc83` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.703039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d9881677454ff38ad03df3af31fc83)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.706221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.752135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e3d08-c54e-4e70-b747-2057fb177d49 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.757687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.758310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.758590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.758867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.759510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.761508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.779702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d9881677454ff38ad03df3af31fc83'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.780317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d9881677454ff38ad03df3af31fc83'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.790101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d9881677454ff38ad03df3af31fc83'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.790989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d9881677454ff38ad03df3af31fc83'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.794978 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1011/1999] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:58 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:58.795849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.799090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.806928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d9881677454ff38ad03df3af31fc83'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.823485 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 989/2000] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:50:58 2022] DELETE /identity/v3/users/84d9881677454ff38ad03df3af31fc83 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:58.824305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.837487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.848342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.889517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.895561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad0586-f2d1-48c4-9b9f-81a6e48105b7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.910228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4e82f-720a-42c7-b549-d0034746509c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.918735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.919530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.919962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.920305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] PATH_INFO: `/v3/projects/8aafc4a713d94652bd4ee89c1448fc39` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.921670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aafc4a713d94652bd4ee89c1448fc39)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.924781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.967577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.978402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.978688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.978964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.983642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.987705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aafc4a713d94652bd4ee89c1448fc39', 'request_id': 'req-b057eb7e-1abd-45d1-9664-440f507e86a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:58.995132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aafc4a713d94652bd4ee89c1448fc39', 'request_id': 'req-b057eb7e-1abd-45d1-9664-440f507e86a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.001885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.004336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.008200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aafc4a713d94652bd4ee89c1448fc39', 'request_id': 'req-b057eb7e-1abd-45d1-9664-440f507e86a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.018264 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1012/2001] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:50:59.019501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.021311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.024959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] The token cache is being invalidate because project 8aafc4a713d94652bd4ee89c1448fc39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:59.025403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.026360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b057eb7e-1abd-45d1-9664-440f507e86a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.032962 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 990/2002] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:58 2022] DELETE /identity/v3/projects/8aafc4a713d94652bd4ee89c1448fc39 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:59.036633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370b448-ee4d-4e7b-82a9-c1913854bbd0 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.040837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610566eb-eaf7-40d8-900b-96b3902bc1f8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.042264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610566eb-eaf7-40d8-900b-96b3902bc1f8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.042933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610566eb-eaf7-40d8-900b-96b3902bc1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.069120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-610566eb-eaf7-40d8-900b-96b3902bc1f8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:59.071136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-610566eb-eaf7-40d8-900b-96b3902bc1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.145653 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 991/2003] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:59.154304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35852616-8536-4d40-b1f4-0503f033767a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.154981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35852616-8536-4d40-b1f4-0503f033767a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.155487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35852616-8536-4d40-b1f4-0503f033767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.160158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.160944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.162022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.163019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.163786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.166321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.187945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-35852616-8536-4d40-b1f4-0503f033767a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:59.189693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-35852616-8536-4d40-b1f4-0503f033767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.199262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1013/2004] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:59 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:59.208558 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 992/2005] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:59.208652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.208737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35852616-8536-4d40-b1f4-0503f033767a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.211822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-35852616-8536-4d40-b1f4-0503f033767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.213335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.227998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f1ea-913d-46e4-bee0-085d2e471887 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.233193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.233974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.234463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.234918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.236204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.237719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.240900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35852616-8536-4d40-b1f4-0503f033767a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.247972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.248747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.249402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.249726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] PATH_INFO: `/v3/projects/6c6ebac696a940c0946c16dd7d8ca45d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.251075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6ebac696a940c0946c16dd7d8ca45d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.253680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.258069 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1014/2006] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:59.259978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.275418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532f9b3-7fbe-4f0a-ae64-a26465383de4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.281089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.282135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.282635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.283122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.283892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:59.284207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.285861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.299577 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1015/2007] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:59.300426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.313263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6ebac696a940c0946c16dd7d8ca45d', 'request_id': 'req-1c67d442-48d5-4691-a674-50b3751e9c53'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.314762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5def5-7e87-4105-aac5-ae7d64cf58f3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.321793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6ebac696a940c0946c16dd7d8ca45d', 'request_id': 'req-1c67d442-48d5-4691-a674-50b3751e9c53'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.331703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6ebac696a940c0946c16dd7d8ca45d', 'request_id': 'req-1c67d442-48d5-4691-a674-50b3751e9c53'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.341503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.346222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] The token cache is being invalidate because project 6c6ebac696a940c0946c16dd7d8ca45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:59.346904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.348426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.354266 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 993/2008] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:59 2022] DELETE /identity/v3/projects/6c6ebac696a940c0946c16dd7d8ca45d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:59.354266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.355724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.386157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.404193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e736c7e-0412-4db5-9e0d-1a189c2f2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.407015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e736c7e-0412-4db5-9e0d-1a189c2f2605 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.407339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e736c7e-0412-4db5-9e0d-1a189c2f2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.407742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e736c7e-0412-4db5-9e0d-1a189c2f2605 admin admin] PATH_INFO: `/v3/projects/53354e27dd764bfda07c7355286b951f/users/807a047da85a4724ac4518dd5c870af5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.454907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e736c7e-0412-4db5-9e0d-1a189c2f2605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53354e27dd764bfda07c7355286b951f, user_id=807a047da85a4724ac4518dd5c870af5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.458176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e736c7e-0412-4db5-9e0d-1a189c2f2605 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.479230 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1016/2009] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:59 2022] PUT /identity/v3/projects/53354e27dd764bfda07c7355286b951f/users/807a047da85a4724ac4518dd5c870af5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:59.481247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.481803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.482303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.512054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:59.513553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.567768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d442-48d5-4691-a674-50b3751e9c53 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.623539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1017/2010] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:59.624386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.629433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.664810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2f9e7-3dd4-4359-b2b1-18f7f1f16ac0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.678171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f00ac-5ab6-460e-8f27-2cf74ad7e6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.678933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f00ac-5ab6-460e-8f27-2cf74ad7e6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.679207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.679371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f00ac-5ab6-460e-8f27-2cf74ad7e6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.679457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f00ac-5ab6-460e-8f27-2cf74ad7e6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.680013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.680118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f00ac-5ab6-460e-8f27-2cf74ad7e6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.680299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.680804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.681524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.681956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f00ac-5ab6-460e-8f27-2cf74ad7e6e8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.684687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.724649 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1018/2011] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:50:59.726160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4f27-d07c-4d78-a754-4414e3ffe418 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.726442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4f27-d07c-4d78-a754-4414e3ffe418 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.726714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4f27-d07c-4d78-a754-4414e3ffe418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.739661 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 994/2012] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:50:59 2022] POST /identity/v3/projects => generated 381 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:59.740609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.745782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.763889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a53b4f27-d07c-4d78-a754-4414e3ffe418 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:59.765516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a53b4f27-d07c-4d78-a754-4414e3ffe418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.830448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57309751-df7b-4717-aaf1-4fe2067c780f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.836060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.836852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.837195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.837561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] PATH_INFO: `/v3/projects/33f1d412d4a44e09b51be3b14530c077` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.838834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33f1d412d4a44e09b51be3b14530c077)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.840993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.843825 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1019/2013] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:50:59.844743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4f27-d07c-4d78-a754-4414e3ffe418 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.856138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4f27-d07c-4d78-a754-4414e3ffe418 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.861718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.862389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.862669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.862942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] PATH_INFO: `/v3/projects/53354e27dd764bfda07c7355286b951f/users/807a047da85a4724ac4518dd5c870af5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.865102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=53354e27dd764bfda07c7355286b951f, user_id=807a047da85a4724ac4518dd5c870af5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.869008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.883341 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1020/2014] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:50:59 2022] PUT /identity/v3/projects/53354e27dd764bfda07c7355286b951f/users/807a047da85a4724ac4518dd5c870af5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:59.884269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.886802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.909684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33f1d412d4a44e09b51be3b14530c077', 'request_id': 'req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.918778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33f1d412d4a44e09b51be3b14530c077', 'request_id': 'req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.930940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33f1d412d4a44e09b51be3b14530c077', 'request_id': 'req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.947219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] The token cache is being invalidate because project 33f1d412d4a44e09b51be3b14530c077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:50:59.947721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.948771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:50:59.950903 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 995/2015] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:50:59 2022] DELETE /identity/v3/projects/33f1d412d4a44e09b51be3b14530c077 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:59.951674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.954676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:59.970028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.982211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6a44e-fae1-4c98-8af7-c040904f44ea admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.029512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab68c5-c379-4fe9-a092-ba9c1d9360d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.035126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117fa07-80cc-4ca9-9c7f-fee3a36f23f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.035750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117fa07-80cc-4ca9-9c7f-fee3a36f23f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.036029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117fa07-80cc-4ca9-9c7f-fee3a36f23f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.036301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117fa07-80cc-4ca9-9c7f-fee3a36f23f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.036962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117fa07-80cc-4ca9-9c7f-fee3a36f23f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:00.037260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117fa07-80cc-4ca9-9c7f-fee3a36f23f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.038915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117fa07-80cc-4ca9-9c7f-fee3a36f23f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.041600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.042220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.042500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.042813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.043449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.045537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.051568 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 996/2016] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:00.052815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155813-2104-47a4-9366-0b05aec7d53b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.053102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155813-2104-47a4-9366-0b05aec7d53b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.053379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155813-2104-47a4-9366-0b05aec7d53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.068189 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1021/2017] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:50:59 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:00.069042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.070656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.082905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60295d8a-1a6f-4ade-973d-c993d189d9ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.085189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-92155813-2104-47a4-9366-0b05aec7d53b None None] MFA Rules not processed for user `807a047da85a4724ac4518dd5c870af5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:00.086487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-92155813-2104-47a4-9366-0b05aec7d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.087621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.088256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.088540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.088814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] PATH_INFO: `/v3/users/d11a5517ab2e4991bc00b29ae4d6cd7f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.090061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11a5517ab2e4991bc00b29ae4d6cd7f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.092029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.154439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11a5517ab2e4991bc00b29ae4d6cd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:00.172321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11a5517ab2e4991bc00b29ae4d6cd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:00.181676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11a5517ab2e4991bc00b29ae4d6cd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:00.182453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11a5517ab2e4991bc00b29ae4d6cd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:00.183509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11a5517ab2e4991bc00b29ae4d6cd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:00.187933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 997/2018] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:00.189741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92155813-2104-47a4-9366-0b05aec7d53b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.203056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92155813-2104-47a4-9366-0b05aec7d53b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.207945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb1576-625c-4817-9475-57209e370304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.208767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb1576-625c-4817-9475-57209e370304 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.209072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb1576-625c-4817-9475-57209e370304 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.209345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb1576-625c-4817-9475-57209e370304 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.210033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb1576-625c-4817-9475-57209e370304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.212514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1022/2019] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:00 2022] DELETE /identity/v3/users/d11a5517ab2e4991bc00b29ae4d6cd7f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:00.212883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb1576-625c-4817-9475-57209e370304 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.213945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.216823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.246281 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 998/2020] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:00.248843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.249571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.250015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.269357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] MFA Rules not processed for user `ec6d47d564174aceb110b3354ff7db2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:00.270866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.285746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:00.293633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f404fb6-07fa-4125-a40a-b34d8654795e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.297885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:00.339587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 999/2021] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:00.344646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.344646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.372263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.372891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.373169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.374232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.375454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.378232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.391298 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1023/2022] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:00.392147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.394893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.426578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc48f1e-935b-45ed-a402-30f3d81ea838 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.432502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.433132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.433494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.433795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.434436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.436821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.481696 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1000/2023] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:00.482653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.495496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e313-43a1-4d31-b190-b14975ae4345 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.502068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.502691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.502967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.503366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.504022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.505718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.520067 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1001/2024] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:00.520067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.520345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3476d87-ead6-439a-a801-90b37078733c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.530772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.531775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.532152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d93db2-c5a4-457c-8924-ea2f10d3c1e8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.532262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.532744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.534881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.537051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.537704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.537988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.538267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.538918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.540531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.547515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.549147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.552238 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1002/2025] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:00.553245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.559935 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1024/2026] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:00.560871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.563643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115d969-cda1-482a-ad63-af17f951c232 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.568969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62f2b4-40cb-4e3b-8eb1-f393e7283920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.569739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62f2b4-40cb-4e3b-8eb1-f393e7283920 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.570021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62f2b4-40cb-4e3b-8eb1-f393e7283920 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.570295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62f2b4-40cb-4e3b-8eb1-f393e7283920 admin admin] PATH_INFO: `/v3/projects/2e6de4c63a374e488215e96b63c460ad/users/9f015f01d15940aab68d335021f71d32/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.578383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0a7cb-9d7e-4ca3-a38c-932383eb11e5 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.584269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.586142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.586637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.587178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.587359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62f2b4-40cb-4e3b-8eb1-f393e7283920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6de4c63a374e488215e96b63c460ad, user_id=9f015f01d15940aab68d335021f71d32, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.588217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.590098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.590211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62f2b4-40cb-4e3b-8eb1-f393e7283920 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.609116 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1025/2027] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:00.609295 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1003/2028] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:00 2022] PUT /identity/v3/projects/2e6de4c63a374e488215e96b63c460ad/users/9f015f01d15940aab68d335021f71d32/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:00.619898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.621410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31b4e7-0d23-4dcb-81bc-ca868e211b15 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.622127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31b4e7-0d23-4dcb-81bc-ca868e211b15 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.622551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31b4e7-0d23-4dcb-81bc-ca868e211b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.632602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98779-b7bb-439e-8a65-60f209da6567 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.657965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9d31b4e7-0d23-4dcb-81bc-ca868e211b15 None None] MFA Rules not processed for user `9f015f01d15940aab68d335021f71d32`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:00.659190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9d31b4e7-0d23-4dcb-81bc-ca868e211b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.715325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1391524-ef8f-49bc-ba53-d2505a04b7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.716402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391524-ef8f-49bc-ba53-d2505a04b7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.716640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391524-ef8f-49bc-ba53-d2505a04b7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.717001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391524-ef8f-49bc-ba53-d2505a04b7f6 admin admin] PATH_INFO: `/v3/projects/755d615547744fb2870b838ddc13af1d/users/50ca37f3ec00403c82d39469681f2c3e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.730393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:00.734893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1391524-ef8f-49bc-ba53-d2505a04b7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755d615547744fb2870b838ddc13af1d, user_id=50ca37f3ec00403c82d39469681f2c3e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.737231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1391524-ef8f-49bc-ba53-d2505a04b7f6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.747894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:00.749140 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1004/2029] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:00.753673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31b4e7-0d23-4dcb-81bc-ca868e211b15 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.753975 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1026/2030] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:00 2022] PUT /identity/v3/projects/755d615547744fb2870b838ddc13af1d/users/50ca37f3ec00403c82d39469681f2c3e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:00.765937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.766300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.766640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.768540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31b4e7-0d23-4dcb-81bc-ca868e211b15 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.801726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] MFA Rules not processed for user `50ca37f3ec00403c82d39469681f2c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:00.802947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.877423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.878070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.878279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.878551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:00.879193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:00.881582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:00.894015 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1027/2031] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:00.898982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.901069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:00.919191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789605b7-eefa-4278-9478-78a96ec9b823 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:00.921985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1005/2032] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:00.922976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:00.997115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:00.998052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:00.998397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:00.998925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] PATH_INFO: `/v3/projects/06fddef025ac4c76aba4443a4f467108` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.000155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06fddef025ac4c76aba4443a4f467108)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.002256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.012924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f5f53-0890-451e-b3c7-e2c685b1b708 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.024602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.025305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.026341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.026687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.027448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.030434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.079320 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1006/2033] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:00 2022] POST /identity/v3/projects => generated 391 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:01.082086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.086838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06fddef025ac4c76aba4443a4f467108', 'request_id': 'req-f5c39748-af2d-4070-9c0e-d12a70fef9d1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:01.108130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06fddef025ac4c76aba4443a4f467108', 'request_id': 'req-f5c39748-af2d-4070-9c0e-d12a70fef9d1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:01.118076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06fddef025ac4c76aba4443a4f467108', 'request_id': 'req-f5c39748-af2d-4070-9c0e-d12a70fef9d1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:01.142760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] The token cache is being invalidate because project 06fddef025ac4c76aba4443a4f467108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:01.143137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:01.143871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:01.146504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1028/2034] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:00 2022] DELETE /identity/v3/projects/06fddef025ac4c76aba4443a4f467108 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:01.147612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.149368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:01.197797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:01.211138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e444e-2f02-4ec6-82b2-ce55a09b396a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.217758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.218481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.218852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.219242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.219952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.224581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.250558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:01.261235 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1007/2035] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:01 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:01.262323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.265440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c39748-af2d-4070-9c0e-d12a70fef9d1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.267185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:01.285317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f127b9-c671-4a17-98f9-2aa904e13805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.286139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f127b9-c671-4a17-98f9-2aa904e13805 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.286515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f127b9-c671-4a17-98f9-2aa904e13805 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.286888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f127b9-c671-4a17-98f9-2aa904e13805 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.287813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f127b9-c671-4a17-98f9-2aa904e13805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.291822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f127b9-c671-4a17-98f9-2aa904e13805 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.325636 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1029/2036] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:01 2022] POST /identity/v3/users => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:01.325636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.325636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.325636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.352721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:01.354256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:01.364887 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1030/2037] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:01.365907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.382980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:01.402271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20336b53-558d-4fd2-a19a-f4df46362b90 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.411520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.414054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.414547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.415021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.415881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.418646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.434764 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1031/2038] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:01.436377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.449777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee96e-212a-4e0f-940c-a49d855966a2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.458883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.459538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea69ea-8022-45d9-a27e-77323845cd73 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.459654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.459835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.460108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.462312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:01.474587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b16836-68db-4d21-8de5-bcba685ff477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.477678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16836-68db-4d21-8de5-bcba685ff477 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.478249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16836-68db-4d21-8de5-bcba685ff477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.479087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16836-68db-4d21-8de5-bcba685ff477 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.480222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b16836-68db-4d21-8de5-bcba685ff477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.485207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b16836-68db-4d21-8de5-bcba685ff477 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.510731 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1032/2039] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:01.519043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284726-a89b-40ab-b893-f488c2dbbe54 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.519375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284726-a89b-40ab-b893-f488c2dbbe54 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.519591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284726-a89b-40ab-b893-f488c2dbbe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.581307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f5284726-a89b-40ab-b893-f488c2dbbe54 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:01.582838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f5284726-a89b-40ab-b893-f488c2dbbe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:01.582990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.584790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.596190 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1033/2040] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:01.597256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284726-a89b-40ab-b893-f488c2dbbe54 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.615203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284726-a89b-40ab-b893-f488c2dbbe54 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.622948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.623132 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1008/2041] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:01.623312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.623661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.623834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] PATH_INFO: `/v3/projects/ac33bbb403db442483575890ac577618/users/bf2f08938a3e401183f5a4d2cc7c70bf/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.624044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.643427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac33bbb403db442483575890ac577618, user_id=bf2f08938a3e401183f5a4d2cc7c70bf, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.649849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.673879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163981e6-066a-4bed-8b34-a59e1227c6a9 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.679900 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1034/2042] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:01 2022] PUT /identity/v3/projects/ac33bbb403db442483575890ac577618/users/bf2f08938a3e401183f5a4d2cc7c70bf/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:01.681293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.765399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.766948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.767350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.767741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] PATH_INFO: `/v3/projects/e093c21fb3bf41ab8d7b10fe59dae5d3/users/e1e607a86ce240a7ab313ae1c71aaf6b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.789327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e093c21fb3bf41ab8d7b10fe59dae5d3, user_id=e1e607a86ce240a7ab313ae1c71aaf6b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.792559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.820603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98257f9-2326-4734-a0c2-ffdf672f696e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.823923 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1009/2043] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:01 2022] PUT /identity/v3/projects/e093c21fb3bf41ab8d7b10fe59dae5d3/users/e1e607a86ce240a7ab313ae1c71aaf6b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:01.823923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.854425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:01.925491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46a0d6-1e21-4ea1-a045-6254942a1c40 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:01.931552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:01.937630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.939125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.940896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.941062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.941790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.942377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:01.943292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:01.943693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:01.943803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.944116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:01.949380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:01.952050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:01.955923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:01.961132 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1010/2044] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:01.965927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:01.999849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e10a7-9e19-47c8-b544-504b1eb0f62b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.001086 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1035/2045] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:02.006122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.012624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.016531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.016862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.018493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.020027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.023219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.025577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008e0b6-78dd-4464-9e7a-1f23a45df7ef admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.033061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.033898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.034275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.034650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] PATH_INFO: `/v3/projects/ac33bbb403db442483575890ac577618/users/bf2f08938a3e401183f5a4d2cc7c70bf/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.036994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac33bbb403db442483575890ac577618, user_id=bf2f08938a3e401183f5a4d2cc7c70bf, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.040101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.049226 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1011/2046] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:02.052377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.061094 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1036/2047] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:02 2022] PUT /identity/v3/projects/ac33bbb403db442483575890ac577618/users/bf2f08938a3e401183f5a4d2cc7c70bf/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:02.062140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.081370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bc5ce-7472-4b6d-a9a0-80790c911ecd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.152522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48ac87-82a3-4e29-9a8f-b19aedbfcc8d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.158869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f168a3-e1e3-4df1-a1e5-b361a821b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.160268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f168a3-e1e3-4df1-a1e5-b361a821b587 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.160654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f168a3-e1e3-4df1-a1e5-b361a821b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.161029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f168a3-e1e3-4df1-a1e5-b361a821b587 admin admin] PATH_INFO: `/v3/projects/e093c21fb3bf41ab8d7b10fe59dae5d3/users/e1e607a86ce240a7ab313ae1c71aaf6b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.163271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f168a3-e1e3-4df1-a1e5-b361a821b587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e093c21fb3bf41ab8d7b10fe59dae5d3, user_id=e1e607a86ce240a7ab313ae1c71aaf6b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.166201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f168a3-e1e3-4df1-a1e5-b361a821b587 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.202410 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1037/2048] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:02 2022] PUT /identity/v3/projects/e093c21fb3bf41ab8d7b10fe59dae5d3/users/e1e607a86ce240a7ab313ae1c71aaf6b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:02.203983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d581f80-159b-4807-8aea-3754bdebd27f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.204280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d581f80-159b-4807-8aea-3754bdebd27f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.204702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d581f80-159b-4807-8aea-3754bdebd27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.242363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abcdf1a-2d93-4643-8b10-b2ce0bef6674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.242999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcdf1a-2d93-4643-8b10-b2ce0bef6674 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.243289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcdf1a-2d93-4643-8b10-b2ce0bef6674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.243563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abcdf1a-2d93-4643-8b10-b2ce0bef6674 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.244231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abcdf1a-2d93-4643-8b10-b2ce0bef6674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.246247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abcdf1a-2d93-4643-8b10-b2ce0bef6674 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.269603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6d581f80-159b-4807-8aea-3754bdebd27f None None] MFA Rules not processed for user `bf2f08938a3e401183f5a4d2cc7c70bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:02.270816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6d581f80-159b-4807-8aea-3754bdebd27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.309418 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1012/2049] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/projects => generated 387 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:02.314132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae09a26-e0b1-4ff3-ad4a-7654cddc6b2a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.314420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae09a26-e0b1-4ff3-ad4a-7654cddc6b2a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.314699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae09a26-e0b1-4ff3-ad4a-7654cddc6b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.371790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4ae09a26-e0b1-4ff3-ad4a-7654cddc6b2a None None] MFA Rules not processed for user `e1e607a86ce240a7ab313ae1c71aaf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:02.373023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ae09a26-e0b1-4ff3-ad4a-7654cddc6b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.388058 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1038/2050] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:02.388939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d581f80-159b-4807-8aea-3754bdebd27f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.460891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:02.469172 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1013/2051] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:02.470091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae09a26-e0b1-4ff3-ad4a-7654cddc6b2a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.484052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:02.485299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d581f80-159b-4807-8aea-3754bdebd27f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.539330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae09a26-e0b1-4ff3-ad4a-7654cddc6b2a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.544297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df9ab2-0045-4f27-864b-807c0dd64947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.544920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9ab2-0045-4f27-864b-807c0dd64947 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.545227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9ab2-0045-4f27-864b-807c0dd64947 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.545546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9ab2-0045-4f27-864b-807c0dd64947 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.546238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df9ab2-0045-4f27-864b-807c0dd64947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.548237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df9ab2-0045-4f27-864b-807c0dd64947 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.549185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.549831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.550125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.550372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.552083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.569998 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1014/2052] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:02.571919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac17dd-2267-4041-b1f0-14a22cd4c50c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.572442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac17dd-2267-4041-b1f0-14a22cd4c50c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.572856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac17dd-2267-4041-b1f0-14a22cd4c50c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.574679 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1015/2053] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:02.575791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac17dd-2267-4041-b1f0-14a22cd4c50c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.589194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac17dd-2267-4041-b1f0-14a22cd4c50c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.593902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0fc9e-3d81-4c9c-accc-4a3c49d6edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.594495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0fc9e-3d81-4c9c-accc-4a3c49d6edcf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.594767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0fc9e-3d81-4c9c-accc-4a3c49d6edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.595032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0fc9e-3d81-4c9c-accc-4a3c49d6edcf admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.595661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0fc9e-3d81-4c9c-accc-4a3c49d6edcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.597280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0fc9e-3d81-4c9c-accc-4a3c49d6edcf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.607346 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1016/2054] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:02.608326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.608605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.608879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.622878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.624506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.627707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:02.628889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.634052 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1039/2055] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:02.634849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.645001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ce65-c93b-4ce8-8ee0-e6f5044fa2bc service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.650355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.650954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.651354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.651625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.652300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.654332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.668127 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1040/2056] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:02.669682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.687853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f666-13c2-4686-92b0-b76924acb685 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.701750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9319cf-ea2f-440e-85d6-a713a70c7a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.702430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9319cf-ea2f-440e-85d6-a713a70c7a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.702710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9319cf-ea2f-440e-85d6-a713a70c7a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.702991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9319cf-ea2f-440e-85d6-a713a70c7a37 admin admin] PATH_INFO: `/v3/projects/e0f0332723fc47db82faca60313db4ca/users/85906ce95c064aa7ab55af182215f77a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.715645 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1017/2057] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:02.723796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9319cf-ea2f-440e-85d6-a713a70c7a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f0332723fc47db82faca60313db4ca, user_id=85906ce95c064aa7ab55af182215f77a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.727073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9319cf-ea2f-440e-85d6-a713a70c7a37 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.729173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.730886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.742638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfca48-084d-45db-a64e-5b280f5400db None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.748395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.748395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.748395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.748395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.759398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:02.761627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:02.768914 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1041/2058] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:02 2022] PUT /identity/v3/projects/e0f0332723fc47db82faca60313db4ca/users/85906ce95c064aa7ab55af182215f77a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:02.776825 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1018/2059] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:02.799508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa842b-85d8-40ae-bf4c-9da6f3da7770 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.799914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa842b-85d8-40ae-bf4c-9da6f3da7770 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.801160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa842b-85d8-40ae-bf4c-9da6f3da7770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.847949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-20aa842b-85d8-40ae-bf4c-9da6f3da7770 None None] MFA Rules not processed for user `85906ce95c064aa7ab55af182215f77a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:02.849521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-20aa842b-85d8-40ae-bf4c-9da6f3da7770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.904381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:02.935569 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1042/2060] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:02.940279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.953236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7b2a-c99d-42b8-9afc-a00343725df7 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.027816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa842b-85d8-40ae-bf4c-9da6f3da7770 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.041479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.042119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.042402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.042786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.043479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.045517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.107500 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1019/2061] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/projects => generated 391 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:03.107500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.110903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:03.122202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa842b-85d8-40ae-bf4c-9da6f3da7770 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.135857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.214329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.214930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.215210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.215485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.238999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7b3bc-05cd-405a-87ef-207f02b639a6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.248401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b8011-bcd7-431c-addd-3bd3440189b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.249311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b8011-bcd7-431c-addd-3bd3440189b4 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.249651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b8011-bcd7-431c-addd-3bd3440189b4 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.250916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b8011-bcd7-431c-addd-3bd3440189b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.284814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.286613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.299726 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1043/2062] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:03.300572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.314656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.340596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b8011-bcd7-431c-addd-3bd3440189b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.342335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b8011-bcd7-431c-addd-3bd3440189b4 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.354903 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1020/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:03.356697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f49590-2108-462b-8792-9c24aac9ef1d None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.357017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f49590-2108-462b-8792-9c24aac9ef1d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.357266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f49590-2108-462b-8792-9c24aac9ef1d None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.359375 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1021/2064] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:03.366003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e5a97-2d7a-4b7f-be85-6226ba216e6b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.366340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e5a97-2d7a-4b7f-be85-6226ba216e6b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.366614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e5a97-2d7a-4b7f-be85-6226ba216e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.387927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da5a0d-0205-4b19-be4e-3351c82b87f0 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.395317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.395964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5b8e5a97-2d7a-4b7f-be85-6226ba216e6b None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:03.396661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.397238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.397364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5b8e5a97-2d7a-4b7f-be85-6226ba216e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:03.398471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.399228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.401693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.452468 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1044/2065] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:03 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:03.453673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.473677 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1022/2066] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:03.474017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce182d-784a-47e8-b9fa-31166cc5a041 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.479728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.480443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e5a97-2d7a-4b7f-be85-6226ba216e6b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.481123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.483524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.484880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.485831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.487686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.495412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e5a97-2d7a-4b7f-be85-6226ba216e6b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.504148 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1045/2067] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:03.505708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.508719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.509012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.509281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.511742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.524831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.526345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.536307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123ee6-2de5-479a-aa28-0f20b698117b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.547763 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1023/2068] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:03.549774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.550464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.550908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.551195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] PATH_INFO: `/v3/projects/e448135a65c349d19b25015d016f6e3f/users/52c58e664bdd4e459c4d1e82e43f127b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.581347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e448135a65c349d19b25015d016f6e3f, user_id=52c58e664bdd4e459c4d1e82e43f127b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.584432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.606822 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1046/2069] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:03 2022] PUT /identity/v3/projects/e448135a65c349d19b25015d016f6e3f/users/52c58e664bdd4e459c4d1e82e43f127b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:03.611003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.625254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd8509-ff81-467b-81e7-a0cb77ccdc92 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.677317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.707792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b57a5-f870-485f-b4c0-0a7d819a748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.708477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b57a5-f870-485f-b4c0-0a7d819a748f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.708770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b57a5-f870-485f-b4c0-0a7d819a748f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.709044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b57a5-f870-485f-b4c0-0a7d819a748f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.709941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b57a5-f870-485f-b4c0-0a7d819a748f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.711575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b57a5-f870-485f-b4c0-0a7d819a748f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.725597 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1024/2070] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:03.728354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.745359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1216fb-9c31-4f59-8328-bfc2857e9e8e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.751277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.752104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.752530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.753030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] PATH_INFO: `/v3/projects/e448135a65c349d19b25015d016f6e3f/users/52c58e664bdd4e459c4d1e82e43f127b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.755396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e448135a65c349d19b25015d016f6e3f, user_id=52c58e664bdd4e459c4d1e82e43f127b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.759532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.783100 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1047/2071] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:03 2022] PUT /identity/v3/projects/e448135a65c349d19b25015d016f6e3f/users/52c58e664bdd4e459c4d1e82e43f127b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:03.785572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7a8f2-c57f-4d44-9721-edc447554fd5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.785951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7a8f2-c57f-4d44-9721-edc447554fd5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.786254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7a8f2-c57f-4d44-9721-edc447554fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.794578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.797118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:03.825109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-53b7a8f2-c57f-4d44-9721-edc447554fd5 None None] MFA Rules not processed for user `52c58e664bdd4e459c4d1e82e43f127b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:03.826835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-53b7a8f2-c57f-4d44-9721-edc447554fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:03.951325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a960-c076-42d4-9274-a352c76481a6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.952088 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1025/2072] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:03.953067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7a8f2-c57f-4d44-9721-edc447554fd5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.972247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.973655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.973655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.973905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.981343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.010778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.013096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.029389 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1048/2073] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:04.030717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.095758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7a8f2-c57f-4d44-9721-edc447554fd5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.109026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.109723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.110002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.110274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] PATH_INFO: `/v3/users/9f015f01d15940aab68d335021f71d32` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.114165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f015f01d15940aab68d335021f71d32)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.114165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.143864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.196964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f07e1-c7fe-48e4-9336-acfe0cdd7acf service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.208227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864c987-60bf-4716-a90f-e8a2c2877795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.209430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864c987-60bf-4716-a90f-e8a2c2877795 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.209930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864c987-60bf-4716-a90f-e8a2c2877795 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.210254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864c987-60bf-4716-a90f-e8a2c2877795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.243442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f015f01d15940aab68d335021f71d32'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.244310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f015f01d15940aab68d335021f71d32'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.255614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f015f01d15940aab68d335021f71d32'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.256460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f015f01d15940aab68d335021f71d32'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.265950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.284454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f015f01d15940aab68d335021f71d32'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.311400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1026/2074] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:03 2022] DELETE /identity/v3/users/9f015f01d15940aab68d335021f71d32 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:04.313147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.325956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.361396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864c987-60bf-4716-a90f-e8a2c2877795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.363328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864c987-60bf-4716-a90f-e8a2c2877795 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.375196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1049/2075] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:04.376655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1904b7-fd15-4479-b56a-d2239d9d4f36 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.377034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1904b7-fd15-4479-b56a-d2239d9d4f36 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.377420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1904b7-fd15-4479-b56a-d2239d9d4f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.400119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7f1904b7-fd15-4479-b56a-d2239d9d4f36 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:04.402489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7f1904b7-fd15-4479-b56a-d2239d9d4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.413329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1050/2076] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:04.414852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c29d-320c-4074-ba63-b7adc03c95d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.415102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ea40-c90a-4840-a428-d324e8069cfb None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.415569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ea40-c90a-4840-a428-d324e8069cfb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.415985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ea40-c90a-4840-a428-d324e8069cfb None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.417956 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1051/2077] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:04.419308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e58d9-da80-4db3-9706-1520caa332c4 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.419805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e58d9-da80-4db3-9706-1520caa332c4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.420148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e58d9-da80-4db3-9706-1520caa332c4 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.421514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1052/2078] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:04.424195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14a87-074c-43d6-8ce5-0a6f92cc9d04 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.424478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14a87-074c-43d6-8ce5-0a6f92cc9d04 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.425054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e14a87-074c-43d6-8ce5-0a6f92cc9d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.426827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.427868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.428310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.428827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] PATH_INFO: `/v3/users/50ca37f3ec00403c82d39469681f2c3e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.432216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ca37f3ec00403c82d39469681f2c3e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.435020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.452103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-84e14a87-074c-43d6-8ce5-0a6f92cc9d04 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:04.453841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-84e14a87-074c-43d6-8ce5-0a6f92cc9d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.511178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ca37f3ec00403c82d39469681f2c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.512252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ca37f3ec00403c82d39469681f2c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.526183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ca37f3ec00403c82d39469681f2c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.527969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ca37f3ec00403c82d39469681f2c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.549196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ca37f3ec00403c82d39469681f2c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:04.551923 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1053/2079] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:04.554189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51eb3c-9c22-44c0-b394-a76839c7d028 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.554491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51eb3c-9c22-44c0-b394-a76839c7d028 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.554753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51eb3c-9c22-44c0-b394-a76839c7d028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.583591 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1027/2080] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:04 2022] DELETE /identity/v3/users/50ca37f3ec00403c82d39469681f2c3e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:04.587610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-dd51eb3c-9c22-44c0-b394-a76839c7d028 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:04.588415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.589100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dd51eb3c-9c22-44c0-b394-a76839c7d028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.591217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.681768 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1054/2081] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:04.684325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.684718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.684853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d4419-2cf5-4081-9065-4654b169b31f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.685298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.698316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.699596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.700156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.700605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.702817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.705009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.721282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:04.722806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.734062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.836024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1055/2082] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:04.836024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.837342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.872465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.874236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.889673 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1028/2083] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:04.890924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.896765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.912067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649872-1a44-4f0e-9881-5c6680f62d12 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.920232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.921355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.922506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.922978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.942938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.944931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33131540-746d-4da7-ad10-155ea7b8cc49 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.945172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.960389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.961190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.961489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.961763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.963024 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1029/2084] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:04.964095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.972326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.974045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d825dde-2751-4c4d-80b9-766e4531b97f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.974325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.979818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.981083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.981587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.982791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.987678 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1056/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:04.995667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.997742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.019009 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1030/2086] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:05.709300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.709300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:05.735213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce8297-32b8-44f8-87cf-131622954a05 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.742808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.743875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.744341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.744817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.747606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:05.868824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.871573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.887734 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1057/2087] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:06.219443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.222154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:06.242201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c07103-edde-41f2-9ca5-a2fc45e34478 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.249202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.262888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.264971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.265223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] PATH_INFO: `/v3/projects/2e6de4c63a374e488215e96b63c460ad` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.268055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6de4c63a374e488215e96b63c460ad)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.276570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.376751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.385570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:06.404460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6de4c63a374e488215e96b63c460ad', 'request_id': 'req-2614b03d-084a-46f7-aa3f-3666c8328e0d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.437501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6de4c63a374e488215e96b63c460ad', 'request_id': 'req-2614b03d-084a-46f7-aa3f-3666c8328e0d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.439068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c907c6f-d5f5-4f79-ba88-e82ba50f70d8 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.450090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6de4c63a374e488215e96b63c460ad', 'request_id': 'req-2614b03d-084a-46f7-aa3f-3666c8328e0d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.471092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] The token cache is being invalidate because project 2e6de4c63a374e488215e96b63c460ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:06.471397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.472363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.474396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1031/2088] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:06 2022] DELETE /identity/v3/projects/2e6de4c63a374e488215e96b63c460ad => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:06.479521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.482670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:06.504564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:06.550388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.553187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.553414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.553972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] PATH_INFO: `/v3/projects/755d615547744fb2870b838ddc13af1d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.555760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=755d615547744fb2870b838ddc13af1d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.558775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.572196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614b03d-084a-46f7-aa3f-3666c8328e0d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.578853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.580193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.580894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.582444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] PATH_INFO: `/v3/users/bf2f08938a3e401183f5a4d2cc7c70bf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.583831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2f08938a3e401183f5a4d2cc7c70bf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.586083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.592672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:06.632054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755d615547744fb2870b838ddc13af1d', 'request_id': 'req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.641248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755d615547744fb2870b838ddc13af1d', 'request_id': 'req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.649728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755d615547744fb2870b838ddc13af1d', 'request_id': 'req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.664609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2f08938a3e401183f5a4d2cc7c70bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.665438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2f08938a3e401183f5a4d2cc7c70bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.673115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] The token cache is being invalidate because project 755d615547744fb2870b838ddc13af1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:06.673756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.674944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.676838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2f08938a3e401183f5a4d2cc7c70bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.676990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1058/2089] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:06 2022] DELETE /identity/v3/projects/755d615547744fb2870b838ddc13af1d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:06.678215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2f08938a3e401183f5a4d2cc7c70bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.681366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.683779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:06.706117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-14c662e9-cd77-467a-9407-9f5cd2fec3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2f08938a3e401183f5a4d2cc7c70bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:06.757950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1032/2090] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:06 2022] DELETE /identity/v3/users/bf2f08938a3e401183f5a4d2cc7c70bf => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:06.807733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d3cb4-3b7f-420e-a6ab-1adf55bc09eb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.902704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.903758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.903996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.904413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] PATH_INFO: `/v3/users/e1e607a86ce240a7ab313ae1c71aaf6b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.905734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e607a86ce240a7ab313ae1c71aaf6b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.907877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.915515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:06.982067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e607a86ce240a7ab313ae1c71aaf6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.012572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e607a86ce240a7ab313ae1c71aaf6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.033681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e607a86ce240a7ab313ae1c71aaf6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.035531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e607a86ce240a7ab313ae1c71aaf6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.037520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e607a86ce240a7ab313ae1c71aaf6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.061238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.061971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.062620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:07.093290 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1059/2091] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:06 2022] DELETE /identity/v3/users/e1e607a86ce240a7ab313ae1c71aaf6b => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:07.102380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] MFA Rules not processed for user `eb8bfe1d590a4500b7e7b5c79a6a4e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:07.103893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:07.192664 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1033/2092] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:07.700239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:07.701886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:07.753406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:07.755008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:07.818336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a2b33-d89f-404a-99d8-3256d9592e7b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:07.823210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:07.824126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.824479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.824822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] PATH_INFO: `/v3/projects/ac33bbb403db442483575890ac577618` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:07.826079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac33bbb403db442483575890ac577618)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:07.828217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:07.853540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3e849-28ef-4e5c-aae3-a223abf21bb1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:07.860655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:07.861341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.861685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.861977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] PATH_INFO: `/v3/projects/e093c21fb3bf41ab8d7b10fe59dae5d3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:07.863312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e093c21fb3bf41ab8d7b10fe59dae5d3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:07.865876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:07.887760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac33bbb403db442483575890ac577618', 'request_id': 'req-29c7db01-3f7c-4fef-b278-50eab93bfb64'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.901155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac33bbb403db442483575890ac577618', 'request_id': 'req-29c7db01-3f7c-4fef-b278-50eab93bfb64'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.911364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac33bbb403db442483575890ac577618', 'request_id': 'req-29c7db01-3f7c-4fef-b278-50eab93bfb64'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.920261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:07.933739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] The token cache is being invalidate because project ac33bbb403db442483575890ac577618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:07.934042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.934858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29c7db01-3f7c-4fef-b278-50eab93bfb64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.937700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e093c21fb3bf41ab8d7b10fe59dae5d3', 'request_id': 'req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.940761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1060/2093] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:07 2022] DELETE /identity/v3/projects/ac33bbb403db442483575890ac577618 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:07.943471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5210a-c8bd-48d0-9d16-e5d3b63e4deb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.944028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5210a-c8bd-48d0-9d16-e5d3b63e4deb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.944245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5210a-c8bd-48d0-9d16-e5d3b63e4deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:07.946192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e093c21fb3bf41ab8d7b10fe59dae5d3', 'request_id': 'req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.957053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e093c21fb3bf41ab8d7b10fe59dae5d3', 'request_id': 'req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.969499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bfb5210a-c8bd-48d0-9d16-e5d3b63e4deb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:07.970790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bfb5210a-c8bd-48d0-9d16-e5d3b63e4deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:07.975727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] The token cache is being invalidate because project e093c21fb3bf41ab8d7b10fe59dae5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:07.976132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.976895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0cd2b81c-af61-4c37-8fb8-71b49077bf1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:07.983057 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1034/2094] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:07 2022] DELETE /identity/v3/projects/e093c21fb3bf41ab8d7b10fe59dae5d3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:07.983216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:07.990528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc286a2-533c-487f-806f-21ca27286b13 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.990900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc286a2-533c-487f-806f-21ca27286b13 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.991270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc286a2-533c-487f-806f-21ca27286b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.012991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9fc286a2-533c-487f-806f-21ca27286b13 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:08.014640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9fc286a2-533c-487f-806f-21ca27286b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:08.057580 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1061/2095] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:08.059755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7c3f3-f564-45f8-8b7d-69afdbabed3b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.060216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7c3f3-f564-45f8-8b7d-69afdbabed3b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.060588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7c3f3-f564-45f8-8b7d-69afdbabed3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.093880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fcd7c3f3-f564-45f8-8b7d-69afdbabed3b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:08.095426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fcd7c3f3-f564-45f8-8b7d-69afdbabed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:08.116233 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1062/2096] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:08.117532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7c3f3-f564-45f8-8b7d-69afdbabed3b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.131386 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1035/2097] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:08.134768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7c3f3-f564-45f8-8b7d-69afdbabed3b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.139164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aef53-0454-4a17-8be9-321b61eb67ce None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.139470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aef53-0454-4a17-8be9-321b61eb67ce None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.140135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405aef53-0454-4a17-8be9-321b61eb67ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.141821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.142449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.142727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.143000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.143736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:08.143956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.145613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.160968 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1063/2098] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:08.164383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-405aef53-0454-4a17-8be9-321b61eb67ce None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:08.165132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.166863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-405aef53-0454-4a17-8be9-321b61eb67ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:08.175693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc51cb-50d2-4989-ba02-9aa85d6b61a6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.180443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.181055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.181408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.181632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.182484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.184627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.192742 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1036/2099] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:08.198497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aef53-0454-4a17-8be9-321b61eb67ce None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.211277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405aef53-0454-4a17-8be9-321b61eb67ce None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.223883 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1064/2100] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/projects => generated 407 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:08.227929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.231110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.231744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.232016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.232363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.233716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:08.234018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.240238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.264221 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1037/2101] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:08.265737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.289002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c024b1-f1ad-4596-ae8f-d94688fa74a3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.320065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18df5a8-4eba-4e28-9011-dfdb5ee0e3e5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.326890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.327722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.328007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.328641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.330353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.333742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.367776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.368843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.369271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.369749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.370554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.371568 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1065/2102] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/users => generated 304 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:08.372842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.374538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.385660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad414-73cb-415d-a3de-58395e7ecdd6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.392158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.394187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.394474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.394752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.396001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.398224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.422835 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1066/2103] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:08.424175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.438863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dab5d-69bd-405e-b184-678a63910969 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.445791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.446501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.446989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.447408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.448235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.450418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.456251 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1038/2104] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/projects => generated 391 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:08.458017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.472082 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1067/2105] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:08.473359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.494716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ec3ae-c4fb-4e15-9c8f-e8b70560cc7e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.543098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:08.570174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:08.597285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e2315-7f63-4da8-915a-498cc28aa18a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.598588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2931fe-d63b-400d-8e22-856c61e23261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.599763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2931fe-d63b-400d-8e22-856c61e23261 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.601143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2931fe-d63b-400d-8e22-856c61e23261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.601926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2931fe-d63b-400d-8e22-856c61e23261 admin admin] PATH_INFO: `/v3/projects/cb661478603944d4b3fef3ef84a2eaba/users/c46036360ef3496c9104d08a7251ac6b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.603242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.603951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.604295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.604630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.605685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.608680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.622965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2931fe-d63b-400d-8e22-856c61e23261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb661478603944d4b3fef3ef84a2eaba, user_id=c46036360ef3496c9104d08a7251ac6b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.626664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2931fe-d63b-400d-8e22-856c61e23261 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.646701 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1039/2106] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/users => generated 296 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:08.652877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.661425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1068/2107] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:08 2022] PUT /identity/v3/projects/cb661478603944d4b3fef3ef84a2eaba/users/c46036360ef3496c9104d08a7251ac6b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:08.676758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591bbc1-87d4-453c-ad13-cf40c9811201 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.676758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591bbc1-87d4-453c-ad13-cf40c9811201 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.676758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591bbc1-87d4-453c-ad13-cf40c9811201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.687900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccf834-55e9-46b3-9c69-7fc74e33c54e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.738444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b591bbc1-87d4-453c-ad13-cf40c9811201 None None] MFA Rules not processed for user `c46036360ef3496c9104d08a7251ac6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:08.739717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b591bbc1-87d4-453c-ad13-cf40c9811201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:08.762498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.763196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.763481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.763754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.764468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.766256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.781635 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1040/2108] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:08.785994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.804592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad40dbf-0035-4fc2-aebf-4236cac099dd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.811800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.814764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.815122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.816025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.817128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.819010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.831444 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1041/2109] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:08.836938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.858329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432fbc0-522a-4c7d-9023-ca549c7df718 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.866067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e91f4-1983-4cbf-9512-627a90709b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:08.866816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e91f4-1983-4cbf-9512-627a90709b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.867160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e91f4-1983-4cbf-9512-627a90709b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.867487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e91f4-1983-4cbf-9512-627a90709b2e admin admin] PATH_INFO: `/v3/projects/e04955b84b484b11bc91ab94ba749fe1/users/a2d0c7a21257489fa89d0d0b60e83a2f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.890420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e91f4-1983-4cbf-9512-627a90709b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04955b84b484b11bc91ab94ba749fe1, user_id=a2d0c7a21257489fa89d0d0b60e83a2f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:08.893819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e91f4-1983-4cbf-9512-627a90709b2e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:08.908714 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1069/2110] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:08.914035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591bbc1-87d4-453c-ad13-cf40c9811201 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:08.922186 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1042/2111] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:08 2022] PUT /identity/v3/projects/e04955b84b484b11bc91ab94ba749fe1/users/a2d0c7a21257489fa89d0d0b60e83a2f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:08.940188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbd0b-2ed1-4c69-82a8-0d28b23abacd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.940770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbd0b-2ed1-4c69-82a8-0d28b23abacd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.941199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591bbc1-87d4-453c-ad13-cf40c9811201 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:08.941321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbd0b-2ed1-4c69-82a8-0d28b23abacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.000577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7f1dbd0b-2ed1-4c69-82a8-0d28b23abacd None None] MFA Rules not processed for user `a2d0c7a21257489fa89d0d0b60e83a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:09.002636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f1dbd0b-2ed1-4c69-82a8-0d28b23abacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:09.063422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.064060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.064613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.074173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.075178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.077153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.117883 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1043/2112] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:09.122383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dbd0b-2ed1-4c69-82a8-0d28b23abacd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.127221 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1070/2113] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/projects => generated 409 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:09.128385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.130238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.138545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dbd0b-2ed1-4c69-82a8-0d28b23abacd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.252706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.253547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.253978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.254567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.255381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.264397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.265502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.311696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8add1-910f-4308-81d5-e377be4b9f08 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.315897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1044/2114] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:09 2022] POST /identity/v3/projects => generated 395 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:09.316981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.404085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f024f-e500-40f1-93d2-b0ed42670400 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.409379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.410030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.410308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.410598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.411401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.413578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.417043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.417894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.418273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.419785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.419785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.421666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.448181 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1045/2115] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:09 2022] POST /identity/v3/users => generated 304 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:09.449304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.450621 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1071/2116] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:09 2022] POST /identity/v3/users => generated 311 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:09.451524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.461610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b31fa-c084-447e-b404-0d50f918027a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.463197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f212-2a0e-4a3d-92a7-b7d2c49658c2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.468191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.468336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.468936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.469071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.469218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.469521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.470059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.470192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.470984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.471908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.474127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.475841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.486978 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1072/2117] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:09.487829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.495181 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1046/2118] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:09.496214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.500463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0d231-423b-4fa4-87db-4e15ef306233 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.512855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.513501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5c961-18a7-410c-a82c-ed0ea431bd48 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.513947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.514339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.514707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] PATH_INFO: `/v3/projects/21281fd8f1714b81a541a5711dddef9f/users/1905e658a7984328a82ac7cae4e7d5d9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.522398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.522600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.522854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.523139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] PATH_INFO: `/v3/projects/f6b93b2ffb1348d892cc31d4d5e0cc2e/users/1eaedffabf1d4df8870a2fe5aca5f8bc/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.548323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21281fd8f1714b81a541a5711dddef9f, user_id=1905e658a7984328a82ac7cae4e7d5d9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.554011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.564776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b93b2ffb1348d892cc31d4d5e0cc2e, user_id=1eaedffabf1d4df8870a2fe5aca5f8bc, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.568385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.598575 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1073/2119] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:09 2022] PUT /identity/v3/projects/21281fd8f1714b81a541a5711dddef9f/users/1905e658a7984328a82ac7cae4e7d5d9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:09.600175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.602102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1047/2120] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:09 2022] PUT /identity/v3/projects/f6b93b2ffb1348d892cc31d4d5e0cc2e/users/1eaedffabf1d4df8870a2fe5aca5f8bc/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:09.610628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.720659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.743496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ef9da-7e02-4afb-88d4-b51a66400537 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.754394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e637779-e9a5-45e0-80de-c38a5d6a1e8b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.755333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.756077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.756381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.756672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.757376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.759133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.760087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.760727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.761000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.761957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.762095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.763900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.782763 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1074/2121] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:09.787137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.806123 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1048/2122] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:09.810364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.820171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ef17-04b8-40ef-b87f-b60baf5ab72c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.828592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5c2cd-e9bc-476a-9fbb-ede7b9d60ba8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.833498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9224e-03a4-4487-b3be-86a319ed560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.834171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9224e-03a4-4487-b3be-86a319ed560c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.834445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9224e-03a4-4487-b3be-86a319ed560c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.834720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9224e-03a4-4487-b3be-86a319ed560c admin admin] PATH_INFO: `/v3/projects/f6b93b2ffb1348d892cc31d4d5e0cc2e/users/1eaedffabf1d4df8870a2fe5aca5f8bc/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.837387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9224e-03a4-4487-b3be-86a319ed560c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b93b2ffb1348d892cc31d4d5e0cc2e, user_id=1eaedffabf1d4df8870a2fe5aca5f8bc, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.840031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87778f8-4fc3-40ee-9bc3-3544786c0d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.840728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87778f8-4fc3-40ee-9bc3-3544786c0d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.840998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87778f8-4fc3-40ee-9bc3-3544786c0d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.841269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87778f8-4fc3-40ee-9bc3-3544786c0d84 admin admin] PATH_INFO: `/v3/projects/21281fd8f1714b81a541a5711dddef9f/users/1905e658a7984328a82ac7cae4e7d5d9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.842191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9224e-03a4-4487-b3be-86a319ed560c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.849402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87778f8-4fc3-40ee-9bc3-3544786c0d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21281fd8f1714b81a541a5711dddef9f, user_id=1905e658a7984328a82ac7cae4e7d5d9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.852409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87778f8-4fc3-40ee-9bc3-3544786c0d84 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.856450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1049/2123] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:09 2022] PUT /identity/v3/projects/f6b93b2ffb1348d892cc31d4d5e0cc2e/users/1eaedffabf1d4df8870a2fe5aca5f8bc/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:09.872314 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1075/2124] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:09 2022] PUT /identity/v3/projects/21281fd8f1714b81a541a5711dddef9f/users/1905e658a7984328a82ac7cae4e7d5d9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:09.872612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.872922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.873286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.883487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.883777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.884050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.911994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] MFA Rules not processed for user `1eaedffabf1d4df8870a2fe5aca5f8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:09.914572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:09.956798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] MFA Rules not processed for user `1905e658a7984328a82ac7cae4e7d5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:09.958701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:09.981038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.058815 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1050/2125] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:09 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:10.068690 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1076/2126] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:09 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:10.245134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.246797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:10.336621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.336823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d4f60-323e-4b3f-8fcf-6f84a8a4b657 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.338087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:10.345879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.346516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.346793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.347065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.356883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.358143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8f03c-4e47-47f2-8d73-4c73d572a8d5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.358533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.364749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.365230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.365652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.366042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.369957 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1051/2127] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:10.377618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.379347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.390842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1077/2128] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:51:10.605957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.626373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.644257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eb2da-1696-47e4-9e43-c69c97e7f571 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.661411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a0387-cd8e-455c-897a-0ce6b45d9461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.662695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0387-cd8e-455c-897a-0ce6b45d9461 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.663214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0387-cd8e-455c-897a-0ce6b45d9461 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.663799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0387-cd8e-455c-897a-0ce6b45d9461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.667084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.726201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfe91d-8975-45d1-86ef-65888ca61e3a service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.731140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.732078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.732355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.732739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] PATH_INFO: `/v3/users/a2d0c7a21257489fa89d0d0b60e83a2f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.734056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d0c7a21257489fa89d0d0b60e83a2f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.736475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.758086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a0387-cd8e-455c-897a-0ce6b45d9461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.760711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a0387-cd8e-455c-897a-0ce6b45d9461 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.786357 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1078/2129] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:10.859513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d0c7a21257489fa89d0d0b60e83a2f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:10.860134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d0c7a21257489fa89d0d0b60e83a2f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:10.871826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d0c7a21257489fa89d0d0b60e83a2f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:10.873838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d0c7a21257489fa89d0d0b60e83a2f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:10.907041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d0c7a21257489fa89d0d0b60e83a2f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:10.925518 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1052/2130] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:10 2022] DELETE /identity/v3/users/a2d0c7a21257489fa89d0d0b60e83a2f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:10.929512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.929793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.930067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.956675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:10.958278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:11.031207 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1079/2131] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:11.055135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.056699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:11.210400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3911a-5bf4-485d-8718-1014e0a469b7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:11.221222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.222139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.222514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.222882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.244267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.246675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.260797 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1053/2132] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:11.973849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.975960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:11.989320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.990937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:11.991626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910085-70b4-42ca-bf41-5a475ff53da3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:12.011340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:12.012577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.013378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.013838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33b04f-014f-4d9a-9395-4ef2526c49bf service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:12.015004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] PATH_INFO: `/v3/projects/e04955b84b484b11bc91ab94ba749fe1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.018748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04955b84b484b11bc91ab94ba749fe1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:12.024116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:12.027474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:12.030569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.031503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.034811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] PATH_INFO: `/v3/users/85906ce95c064aa7ab55af182215f77a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.037954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85906ce95c064aa7ab55af182215f77a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:12.050314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:12.130729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:12.159553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04955b84b484b11bc91ab94ba749fe1', 'request_id': 'req-c7484fd2-f411-41b4-8911-d4b833cfd1b6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.160037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85906ce95c064aa7ab55af182215f77a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.160998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85906ce95c064aa7ab55af182215f77a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.164585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04955b84b484b11bc91ab94ba749fe1', 'request_id': 'req-c7484fd2-f411-41b4-8911-d4b833cfd1b6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.171982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85906ce95c064aa7ab55af182215f77a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.173449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85906ce95c064aa7ab55af182215f77a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.174618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04955b84b484b11bc91ab94ba749fe1', 'request_id': 'req-c7484fd2-f411-41b4-8911-d4b833cfd1b6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.175946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:12.192544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] The token cache is being invalidate because project e04955b84b484b11bc91ab94ba749fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:12.193027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.194340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.197135 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1080/2133] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:11 2022] DELETE /identity/v3/projects/e04955b84b484b11bc91ab94ba749fe1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:12.197935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd58929e-bf29-4456-a760-d6517e765492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85906ce95c064aa7ab55af182215f77a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.201532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:12.203086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.218933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1054/2134] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:11 2022] DELETE /identity/v3/users/85906ce95c064aa7ab55af182215f77a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:12.222995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.223378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.224063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.253010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:12.255957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.294975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7484fd2-f411-41b4-8911-d4b833cfd1b6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:12.339561 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1055/2135] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:12.351091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:12.353161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.390165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:12.394210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.394498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.396259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] PATH_INFO: `/v3/users/1eaedffabf1d4df8870a2fe5aca5f8bc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.398564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eaedffabf1d4df8870a2fe5aca5f8bc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:12.400751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:12.425495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f0225-25ac-4149-926b-decf5de4f4ab None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:12.433325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:12.434009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.434311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.434589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.444597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:12.446096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:12.459620 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1056/2136] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:12.466166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:12.467707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.482256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916af3ad-ae12-4819-b6f4-0035f6728b37 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:12.489212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:12.490257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.490550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.490822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] PATH_INFO: `/v3/users/23cc6d06eba24928a2e50e1d1e6c4c8a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.492216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23cc6d06eba24928a2e50e1d1e6c4c8a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:12.495030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:12.500343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eaedffabf1d4df8870a2fe5aca5f8bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.519896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eaedffabf1d4df8870a2fe5aca5f8bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.529014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:12.542393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eaedffabf1d4df8870a2fe5aca5f8bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.543800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eaedffabf1d4df8870a2fe5aca5f8bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.545080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eaedffabf1d4df8870a2fe5aca5f8bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.575248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23cc6d06eba24928a2e50e1d1e6c4c8a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.576500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23cc6d06eba24928a2e50e1d1e6c4c8a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.593683 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1081/2137] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:12 2022] DELETE /identity/v3/users/1eaedffabf1d4df8870a2fe5aca5f8bc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:12.593971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23cc6d06eba24928a2e50e1d1e6c4c8a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.594696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23cc6d06eba24928a2e50e1d1e6c4c8a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.596310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:12.603710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.617226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:12.619612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4af38702-48ca-4969-a007-bd50d2b8ab8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23cc6d06eba24928a2e50e1d1e6c4c8a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.651904 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1057/2138] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:12 2022] DELETE /identity/v3/users/23cc6d06eba24928a2e50e1d1e6c4c8a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:12.657769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.657939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.658214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.704081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:12.705749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.769090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4fc96-96b2-4af5-b565-b267e123fc24 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:12.855470 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1058/2139] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:12.857184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:12.861214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.871525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:12.872183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.872462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.872733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] PATH_INFO: `/v3/users/c46036360ef3496c9104d08a7251ac6b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.874790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46036360ef3496c9104d08a7251ac6b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:12.883858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:12.898796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977a46e-cbd1-49cc-88de-1a7a521de115 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:12.927638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:12.929989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.930344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.930616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] PATH_INFO: `/v3/projects/e0f0332723fc47db82faca60313db4ca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.932736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f0332723fc47db82faca60313db4ca)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:12.935799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:12.974614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46036360ef3496c9104d08a7251ac6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:12.993867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46036360ef3496c9104d08a7251ac6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.006598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46036360ef3496c9104d08a7251ac6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.007473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46036360ef3496c9104d08a7251ac6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.007961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46036360ef3496c9104d08a7251ac6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.040042 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1082/2140] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:12 2022] DELETE /identity/v3/users/c46036360ef3496c9104d08a7251ac6b => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:13.041367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.043149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.046169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f0332723fc47db82faca60313db4ca', 'request_id': 'req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.077494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:13.078909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f0332723fc47db82faca60313db4ca', 'request_id': 'req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.097118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f0332723fc47db82faca60313db4ca', 'request_id': 'req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.149297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] The token cache is being invalidate because project e0f0332723fc47db82faca60313db4ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:13.151369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.155510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ae8d3a10-0e44-4ee7-8977-e9c108eb6585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.156104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920aedf-d912-41f9-972a-2a19d21a512d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.161532 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1059/2141] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:12 2022] DELETE /identity/v3/projects/e0f0332723fc47db82faca60313db4ca => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:13.166514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.167804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.168319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.173065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.173769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.174681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.175185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.177770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.236304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:13.239139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.326364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:13.327893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:13.340055 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1083/2142] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:13.341008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.342361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.363510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12835c-5abd-4ff3-aba4-abf88229da64 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.369752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.370422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.370943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.371222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] PATH_INFO: `/v3/users/52c58e664bdd4e459c4d1e82e43f127b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.372986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c58e664bdd4e459c4d1e82e43f127b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:13.375165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:13.393221 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1060/2143] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:13.397037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.401761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.425804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2f11c-fe61-41d6-b74d-89ecdec68879 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.455515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.456271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.456665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.456974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] PATH_INFO: `/v3/projects/f6b93b2ffb1348d892cc31d4d5e0cc2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.458191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b93b2ffb1348d892cc31d4d5e0cc2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:13.459651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c58e664bdd4e459c4d1e82e43f127b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.462104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:13.482429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:13.488600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c58e664bdd4e459c4d1e82e43f127b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.508574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c58e664bdd4e459c4d1e82e43f127b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.509423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c58e664bdd4e459c4d1e82e43f127b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.510180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c58e664bdd4e459c4d1e82e43f127b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.577807 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1084/2144] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:13 2022] DELETE /identity/v3/users/52c58e664bdd4e459c4d1e82e43f127b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:13.579250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.583467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.617034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b93b2ffb1348d892cc31d4d5e0cc2e', 'request_id': 'req-18613721-e21c-45d7-95f4-e256a55be055'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.626801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b93b2ffb1348d892cc31d4d5e0cc2e', 'request_id': 'req-18613721-e21c-45d7-95f4-e256a55be055'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.641298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b93b2ffb1348d892cc31d4d5e0cc2e', 'request_id': 'req-18613721-e21c-45d7-95f4-e256a55be055'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.657983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] The token cache is being invalidate because project f6b93b2ffb1348d892cc31d4d5e0cc2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:13.658585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.659593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.665772 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1061/2145] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:13 2022] DELETE /identity/v3/projects/f6b93b2ffb1348d892cc31d4d5e0cc2e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:13.667159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.669007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.681152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:13.747698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40188ab-0557-4d4a-b982-72eff9979933 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.754829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18613721-e21c-45d7-95f4-e256a55be055 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.759667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595d0bc-995c-45c8-83e9-fdf138aa6931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.760699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d0bc-995c-45c8-83e9-fdf138aa6931 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.760699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d0bc-995c-45c8-83e9-fdf138aa6931 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.760956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d0bc-995c-45c8-83e9-fdf138aa6931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.762874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1595d0bc-995c-45c8-83e9-fdf138aa6931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.843367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.844125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.844503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.845220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] PATH_INFO: `/v3/users/f05aab669547487d8876a70e338d3e4a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.847539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05aab669547487d8876a70e338d3e4a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:13.849802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595d0bc-995c-45c8-83e9-fdf138aa6931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:13.850278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:13.851944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595d0bc-995c-45c8-83e9-fdf138aa6931 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:13.868734 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1062/2146] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:13.870262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baed0c6-c8a6-4adf-9c7c-6204bfc28521 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.870967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baed0c6-c8a6-4adf-9c7c-6204bfc28521 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.871524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baed0c6-c8a6-4adf-9c7c-6204bfc28521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.895861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5baed0c6-c8a6-4adf-9c7c-6204bfc28521 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:13.897439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5baed0c6-c8a6-4adf-9c7c-6204bfc28521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.917916 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1063/2147] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:13.919795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a19d3-104f-453d-9624-2964ea0e4854 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.920226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a19d3-104f-453d-9624-2964ea0e4854 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.920595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a19d3-104f-453d-9624-2964ea0e4854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.942712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05aab669547487d8876a70e338d3e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.944674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-839a19d3-104f-453d-9624-2964ea0e4854 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:13.946148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-839a19d3-104f-453d-9624-2964ea0e4854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.957501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1064/2148] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:13.963456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a19d3-104f-453d-9624-2964ea0e4854 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.963914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05aab669547487d8876a70e338d3e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.976811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a19d3-104f-453d-9624-2964ea0e4854 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.977734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05aab669547487d8876a70e338d3e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.978985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05aab669547487d8876a70e338d3e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.980465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f3c33b1-4b89-4105-ab89-b9e4d484a879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05aab669547487d8876a70e338d3e4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:13.992079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.993032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.993437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.993869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.994710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:13.995652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:13.998019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.019235 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1085/2149] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:13 2022] DELETE /identity/v3/users/f05aab669547487d8876a70e338d3e4a => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:14.023087 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1065/2150] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:14.024150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b559-8d16-4f63-8e1f-4f1532b8b945 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.024435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b559-8d16-4f63-8e1f-4f1532b8b945 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.024711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b559-8d16-4f63-8e1f-4f1532b8b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.027486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.029673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.043538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38264032-b54e-474a-bb35-1b349583fe95 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.054574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0310b559-8d16-4f63-8e1f-4f1532b8b945 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:14.056790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0310b559-8d16-4f63-8e1f-4f1532b8b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.058365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.127502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.128141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.128502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.128688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.129733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.132495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.177756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.187602 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1066/2151] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:14 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:14.189410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.197805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.207567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1086/2152] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:14.207567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310b559-8d16-4f63-8e1f-4f1532b8b945 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.234761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310b559-8d16-4f63-8e1f-4f1532b8b945 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.330456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef1df-dac8-4668-9271-1980aa1dc4de admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.338294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.339271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.339911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.340477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] PATH_INFO: `/v3/projects/e448135a65c349d19b25015d016f6e3f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.342042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e448135a65c349d19b25015d016f6e3f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.344357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.344778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.345104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.345386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.345684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.346479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.350230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.388081 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1087/2153] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:14 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:14.389241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.391320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.441877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e448135a65c349d19b25015d016f6e3f', 'request_id': 'req-78f4dccc-2027-47d8-9c51-a0ccd75aa996'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.450542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e448135a65c349d19b25015d016f6e3f', 'request_id': 'req-78f4dccc-2027-47d8-9c51-a0ccd75aa996'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.471131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e448135a65c349d19b25015d016f6e3f', 'request_id': 'req-78f4dccc-2027-47d8-9c51-a0ccd75aa996'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.492059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] The token cache is being invalidate because project e448135a65c349d19b25015d016f6e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:14.492401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.493865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.498412 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1067/2154] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:14 2022] DELETE /identity/v3/projects/e448135a65c349d19b25015d016f6e3f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:14.506203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.507948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.534364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938991c-37d4-447a-877a-bca770704fb1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.552597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.590788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4dccc-2027-47d8-9c51-a0ccd75aa996 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.595845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c7f62-405a-498a-9060-0f666b1afce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.596477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7f62-405a-498a-9060-0f666b1afce3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.596756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7f62-405a-498a-9060-0f666b1afce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.596858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.597055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c7f62-405a-498a-9060-0f666b1afce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.597385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.597698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.597968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.598617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.599111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5a2c7f62-405a-498a-9060-0f666b1afce3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.600342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.611709 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1068/2155] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:14.612645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.616303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c7f62-405a-498a-9060-0f666b1afce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.616430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.617793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c7f62-405a-498a-9060-0f666b1afce3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.628829 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1088/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:14.630638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.630918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.631186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.640454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92cb99-7ece-4edd-9eb4-c028cf4801a3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.647115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.647884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.648341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.648703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] PATH_INFO: `/v3/projects/2378d365d8774876b60c6f00a61ffae4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.650030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2378d365d8774876b60c6f00a61ffae4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.651079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:14.652307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.652441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.669592 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1089/2157] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:14.673528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.673528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.684486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19c9ad-d578-4489-bf49-36d9a35b1760 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.689834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.690479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.691225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.691225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] PATH_INFO: `/v3/projects/cb661478603944d4b3fef3ef84a2eaba` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.692134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb661478603944d4b3fef3ef84a2eaba)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.694289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.739917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2378d365d8774876b60c6f00a61ffae4', 'request_id': 'req-56de2431-dc24-4e21-8777-ebc8c66bfb1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.748553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2378d365d8774876b60c6f00a61ffae4', 'request_id': 'req-56de2431-dc24-4e21-8777-ebc8c66bfb1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.755445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.759335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2378d365d8774876b60c6f00a61ffae4', 'request_id': 'req-56de2431-dc24-4e21-8777-ebc8c66bfb1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.775085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] The token cache is being invalidate because project 2378d365d8774876b60c6f00a61ffae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:14.775375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.776092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.780916 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1069/2158] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:14 2022] DELETE /identity/v3/projects/2378d365d8774876b60c6f00a61ffae4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:14.780916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.781243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.785993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb661478603944d4b3fef3ef84a2eaba', 'request_id': 'req-36038360-bf8d-442c-9caa-517496e2acba'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.795826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb661478603944d4b3fef3ef84a2eaba', 'request_id': 'req-36038360-bf8d-442c-9caa-517496e2acba'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.803082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb661478603944d4b3fef3ef84a2eaba', 'request_id': 'req-36038360-bf8d-442c-9caa-517496e2acba'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.828306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] The token cache is being invalidate because project cb661478603944d4b3fef3ef84a2eaba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:14.828615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.829729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36038360-bf8d-442c-9caa-517496e2acba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:14.832830 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1090/2159] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:14 2022] DELETE /identity/v3/projects/cb661478603944d4b3fef3ef84a2eaba => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:14.835725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.836297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.836576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.860012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:14.861383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.903769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de2431-dc24-4e21-8777-ebc8c66bfb1c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.945674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1091/2160] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:14.945674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.948113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.961676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f088b8-2ba3-4252-b666-9fbd3e28c752 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.967484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.967954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.968146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.968283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.968488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] PATH_INFO: `/v3/users/7fbe8c0f8b184630abba1cb6bf8673f1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.968855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.969178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.969446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.969717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fbe8c0f8b184630abba1cb6bf8673f1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.970112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.971719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.971874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.973510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.985656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1070/2161] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:14.987665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.992312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.007635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc24e5-855e-4f23-a4d3-918e2aec7ec7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.014382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.015401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.015794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.016138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] PATH_INFO: `/v3/users/1905e658a7984328a82ac7cae4e7d5d9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.017905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1905e658a7984328a82ac7cae4e7d5d9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.020452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.028370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fbe8c0f8b184630abba1cb6bf8673f1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.044406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fbe8c0f8b184630abba1cb6bf8673f1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.057158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fbe8c0f8b184630abba1cb6bf8673f1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.058569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fbe8c0f8b184630abba1cb6bf8673f1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.062250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fbe8c0f8b184630abba1cb6bf8673f1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.081354 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1092/2162] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:14 2022] DELETE /identity/v3/users/7fbe8c0f8b184630abba1cb6bf8673f1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:15.090036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.091716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.092246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1905e658a7984328a82ac7cae4e7d5d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.092990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1905e658a7984328a82ac7cae4e7d5d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.108468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1905e658a7984328a82ac7cae4e7d5d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.111124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1905e658a7984328a82ac7cae4e7d5d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.134144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1905e658a7984328a82ac7cae4e7d5d9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.156694 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1071/2163] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:14 2022] DELETE /identity/v3/users/1905e658a7984328a82ac7cae4e7d5d9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:15.157873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.159532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.161284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.205132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383737fb-5a58-416c-b2f7-bb9366fd92a4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.264124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f473497-ccf0-41a1-aa87-92b5e15fed7a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.271218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.271942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.272280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.272617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.273510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:15.273940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.275689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.279330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.279982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.280413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.280600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] PATH_INFO: `/v3/projects/aa4ffd27d7414061b67e053657044400` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.281943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4ffd27d7414061b67e053657044400)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.284394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.289622 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1072/2164] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.290715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.295073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.310223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9196046-12c8-41f4-a851-3397e387413b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.315766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.316627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.316887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.317199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] PATH_INFO: `/v3/projects/b01f4e11e8674f35a6781fa05a0a727f/users/672871e1c5d84e8b8b5eb8ccb66b9730/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.334511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01f4e11e8674f35a6781fa05a0a727f, user_id=672871e1c5d84e8b8b5eb8ccb66b9730, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.337947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.346178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4ffd27d7414061b67e053657044400', 'request_id': 'req-07febfde-32cc-431b-97bb-f73e859000d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.354436 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1073/2165] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:15 2022] PUT /identity/v3/projects/b01f4e11e8674f35a6781fa05a0a727f/users/672871e1c5d84e8b8b5eb8ccb66b9730/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:15.355312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.355444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4ffd27d7414061b67e053657044400', 'request_id': 'req-07febfde-32cc-431b-97bb-f73e859000d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.363220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4ffd27d7414061b67e053657044400', 'request_id': 'req-07febfde-32cc-431b-97bb-f73e859000d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.377714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] The token cache is being invalidate because project aa4ffd27d7414061b67e053657044400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:15.378008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.378721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07febfde-32cc-431b-97bb-f73e859000d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.381818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1093/2166] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:15 2022] DELETE /identity/v3/projects/aa4ffd27d7414061b67e053657044400 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:15.385044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.385434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.386132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.413981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.423663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] MFA Rules not processed for user `672871e1c5d84e8b8b5eb8ccb66b9730`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:15.425066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.450512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d740b-d5da-47ef-bcf3-1a4ec1ce0b7c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.462271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.463278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.463603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.464409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.465391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.475500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.555592 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1074/2167] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:15 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:15.556834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.567204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.581525 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1094/2168] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:15.584368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.588542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.609768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.622018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82249d-495e-4de5-bdfb-78c1cf1b8d3c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.712193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf1115-5227-48e0-9513-871e8bcdac61 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.721126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.722014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.722393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.722836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] PATH_INFO: `/v3/users/553cf96465f34dbcb6cc26d9fe1bd72e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.723163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.723904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.724245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.724515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.724646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=553cf96465f34dbcb6cc26d9fe1bd72e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.725299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.726873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.727555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.803271 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1095/2169] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:15 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:15.817743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.837965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.862216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97932dc4-3998-49d1-b88f-74f88eabbdd6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.871676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.872315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.872643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.872984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.873417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553cf96465f34dbcb6cc26d9fe1bd72e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.873760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.875276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553cf96465f34dbcb6cc26d9fe1bd72e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.876007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.893390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553cf96465f34dbcb6cc26d9fe1bd72e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.894562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553cf96465f34dbcb6cc26d9fe1bd72e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.917374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553cf96465f34dbcb6cc26d9fe1bd72e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:15.922918 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1096/2170] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:15 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:15.923988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.926070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:15.944731 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1075/2171] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:15 2022] DELETE /identity/v3/users/553cf96465f34dbcb6cc26d9fe1bd72e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:15.945488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.947088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.012147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.036743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7cb5-daee-4ffc-b326-1aecf8882a23 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.037342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a828f-453b-4b91-87cb-cd75184dd4d0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.043068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.043238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.043864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.044071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.044372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.044501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.044797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] PATH_INFO: `/v3/projects/21281fd8f1714b81a541a5711dddef9f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.045256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] PATH_INFO: `/v3/projects/575dbc45397b43c4a14811f2ef69616c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.045999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=21281fd8f1714b81a541a5711dddef9f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.046463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575dbc45397b43c4a14811f2ef69616c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.047918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.049116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.106130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575dbc45397b43c4a14811f2ef69616c', 'request_id': 'req-259b1c57-7a5e-4dbd-b175-f207857a6ee1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.110204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21281fd8f1714b81a541a5711dddef9f', 'request_id': 'req-7394d666-c6e9-4c21-bfa8-f047433be9ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.115721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575dbc45397b43c4a14811f2ef69616c', 'request_id': 'req-259b1c57-7a5e-4dbd-b175-f207857a6ee1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.122148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575dbc45397b43c4a14811f2ef69616c', 'request_id': 'req-259b1c57-7a5e-4dbd-b175-f207857a6ee1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.126576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21281fd8f1714b81a541a5711dddef9f', 'request_id': 'req-7394d666-c6e9-4c21-bfa8-f047433be9ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.131809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.137111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] The token cache is being invalidate because project 575dbc45397b43c4a14811f2ef69616c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:16.137761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.138432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21281fd8f1714b81a541a5711dddef9f', 'request_id': 'req-7394d666-c6e9-4c21-bfa8-f047433be9ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.139121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.141549 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1097/2172] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:15 2022] DELETE /identity/v3/projects/575dbc45397b43c4a14811f2ef69616c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:16.142968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.145735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.159306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] The token cache is being invalidate because project 21281fd8f1714b81a541a5711dddef9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:16.159596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.160282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.162616 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1076/2173] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:15 2022] DELETE /identity/v3/projects/21281fd8f1714b81a541a5711dddef9f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:16.168215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.170745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.233762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b1c57-7a5e-4dbd-b175-f207857a6ee1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.244192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469d533-6a8a-4c73-8c95-bf2e316496e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.245102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d533-6a8a-4c73-8c95-bf2e316496e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.245356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d533-6a8a-4c73-8c95-bf2e316496e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.245726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469d533-6a8a-4c73-8c95-bf2e316496e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.246860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d533-6a8a-4c73-8c95-bf2e316496e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.248752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469d533-6a8a-4c73-8c95-bf2e316496e3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.264715 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1098/2174] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:16.266208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394d666-c6e9-4c21-bfa8-f047433be9ff admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.266649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087db33d-cd8f-4687-866a-3052811a3e0a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.266935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087db33d-cd8f-4687-866a-3052811a3e0a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.267210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087db33d-cd8f-4687-866a-3052811a3e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.272305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.273085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.273564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.274437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.275191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.280039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.293736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-087db33d-cd8f-4687-866a-3052811a3e0a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:16.295820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-087db33d-cd8f-4687-866a-3052811a3e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.305064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1099/2175] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:16.306612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.307080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.307527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.311783 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1077/2176] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:16.312870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.314955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.327225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc78ff-c993-4aad-9fe6-aa0499755761 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.330962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:16.332162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.335670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.336392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.336729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.337064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] PATH_INFO: `/v3/projects/13bb5c9671214f449194584247b12e4d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.338279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bb5c9671214f449194584247b12e4d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.346508 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1100/2177] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:16.346691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.346783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.346783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.354017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9621c7b-ec1b-44fd-9bee-887e1a1ee11f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.358732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.359343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.359617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.359885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.360535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.362327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.375913 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1101/2178] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:16.375913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.393533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4501a-3998-4ed9-8b23-d08e3a5dedac admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.403916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bb5c9671214f449194584247b12e4d', 'request_id': 'req-c836ddcc-c78b-4650-82bc-608ea315e0a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.412132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bb5c9671214f449194584247b12e4d', 'request_id': 'req-c836ddcc-c78b-4650-82bc-608ea315e0a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.419642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=11,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.438080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bb5c9671214f449194584247b12e4d', 'request_id': 'req-c836ddcc-c78b-4650-82bc-608ea315e0a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.450829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] The token cache is being invalidate because project 13bb5c9671214f449194584247b12e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:16.451119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.451773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c836ddcc-c78b-4650-82bc-608ea315e0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:16.453294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1078/2179] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:16 2022] DELETE /identity/v3/projects/13bb5c9671214f449194584247b12e4d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:16.454520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.454794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.455067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.460357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f9816-b831-4405-9612-d5d2918771e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.461236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9816-b831-4405-9612-d5d2918771e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.461629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9816-b831-4405-9612-d5d2918771e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.461857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9816-b831-4405-9612-d5d2918771e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.462505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f9816-b831-4405-9612-d5d2918771e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.464212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f9816-b831-4405-9612-d5d2918771e4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.474712 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1102/2180] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:16.477172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:16.478673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.479459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.479735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.480091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.490274 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1079/2181] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:16.490274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.490274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.500094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:16.501285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.501912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074602ad-3aab-4f6c-945e-069d3463c9fe None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.506398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025db214-7142-4873-8a23-a8db075bbc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.507094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025db214-7142-4873-8a23-a8db075bbc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.507369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025db214-7142-4873-8a23-a8db075bbc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.507639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025db214-7142-4873-8a23-a8db075bbc34 admin admin] PATH_INFO: `/v3/projects/06f2b7e4e205458b8cf6f19a43b77296/users/5aae6d573509428db8d4d5db9d7b005c/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.510730 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1103/2182] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:16.511505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.513551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.527692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025db214-7142-4873-8a23-a8db075bbc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f2b7e4e205458b8cf6f19a43b77296, user_id=5aae6d573509428db8d4d5db9d7b005c, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.531211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025db214-7142-4873-8a23-a8db075bbc34 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.545987 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1080/2183] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:16 2022] PUT /identity/v3/projects/06f2b7e4e205458b8cf6f19a43b77296/users/5aae6d573509428db8d4d5db9d7b005c/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:16.549834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.549834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.549834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.566962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:16.568172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.577857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.594976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f20937-1a42-4821-8f8e-07bab99087b5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.628922 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1081/2184] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:16.629873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.631303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.643786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5809-da6e-47d7-b917-bdda13c9f976 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.649478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.650169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.650447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.650748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] PATH_INFO: `/v3/projects/eb972eda012a48b196dc9fa3b82aefe8/users/6ddcba1504da47d58fbaead731fa1200/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.661261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.661912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.662196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.662470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.664262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.666018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb972eda012a48b196dc9fa3b82aefe8, user_id=6ddcba1504da47d58fbaead731fa1200, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.668804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.682742 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1082/2185] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:16 2022] PUT /identity/v3/projects/eb972eda012a48b196dc9fa3b82aefe8/users/6ddcba1504da47d58fbaead731fa1200/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:16.683618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.736535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.738218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.748226 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1104/2186] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:16.749127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.758110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40468109-d0bf-4b27-8f18-441dfd06db0a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.760416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dd2a3-179a-4a5e-b9fe-babd30441389 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.763276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f66c5f-f183-4b65-9ee3-3142c3666b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.764011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f66c5f-f183-4b65-9ee3-3142c3666b7b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.764292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f66c5f-f183-4b65-9ee3-3142c3666b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.764565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f66c5f-f183-4b65-9ee3-3142c3666b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.765383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecaf82-13f3-4403-864a-191b9e49ae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.765544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f66c5f-f183-4b65-9ee3-3142c3666b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:16.765716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f66c5f-f183-4b65-9ee3-3142c3666b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.766011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecaf82-13f3-4403-864a-191b9e49ae49 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.766285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecaf82-13f3-4403-864a-191b9e49ae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.766553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecaf82-13f3-4403-864a-191b9e49ae49 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.767252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecaf82-13f3-4403-864a-191b9e49ae49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:16.767357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f66c5f-f183-4b65-9ee3-3142c3666b7b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.767549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecaf82-13f3-4403-864a-191b9e49ae49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.769138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecaf82-13f3-4403-864a-191b9e49ae49 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.781294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1083/2187] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:16.782889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a51cb-b691-458c-a839-68802fdfba30 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.783419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a51cb-b691-458c-a839-68802fdfba30 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.783880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a51cb-b691-458c-a839-68802fdfba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.785475 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1105/2188] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:16.786822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12a40b-d00f-4ff0-a9c3-5f21e005ad91 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.787109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12a40b-d00f-4ff0-a9c3-5f21e005ad91 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.787382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12a40b-d00f-4ff0-a9c3-5f21e005ad91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.807625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-671a51cb-b691-458c-a839-68802fdfba30 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:16.808953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-671a51cb-b691-458c-a839-68802fdfba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.809989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5e12a40b-d00f-4ff0-a9c3-5f21e005ad91 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:16.811159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e12a40b-d00f-4ff0-a9c3-5f21e005ad91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.816797 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1084/2189] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:16.817432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a51cb-b691-458c-a839-68802fdfba30 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.819714 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1106/2190] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:16.826668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12a40b-d00f-4ff0-a9c3-5f21e005ad91 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.828237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a51cb-b691-458c-a839-68802fdfba30 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.834345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.835330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.835778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.836205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.837379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.839828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12a40b-d00f-4ff0-a9c3-5f21e005ad91 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.840030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.844979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.845650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.846193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.846193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.846813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.848838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.859211 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1085/2191] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:16.859211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.874727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e233-d20b-4424-a128-797c52caf7dc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.879861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.880531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.880804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.881257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.882050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.884017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.891889 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1107/2192] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/projects => generated 399 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:16.892074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.892167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.894110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:16.934763 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1086/2193] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:16 2022] POST /identity/v3/projects => generated 397 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:16.935739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.982580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956c2f8-af48-4b88-b772-03b0c8265835 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.986935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:17.014740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519503ba-7ce5-4963-93c2-359731192a22 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.019805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.020529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.020925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.021296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.022078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:17.022477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.024403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.045165 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1087/2194] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:17.046418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.049304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.050076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.050426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.050893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.058953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b299a-16b5-45bc-b6d1-373d5c41424e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.063426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.063834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.064516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.064792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.065274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.065389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] PATH_INFO: `/v3/projects/eb972eda012a48b196dc9fa3b82aefe8/users/6ddcba1504da47d58fbaead731fa1200/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.068128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb972eda012a48b196dc9fa3b82aefe8, user_id=6ddcba1504da47d58fbaead731fa1200, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.070810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.089876 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1108/2195] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:17.091231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.098624 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1088/2196] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:17 2022] PUT /identity/v3/projects/eb972eda012a48b196dc9fa3b82aefe8/users/6ddcba1504da47d58fbaead731fa1200/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:17.103960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.115139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd697df-2595-4abe-b146-2f776e6a6133 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.188878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b9f6f-d3f1-4798-ad0f-8d4ccf5cd956 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.195980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.196797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.197076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.197497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.199002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.201128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.203239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742cfe9-27b8-45c5-877f-60428a8c65d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.204246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742cfe9-27b8-45c5-877f-60428a8c65d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.204768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742cfe9-27b8-45c5-877f-60428a8c65d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.205155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742cfe9-27b8-45c5-877f-60428a8c65d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.207049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742cfe9-27b8-45c5-877f-60428a8c65d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.212677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742cfe9-27b8-45c5-877f-60428a8c65d6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.237389 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1089/2197] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:17 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:17.237389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.243930 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1109/2198] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:17 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:17.243930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de407b95-2b1a-45e9-bf26-bde1890c3d61 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.243930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de407b95-2b1a-45e9-bf26-bde1890c3d61 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.243930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de407b95-2b1a-45e9-bf26-bde1890c3d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.250008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5416c-5180-411d-aee7-61808e70260f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.262095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.263037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.263495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.263935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.264830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.267018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.275486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-de407b95-2b1a-45e9-bf26-bde1890c3d61 None None] MFA Rules not processed for user `6ddcba1504da47d58fbaead731fa1200`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:17.277234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-de407b95-2b1a-45e9-bf26-bde1890c3d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:17.322796 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1090/2199] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:17 2022] POST /identity/v3/projects => generated 399 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:17.322796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.364330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:17.376571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1110/2200] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:17.377582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de407b95-2b1a-45e9-bf26-bde1890c3d61 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.388483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de407b95-2b1a-45e9-bf26-bde1890c3d61 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.399292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c6b5b-6696-485c-862f-9ab813b84105 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.407628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.408308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.408657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.409029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.409783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.411622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.426775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:17.429037 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1091/2201] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:17.430266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.454749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19ad3-a61f-4c89-9e72-6848cb8be345 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.463299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.464039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.464765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.465046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.466498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.470565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.492422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.493151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.493887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.494224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.495909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.497803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.510011 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1111/2202] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:17.510804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.517380 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1092/2203] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:17 2022] POST /identity/v3/users => generated 300 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:17.519421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.537556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2137a97-0cae-4765-9383-477904f3c5ee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.543116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.544349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.544800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.545223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.547281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.550319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.574992 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1093/2204] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:17.577030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.591104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356448c-a265-48fa-8cce-76ea4c1b7dc8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.591798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bdd4d-88aa-4085-8ba6-feb27f4b6d0b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.598131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.598889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.599238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.599654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.600422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:17.600795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.602723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.603180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.603351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.603630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.603899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.604606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.606665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.619826 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1094/2205] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:17.622002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.623116 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1112/2206] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:17.624231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.635621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9a2f-d13b-4173-8dd6-0e175bfb347d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.644611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a6f3c-d531-4363-b0a2-dfe3fb081b4b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.650011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.650145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.650683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.650803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.650976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.651169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.651280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] PATH_INFO: `/v3/projects/7dc8bc81c2d74bae9612f7c9807350a7/users/f1134231b4a545fdb136a5204cb4f289/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.651460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.652145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.654247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.668275 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1095/2207] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:17.669271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.683347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f91a-9f1c-4e2a-9a39-536e889bce9e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.688630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.689579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.695153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.695379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.696080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.698572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.711121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc8bc81c2d74bae9612f7c9807350a7, user_id=f1134231b4a545fdb136a5204cb4f289, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.714442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.739897 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1113/2208] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:17 2022] PUT /identity/v3/projects/7dc8bc81c2d74bae9612f7c9807350a7/users/f1134231b4a545fdb136a5204cb4f289/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:17.740472 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1096/2209] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:17 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:17.741235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.741554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.833132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43770f79-1c81-4a9b-9165-f2e35aec5b1e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.839235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622d759-40ba-4836-b429-59d69ff49e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.839899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622d759-40ba-4836-b429-59d69ff49e1c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.840175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622d759-40ba-4836-b429-59d69ff49e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.840442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622d759-40ba-4836-b429-59d69ff49e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.841085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622d759-40ba-4836-b429-59d69ff49e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.842721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622d759-40ba-4836-b429-59d69ff49e1c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.853557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04301337-98f1-488e-924b-a60d777ca408 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.859768 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1097/2210] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:17.859768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf914-f925-4720-b98d-d21cf3e09b73 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.859768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf914-f925-4720-b98d-d21cf3e09b73 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.859768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf914-f925-4720-b98d-d21cf3e09b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.861790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.862599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.863436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.863714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] PATH_INFO: `/v3/projects/75e1743a6fc742b094552a4f35de8110/users/bef3da9984ab4171955b0bdfdaa259f2/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:17.878786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-66ecf914-f925-4720-b98d-d21cf3e09b73 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:17.879991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-66ecf914-f925-4720-b98d-d21cf3e09b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:17.882863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e1743a6fc742b094552a4f35de8110, user_id=bef3da9984ab4171955b0bdfdaa259f2, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:17.886881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:17.888766 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1098/2211] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:17.889665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecf914-f925-4720-b98d-d21cf3e09b73 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.907127 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1114/2212] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:17 2022] PUT /identity/v3/projects/75e1743a6fc742b094552a4f35de8110/users/bef3da9984ab4171955b0bdfdaa259f2/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:17.908215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:17.936115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:17.949304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:17.967931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecf914-f925-4720-b98d-d21cf3e09b73 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.991913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af5e06-7f6a-4960-9ac6-70c328b5562a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:17.997310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:17.998149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:17.998435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:17.998708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] PATH_INFO: `/v3/projects/5c06b859ea6d468bb45b4c6b9633d368/users/e075fe44101842fab2d6be231517cd5f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.016180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c06b859ea6d468bb45b4c6b9633d368, user_id=e075fe44101842fab2d6be231517cd5f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.019037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.034832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24401d15-7bfd-4b40-bfdd-0ff7f5177c5e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.035410 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1115/2213] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:17 2022] PUT /identity/v3/projects/5c06b859ea6d468bb45b4c6b9633d368/users/e075fe44101842fab2d6be231517cd5f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:18.035602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24401d15-7bfd-4b40-bfdd-0ff7f5177c5e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.035884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24401d15-7bfd-4b40-bfdd-0ff7f5177c5e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.036236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24401d15-7bfd-4b40-bfdd-0ff7f5177c5e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.036903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24401d15-7bfd-4b40-bfdd-0ff7f5177c5e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:18.037213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24401d15-7bfd-4b40-bfdd-0ff7f5177c5e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.038925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24401d15-7bfd-4b40-bfdd-0ff7f5177c5e tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.046033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.048354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:18.055102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1099/2214] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:18.055102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ab1b-b373-48b2-bdec-ff50b26e38cc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.055102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ab1b-b373-48b2-bdec-ff50b26e38cc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.055102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ab1b-b373-48b2-bdec-ff50b26e38cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.086415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4468ab1b-b373-48b2-bdec-ff50b26e38cc None None] MFA Rules not processed for user `bef3da9984ab4171955b0bdfdaa259f2`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:18.088045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4468ab1b-b373-48b2-bdec-ff50b26e38cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:18.164454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de9176-4269-4a36-94c5-e912ad7f7629 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.176180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.177175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.177579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.177878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.211521 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1100/2215] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:18.213144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d6c2-9710-4fe9-b814-a500c079238f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.213677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d6c2-9710-4fe9-b814-a500c079238f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.214459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d6c2-9710-4fe9-b814-a500c079238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.270603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-43c6d6c2-9710-4fe9-b814-a500c079238f None None] MFA Rules not processed for user `e075fe44101842fab2d6be231517cd5f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:18.272071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43c6d6c2-9710-4fe9-b814-a500c079238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:18.324036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.326560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.355949 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1116/2216] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:18.356854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.361500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:18.389527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1101/2217] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:18.396218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d6c2-9710-4fe9-b814-a500c079238f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.413261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d6c2-9710-4fe9-b814-a500c079238f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.423229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.424140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.424543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.425892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.426720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.429237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.438849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:18.499466 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1102/2218] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:18 2022] POST /identity/v3/projects => generated 397 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:18.500864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.509125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:18.562408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bfbce-b8c0-4ba9-ac9c-1bb752fbc806 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.644604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb10c2-cff5-4727-9c1e-656098487d32 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.651241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.651672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.652163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.652269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.653726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.655802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.700449 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1103/2219] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:18 2022] POST /identity/v3/projects => generated 399 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:18.701302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.721414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.722067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.722341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.723877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.785202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1de1a-7e95-4827-a0da-3147cf5f227b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.800301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.807497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.808391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.808671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.809337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.817251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.851948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.855057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.868577 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1117/2220] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:18.873454 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1104/2221] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:18 2022] POST /identity/v3/users => generated 305 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:18.873454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.887717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.893063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4eaa0-b069-47ce-805b-0674366857ae admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.907651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff0ed6-9ae9-42f8-a4bb-e34ed6438cb0 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.912315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.912955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.913232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.913522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.914163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.915348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd3022-22c6-4eb0-99ad-8b46477e9b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.916201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3022-22c6-4eb0-99ad-8b46477e9b57 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.916564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.917019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3022-22c6-4eb0-99ad-8b46477e9b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.918029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3022-22c6-4eb0-99ad-8b46477e9b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.919704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd3022-22c6-4eb0-99ad-8b46477e9b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:18.922359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd3022-22c6-4eb0-99ad-8b46477e9b57 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:18.940217 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1118/2222] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:18.942273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4055d-8211-4017-90c6-15f4e6cadfda None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.943611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4055d-8211-4017-90c6-15f4e6cadfda None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.944223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4055d-8211-4017-90c6-15f4e6cadfda None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:18.954754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1119/2223] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:18 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:18.956888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4055d-8211-4017-90c6-15f4e6cadfda None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:18.976112 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1105/2224] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:18 2022] POST /identity/v3/users => generated 306 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:18.976927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:18.984124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4055d-8211-4017-90c6-15f4e6cadfda None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:18.991239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:18.992559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:18.992881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:18.993213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] PATH_INFO: `/v3/projects/11c1a049afe54eecbf96b5d469374cda/users/c6261e6f72b24aeb9868a4fc6a08f5b9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.025326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c1a049afe54eecbf96b5d469374cda, user_id=c6261e6f72b24aeb9868a4fc6a08f5b9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:19.036907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:19.064757 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1120/2225] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:18 2022] PUT /identity/v3/projects/11c1a049afe54eecbf96b5d469374cda/users/c6261e6f72b24aeb9868a4fc6a08f5b9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:19.070032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.106677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:19.172860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.189733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:19.234040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db53bb-daea-411f-80bd-34bfbed40f30 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:19.241085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:19.241976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.242440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.242878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.243713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:19.246423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:19.263033 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1121/2226] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:19.264464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.279631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c4a23-8474-41ef-b3de-7f84bf1dd63e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:19.290406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:19.291516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.291972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.292424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.294535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:19.304303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:19.320087 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1122/2227] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:19.325661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.336074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b17de-ee50-4f1c-be40-46fb8e92e51b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:19.350365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c394dbb-9787-4304-873c-52a9ae331de3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:19.366912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:19.367748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.369638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.369998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] PATH_INFO: `/v3/projects/4938af1a380d4294aeac011134799031/users/3db3d87bec8d48e3adb1f9e210152018/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.413095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4938af1a380d4294aeac011134799031, user_id=3db3d87bec8d48e3adb1f9e210152018, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:19.424171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:19.457880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1123/2228] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:19 2022] PUT /identity/v3/projects/4938af1a380d4294aeac011134799031/users/3db3d87bec8d48e3adb1f9e210152018/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:19.457880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.461035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:19.462554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.462835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.463110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.464814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=394b1bbcff6d4a509ae2c2acadcac0b6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:19.473152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:19.476208 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1106/2229] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:18 2022] GET /identity/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6 => generated 371 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:19.480525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.505908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e81447-90c8-4a10-8c2b-94a0c055390a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:19.561147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e20b2-06e6-4f26-b6ec-9a85577bf348 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:19.566088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea92e2-d0e7-4f39-a7d9-9a188016e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:19.566769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea92e2-d0e7-4f39-a7d9-9a188016e697 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.567051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea92e2-d0e7-4f39-a7d9-9a188016e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.567326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea92e2-d0e7-4f39-a7d9-9a188016e697 admin admin] PATH_INFO: `/v3/projects/11c1a049afe54eecbf96b5d469374cda/users/c6261e6f72b24aeb9868a4fc6a08f5b9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.570991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea92e2-d0e7-4f39-a7d9-9a188016e697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c1a049afe54eecbf96b5d469374cda, user_id=c6261e6f72b24aeb9868a4fc6a08f5b9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:19.576185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea92e2-d0e7-4f39-a7d9-9a188016e697 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:19.599980 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1124/2230] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:19 2022] PUT /identity/v3/projects/11c1a049afe54eecbf96b5d469374cda/users/c6261e6f72b24aeb9868a4fc6a08f5b9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:19.601497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736629db-8cc4-4730-a58f-b6920ba0d54c None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.601768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736629db-8cc4-4730-a58f-b6920ba0d54c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.602037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736629db-8cc4-4730-a58f-b6920ba0d54c None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.612016 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1125/2231] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:19.612016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff03b3-95cc-4e97-8287-ea118273f9fa None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.612303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff03b3-95cc-4e97-8287-ea118273f9fa None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.612502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff03b3-95cc-4e97-8287-ea118273f9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.660722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fbff03b3-95cc-4e97-8287-ea118273f9fa None None] MFA Rules not processed for user `c6261e6f72b24aeb9868a4fc6a08f5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:19.664566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fbff03b3-95cc-4e97-8287-ea118273f9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:19.668715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:19.669351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.669663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.669932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.670588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:19.672302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:19.698970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1107/2232] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:19.702393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:19.761969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:19.855232 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1126/2233] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:19 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:19.856175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff03b3-95cc-4e97-8287-ea118273f9fa None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.876008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:19.918995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:19.997044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff03b3-95cc-4e97-8287-ea118273f9fa None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.004288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8332b-4d81-4903-b504-5fa45b508cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.004993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8332b-4d81-4903-b504-5fa45b508cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.005452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8332b-4d81-4903-b504-5fa45b508cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.005779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8332b-4d81-4903-b504-5fa45b508cf6 admin admin] PATH_INFO: `/v3/projects/4938af1a380d4294aeac011134799031/users/3db3d87bec8d48e3adb1f9e210152018/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.007691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8332b-4d81-4903-b504-5fa45b508cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4938af1a380d4294aeac011134799031, user_id=3db3d87bec8d48e3adb1f9e210152018, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.011733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8332b-4d81-4903-b504-5fa45b508cf6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.032296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55185498-ab89-48bb-b058-4970acd86ec2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.033522 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1127/2234] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:19 2022] PUT /identity/v3/projects/4938af1a380d4294aeac011134799031/users/3db3d87bec8d48e3adb1f9e210152018/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:20.045769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c3020-f86d-4406-b235-2a4ed78c92af None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.046114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c3020-f86d-4406-b235-2a4ed78c92af None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.046392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c3020-f86d-4406-b235-2a4ed78c92af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.091304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a23c3020-f86d-4406-b235-2a4ed78c92af None None] MFA Rules not processed for user `3db3d87bec8d48e3adb1f9e210152018`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:20.091304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a23c3020-f86d-4406-b235-2a4ed78c92af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:20.130670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.131331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.131699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.132035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.133228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=394b1bbcff6d4a509ae2c2acadcac0b6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.135444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.145579 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1108/2235] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:19 2022] GET /identity/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6 => generated 371 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:20.224876 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1128/2236] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:20 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:20.236989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.328374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:20.337423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697da45-fa9b-4058-be1b-26862cd08073 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.344901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea2294-d826-4424-85b0-fd9efcb2e980 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.345755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2294-d826-4424-85b0-fd9efcb2e980 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.346154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2294-d826-4424-85b0-fd9efcb2e980 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.346431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea2294-d826-4424-85b0-fd9efcb2e980 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.432184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea2294-d826-4424-85b0-fd9efcb2e980 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.434315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea2294-d826-4424-85b0-fd9efcb2e980 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.445038 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1109/2237] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:20.511772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c3020-f86d-4406-b235-2a4ed78c92af None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.524433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c3020-f86d-4406-b235-2a4ed78c92af None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.531980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.532777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.533061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.533328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.543744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.545343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.555475 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1129/2238] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:20.698089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6242-3cf6-4d84-9505-bf2392207cec None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.698379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6242-3cf6-4d84-9505-bf2392207cec None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.698650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6242-3cf6-4d84-9505-bf2392207cec None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.700404 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1110/2239] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:20.705174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:20.718562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.730830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd7095-54b3-4336-8f52-cc284bb2238b service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.736191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.736831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.737103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.737369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.738515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=394b1bbcff6d4a509ae2c2acadcac0b6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.740623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.743029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1130/2240] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6 => generated 371 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:20.770789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6242-3cf6-4d84-9505-bf2392207cec None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.798063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6242-3cf6-4d84-9505-bf2392207cec None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.827622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.828715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.829005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.831452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.838629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:delete_project(project_id=394b1bbcff6d4a509ae2c2acadcac0b6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.845506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.866760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.883763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5cab9-900f-455f-a4fc-41e3ae2a7ed6 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.892545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38411271-e2df-4c5d-92b5-b738a68295ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.895191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38411271-e2df-4c5d-92b5-b738a68295ad service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.895474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38411271-e2df-4c5d-92b5-b738a68295ad service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.895741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38411271-e2df-4c5d-92b5-b738a68295ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.982637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '394b1bbcff6d4a509ae2c2acadcac0b6', 'request_id': 'req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.010520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '394b1bbcff6d4a509ae2c2acadcac0b6', 'request_id': 'req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.030109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '394b1bbcff6d4a509ae2c2acadcac0b6', 'request_id': 'req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.062881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] The token cache is being invalidate because project 394b1bbcff6d4a509ae2c2acadcac0b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:21.062881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.062881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.065580 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1111/2241] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:20 2022] DELETE /identity/v3/projects/394b1bbcff6d4a509ae2c2acadcac0b6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:21.111863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38411271-e2df-4c5d-92b5-b738a68295ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.113495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38411271-e2df-4c5d-92b5-b738a68295ad service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.144982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1131/2242] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:21.252035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:21.253516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:21.264174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.264718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.264718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.266637 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1132/2243] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:21.279614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:21.282627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:21.325211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:21.473673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dde50-cac2-410a-b4c5-7f838fdbc4f8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.488370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.489102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.489364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.489667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] PATH_INFO: `/v3/users/ec6d47d564174aceb110b3354ff7db2a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.490528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:21.491541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6d47d564174aceb110b3354ff7db2a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.494157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.497104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:21.563991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:21.606311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6d47d564174aceb110b3354ff7db2a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.608058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6d47d564174aceb110b3354ff7db2a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.624104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6d47d564174aceb110b3354ff7db2a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.624732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6d47d564174aceb110b3354ff7db2a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.659816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb7b60b1-9825-4ac0-9848-35fdd954e985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6d47d564174aceb110b3354ff7db2a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:21.673601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75273ba3-33eb-4ca5-ab0a-515bc3b3ec4a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.688673 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1112/2244] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:21 2022] DELETE /identity/v3/users/ec6d47d564174aceb110b3354ff7db2a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:21.695094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.695373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.695646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.704710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.706490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.706971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.707418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/b01f4e11e8674f35a6781fa05a0a727f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.737084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=b01f4e11e8674f35a6781fa05a0a727f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.739270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.750883 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1133/2245] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:21 2022] GET /identity/v3/projects/b01f4e11e8674f35a6781fa05a0a727f => generated 375 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:21.764657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:21.766567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:21.775120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:21.784822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:21.811588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22925ac-ee19-482c-b2ab-9ae64740a6e5 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.021525 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1113/2246] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:22.050509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.052377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:22.064758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.066319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.067411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.068358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.073220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:22.073867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:22.075943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:22.116063 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1134/2247] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:22.119858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.140372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8262-80ff-449e-8383-bbaaefd6446b tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.150833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.151563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.153796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.154195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.155723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:22.160422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:22.208405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123e106-2d8f-4a32-81fb-0b8daa4760ce None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.222345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048909cf-3110-4275-84f6-072ca49aa071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.223015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048909cf-3110-4275-84f6-072ca49aa071 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.224106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048909cf-3110-4275-84f6-072ca49aa071 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.224398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048909cf-3110-4275-84f6-072ca49aa071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.235169 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1135/2248] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:22 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:22.237312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.243954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048909cf-3110-4275-84f6-072ca49aa071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:22.246895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048909cf-3110-4275-84f6-072ca49aa071 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:22.258552 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1114/2249] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:22.293661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:22.365831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518953bd-ce6e-4d36-92ff-2f037b4f5240 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.380977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.381650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.381930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.382198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.382857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:22.383160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:22.384839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:22.404980 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1136/2250] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:22.559683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f521bd0-2495-4e56-b676-3acc944b8e37 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.560306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f521bd0-2495-4e56-b676-3acc944b8e37 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.560425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f521bd0-2495-4e56-b676-3acc944b8e37 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.562107 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1115/2251] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:22.572730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:22.575424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:22.694680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.707309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840dc75-d73c-44a5-9510-080a4c788a78 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.721619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.723161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.723426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.723786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.725077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=dd77aee47f9d487c9fa48a05d2e6f6d7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:22.727665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:22.741988 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1137/2252] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:22 2022] GET /identity/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7 => generated 369 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:22.921052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f521bd0-2495-4e56-b676-3acc944b8e37 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.922931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8f521bd0-2495-4e56-b676-3acc944b8e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:22.948911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.973281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfbde9-d636-41a7-b8ca-c0c877131f04 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:23.004026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d83c9-8980-4f3d-9ef3-010a7ac1b767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:23.004555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d83c9-8980-4f3d-9ef3-010a7ac1b767 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.004835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d83c9-8980-4f3d-9ef3-010a7ac1b767 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.006385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d83c9-8980-4f3d-9ef3-010a7ac1b767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.016248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-701d83c9-8980-4f3d-9ef3-010a7ac1b767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.105701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:23.161476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f521bd0-2495-4e56-b676-3acc944b8e37 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:23.182320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:23.183019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.183364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.183630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] PATH_INFO: `/v3/users/bef3da9984ab4171955b0bdfdaa259f2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.190973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef3da9984ab4171955b0bdfdaa259f2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:23.193024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:23.208671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d83c9-8980-4f3d-9ef3-010a7ac1b767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:23.211328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d83c9-8980-4f3d-9ef3-010a7ac1b767 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:23.248574 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1138/2253] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:51:23.250224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec69f-245a-46a6-a6dd-a03877aa8e8f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.250590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec69f-245a-46a6-a6dd-a03877aa8e8f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.250768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec69f-245a-46a6-a6dd-a03877aa8e8f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.258335 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1139/2254] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:23.260809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ec69f-245a-46a6-a6dd-a03877aa8e8f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:23.265560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9b9ec69f-245a-46a6-a6dd-a03877aa8e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.339458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef3da9984ab4171955b0bdfdaa259f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.348366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef3da9984ab4171955b0bdfdaa259f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.356381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ec69f-245a-46a6-a6dd-a03877aa8e8f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:23.372506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef3da9984ab4171955b0bdfdaa259f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.373476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef3da9984ab4171955b0bdfdaa259f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.377739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:23.380609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.380887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.383350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] PATH_INFO: `/v3/projects/f273bfff7ced4f2e885d1ead94a9a486` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.388405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f273bfff7ced4f2e885d1ead94a9a486)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:23.393897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:23.399715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef3da9984ab4171955b0bdfdaa259f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.444301 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1116/2255] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:22 2022] DELETE /identity/v3/users/bef3da9984ab4171955b0bdfdaa259f2 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:23.445326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:23.452045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.524348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:23.539667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f273bfff7ced4f2e885d1ead94a9a486', 'request_id': 'req-e1462a56-d296-42fb-b998-b48a105889a0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.561054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f273bfff7ced4f2e885d1ead94a9a486', 'request_id': 'req-e1462a56-d296-42fb-b998-b48a105889a0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.579147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f273bfff7ced4f2e885d1ead94a9a486', 'request_id': 'req-e1462a56-d296-42fb-b998-b48a105889a0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.615170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] The token cache is being invalidate because project f273bfff7ced4f2e885d1ead94a9a486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:23.615620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.616332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1462a56-d296-42fb-b998-b48a105889a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.622574 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1140/2256] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:23 2022] DELETE /identity/v3/projects/f273bfff7ced4f2e885d1ead94a9a486 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:23.622574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.624167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.624488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.693885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:23.696776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:23.698061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.700841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.773007 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1141/2257] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:23.773842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:23.778653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.796936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51d25c-d74c-41d6-91e4-599ea95fb423 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:23.804069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:23.804938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.805606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.806760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] PATH_INFO: `/v3/users/807a047da85a4724ac4518dd5c870af5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.808737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=807a047da85a4724ac4518dd5c870af5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:23.811194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:23.862570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc473b28-ad07-435a-822e-2e3a71417ba2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:23.872797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:23.913771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807a047da85a4724ac4518dd5c870af5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.942284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807a047da85a4724ac4518dd5c870af5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.957607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807a047da85a4724ac4518dd5c870af5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.959177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807a047da85a4724ac4518dd5c870af5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.959937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807a047da85a4724ac4518dd5c870af5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:23.974034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:23.975494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.976031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.976635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.980675 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1142/2258] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:23 2022] DELETE /identity/v3/users/807a047da85a4724ac4518dd5c870af5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:23.980675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:23.980675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.983560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=dd77aee47f9d487c9fa48a05d2e6f6d7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:23.987564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:23.994871 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1117/2259] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:23 2022] GET /identity/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7 => generated 369 bytes in 549 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:24.041320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:24.081262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f643cdc-5e8b-4a99-a1b5-bf8a250e3d05 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.091842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.093349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.094374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:24.094577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.095184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.096487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.113076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:24.127228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b6478-b071-4862-91fa-ac4e026116ed tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.334972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.336879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.342099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.342794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.343069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.343335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.346496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:24.346861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.350009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.362439 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1143/2260] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:24.382900 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1118/2261] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:24.384541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.405379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e4e83-75a7-422e-89b8-130d25c64a85 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.429283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0d2b5-ee83-4cbb-aad1-6b83486293c9 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.430161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0d2b5-ee83-4cbb-aad1-6b83486293c9 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.430669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0d2b5-ee83-4cbb-aad1-6b83486293c9 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.431090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0d2b5-ee83-4cbb-aad1-6b83486293c9 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.432042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0d2b5-ee83-4cbb-aad1-6b83486293c9 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.434717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0d2b5-ee83-4cbb-aad1-6b83486293c9 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.482554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1144/2262] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:24 2022] POST /identity/v3/users => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:24.487140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.500352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de88dd-0c5f-4d67-8258-79931567db68 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.515260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.515958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.516263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.516468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.517433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:24.517629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.521391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.586886 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1119/2263] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:24.614474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e5d4b-743a-4d0f-9bd2-b9e80e8ebc28 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.615113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e5d4b-743a-4d0f-9bd2-b9e80e8ebc28 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.615569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e5d4b-743a-4d0f-9bd2-b9e80e8ebc28 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.617454 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1145/2264] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:24.629055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:24.631098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:24.940845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.958144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b64a7-26f7-40cd-a710-8d0a36d3b021 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.964681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.965386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.965713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.965990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.968584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=dd77aee47f9d487c9fa48a05d2e6f6d7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.971274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.974263 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1120/2265] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7 => generated 369 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:25.229205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56b257-47cd-43fb-8ba2-aff1fcb1a0cd None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.229871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56b257-47cd-43fb-8ba2-aff1fcb1a0cd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.230351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56b257-47cd-43fb-8ba2-aff1fcb1a0cd None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.232402 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1146/2266] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:25.244956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:25.267837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.299700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924916df-4848-4298-b10d-92935e0ce028 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.307260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.308359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.308883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.309374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.310959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=dd77aee47f9d487c9fa48a05d2e6f6d7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.314786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.329908 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1121/2267] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7 => generated 369 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:25.362022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56b257-47cd-43fb-8ba2-aff1fcb1a0cd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.382887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56b257-47cd-43fb-8ba2-aff1fcb1a0cd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.397672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.398473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.399226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.400325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/users/83724eaa491747faa79304257713cdcd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.439888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:delete_user(user_id=83724eaa491747faa79304257713cdcd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.443063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.460974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.463156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:25.518024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c1881-9bed-4462-8596-9f3d96522e9a tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.535905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.536568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.536848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.540024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] PATH_INFO: `/v3/projects/53354e27dd764bfda07c7355286b951f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.541760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53354e27dd764bfda07c7355286b951f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.547042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.560641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83724eaa491747faa79304257713cdcd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.578169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.595432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83724eaa491747faa79304257713cdcd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.618971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83724eaa491747faa79304257713cdcd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.619716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83724eaa491747faa79304257713cdcd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.621233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83724eaa491747faa79304257713cdcd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.648236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1147/2268] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:25 2022] DELETE /identity/v3/users/83724eaa491747faa79304257713cdcd => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:25.649232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.654855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:25.716346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53354e27dd764bfda07c7355286b951f', 'request_id': 'req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.742443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.746799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53354e27dd764bfda07c7355286b951f', 'request_id': 'req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.769326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53354e27dd764bfda07c7355286b951f', 'request_id': 'req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.799239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] The token cache is being invalidate because project 53354e27dd764bfda07c7355286b951f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:25.799747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.800739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6e36a6b4-3bd5-4c90-9cfe-b6b5eec62a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:25.803235 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1122/2269] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:25 2022] DELETE /identity/v3/projects/53354e27dd764bfda07c7355286b951f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:25.809721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1a83-c313-4698-9ef0-a8f173c8a917 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.810436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1a83-c313-4698-9ef0-a8f173c8a917 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.811095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1a83-c313-4698-9ef0-a8f173c8a917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.812141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3cbc1-37c9-424e-aadf-d44c10437a77 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.857274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ea7a1a83-c313-4698-9ef0-a8f173c8a917 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:25.859040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ea7a1a83-c313-4698-9ef0-a8f173c8a917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:25.954141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.955356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.957781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.957781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.957781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dd77aee47f9d487c9fa48a05d2e6f6d7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.960011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.974032 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1123/2270] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:25.983148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749abd2-d46e-472b-b166-42d78aac99a2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.983864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749abd2-d46e-472b-b166-42d78aac99a2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.984334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749abd2-d46e-472b-b166-42d78aac99a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.028644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d749abd2-d46e-472b-b166-42d78aac99a2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:26.030700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d749abd2-d46e-472b-b166-42d78aac99a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:26.059739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd77aee47f9d487c9fa48a05d2e6f6d7', 'request_id': 'req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:26.078207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd77aee47f9d487c9fa48a05d2e6f6d7', 'request_id': 'req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:26.089507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd77aee47f9d487c9fa48a05d2e6f6d7', 'request_id': 'req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:26.116504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] The token cache is being invalidate because project dd77aee47f9d487c9fa48a05d2e6f6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:26.116855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:26.120356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:26.126121 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1148/2271] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:25 2022] DELETE /identity/v3/projects/dd77aee47f9d487c9fa48a05d2e6f6d7 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:26.183258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.190425 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1124/2272] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:26.195246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f285c3-b26f-4c61-9409-ac5ec8ab9489 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.202640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.203377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.203717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.204053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.204805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:26.205160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.222250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.261578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749abd2-d46e-472b-b166-42d78aac99a2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.270237 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1149/2273] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:26.274714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749abd2-d46e-472b-b166-42d78aac99a2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.281195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80335029-217d-4370-8c77-5e8672cf3b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.282092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80335029-217d-4370-8c77-5e8672cf3b43 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.282534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80335029-217d-4370-8c77-5e8672cf3b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.282941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80335029-217d-4370-8c77-5e8672cf3b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.286158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80335029-217d-4370-8c77-5e8672cf3b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.288608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80335029-217d-4370-8c77-5e8672cf3b43 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.342482 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1125/2274] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:26 2022] POST /identity/v3/projects => generated 393 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:26.346960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.364092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96a5fa-2ba7-4f81-a725-06e83904d827 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.413793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.414078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.414350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.415765 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1126/2275] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:26.422123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:26.424711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:26.470371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.507079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.537398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.538173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.538403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.540605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.542567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.549768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.615844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1150/2276] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:26 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:26.619810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.645590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842da95-59a1-4f3c-9c23-3ae4ac4dc7e7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.655973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.665869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:26.671572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.674339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.674632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.674946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.683393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.713951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.743750 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1151/2277] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:26.746933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.782975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d64c3-950d-465f-b59d-50db6f6130b2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.840235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.840869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.841141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.841408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.843223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.852544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.889158 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1152/2278] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:26.896529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.942437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bbe7f-80a5-435b-b4f3-53a03b97654d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.975033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.975978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.976256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.976527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] PATH_INFO: `/v3/projects/9ddaad0529a24a0ca7afcfc96d79c945/users/3f911bb25f06441b9a23baa0bcc2f509/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.983656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf153737-1d4b-4dc0-a908-7db365796672 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.998372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.999107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.999390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.999662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/b01f4e11e8674f35a6781fa05a0a727f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.011625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=b01f4e11e8674f35a6781fa05a0a727f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.016087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.016685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ddaad0529a24a0ca7afcfc96d79c945, user_id=3f911bb25f06441b9a23baa0bcc2f509, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.019935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.022705 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1127/2279] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/projects/b01f4e11e8674f35a6781fa05a0a727f => generated 375 bytes in 601 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:27.023449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.031449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:27.067531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f9fcf-ae46-47f7-8d67-d10b51898de8 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.079428 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1153/2280] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:26 2022] PUT /identity/v3/projects/9ddaad0529a24a0ca7afcfc96d79c945/users/3f911bb25f06441b9a23baa0bcc2f509/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:27.080755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.098469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:27.236010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.236723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.237885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.238160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] PATH_INFO: `/v3/projects/75e1743a6fc742b094552a4f35de8110` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.242629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e1743a6fc742b094552a4f35de8110)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.245277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.295780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a6623-0ff7-49c9-9845-ad80d2dd8458 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.307654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.310100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.310379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.311460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] PATH_INFO: `/v3/users/eb8bfe1d590a4500b7e7b5c79a6a4e6b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.313130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb8bfe1d590a4500b7e7b5c79a6a4e6b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.320293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.333886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e1743a6fc742b094552a4f35de8110', 'request_id': 'req-b6eed96d-438d-4ce1-a0cb-484f7849448a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.351801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e1743a6fc742b094552a4f35de8110', 'request_id': 'req-b6eed96d-438d-4ce1-a0cb-484f7849448a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.358030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.373453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e1743a6fc742b094552a4f35de8110', 'request_id': 'req-b6eed96d-438d-4ce1-a0cb-484f7849448a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.396504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] The token cache is being invalidate because project 75e1743a6fc742b094552a4f35de8110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:27.396937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.398208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6eed96d-438d-4ce1-a0cb-484f7849448a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.409038 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1128/2281] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:27 2022] DELETE /identity/v3/projects/75e1743a6fc742b094552a4f35de8110 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:27.410971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648c092-322b-4030-832d-5d2130779aba None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.411302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648c092-322b-4030-832d-5d2130779aba None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.411880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648c092-322b-4030-832d-5d2130779aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.459726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb8bfe1d590a4500b7e7b5c79a6a4e6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.488251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8648c092-322b-4030-832d-5d2130779aba None None] MFA Rules not processed for user `3f911bb25f06441b9a23baa0bcc2f509`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:27.489561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8648c092-322b-4030-832d-5d2130779aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:27.497235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb8bfe1d590a4500b7e7b5c79a6a4e6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.513276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb8bfe1d590a4500b7e7b5c79a6a4e6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.513977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb8bfe1d590a4500b7e7b5c79a6a4e6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.514607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d008b84d-44ef-463b-b008-f53ed328adfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb8bfe1d590a4500b7e7b5c79a6a4e6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.544764 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1154/2282] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:27 2022] DELETE /identity/v3/users/eb8bfe1d590a4500b7e7b5c79a6a4e6b => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:27.548326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206de35-8a16-40ef-aa46-c8f769f9d703 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.548615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206de35-8a16-40ef-aa46-c8f769f9d703 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.548888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206de35-8a16-40ef-aa46-c8f769f9d703 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.552550 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1155/2283] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:27.553391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206de35-8a16-40ef-aa46-c8f769f9d703 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.555838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e206de35-8a16-40ef-aa46-c8f769f9d703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:27.612629 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1129/2284] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:27.614107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648c092-322b-4030-832d-5d2130779aba None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.615124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8648c092-322b-4030-832d-5d2130779aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:27.615689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.722120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648c092-322b-4030-832d-5d2130779aba None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.728462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.728857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.729123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.729392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] PATH_INFO: `/v3/users/c6261e6f72b24aeb9868a4fc6a08f5b9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.731568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6261e6f72b24aeb9868a4fc6a08f5b9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.731779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206de35-8a16-40ef-aa46-c8f769f9d703 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.733572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.738661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.738661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.738919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.739189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] PATH_INFO: `/v3/users/e075fe44101842fab2d6be231517cd5f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.741809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e075fe44101842fab2d6be231517cd5f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.744168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.836960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6261e6f72b24aeb9868a4fc6a08f5b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.838098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6261e6f72b24aeb9868a4fc6a08f5b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.864621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6261e6f72b24aeb9868a4fc6a08f5b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.865238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6261e6f72b24aeb9868a4fc6a08f5b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.872558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e075fe44101842fab2d6be231517cd5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.889825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd79e193-2ffa-463d-890a-af259cb2b2ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6261e6f72b24aeb9868a4fc6a08f5b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.892739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.910501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e075fe44101842fab2d6be231517cd5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.918349 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1130/2285] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:27 2022] DELETE /identity/v3/users/c6261e6f72b24aeb9868a4fc6a08f5b9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:27.919790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.920073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.920398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.937856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e075fe44101842fab2d6be231517cd5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.938570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e075fe44101842fab2d6be231517cd5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.944946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e075fe44101842fab2d6be231517cd5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:27.948296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:27.963467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:27.979923 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1156/2286] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:27 2022] DELETE /identity/v3/users/e075fe44101842fab2d6be231517cd5f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:27.980922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:27.984104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:28.073430 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1131/2287] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:28.080224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.082999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:28.104835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4beac-a31a-4f43-8ed5-edab25cdef4d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.181475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.186921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:28.224748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffa99f-c004-4f60-a53a-ce6c55ff5562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.225393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa99f-c004-4f60-a53a-ce6c55ff5562 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.225722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa99f-c004-4f60-a53a-ce6c55ff5562 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.225948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa99f-c004-4f60-a53a-ce6c55ff5562 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.226591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffa99f-c004-4f60-a53a-ce6c55ff5562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.228593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffa99f-c004-4f60-a53a-ce6c55ff5562 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.275350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.295390 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1132/2288] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:28 2022] POST /identity/v3/projects => generated 391 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:28.296758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.297225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.297363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.345180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460b493-c043-40f7-9fdb-3f047b10b17e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.347035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:28.348288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:28.478248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.479394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.479890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.480548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/b01f4e11e8674f35a6781fa05a0a727f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.486581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=b01f4e11e8674f35a6781fa05a0a727f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.491541 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1133/2289] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:28.492604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.494740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.496606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:28.507932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1157/2290] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/projects/b01f4e11e8674f35a6781fa05a0a727f => generated 375 bytes in 519 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:28.515383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.548798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.552638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a3bd5-a4a7-4758-8df4-213ca9325f43 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.577890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.578678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.579061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.579434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.580325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.586238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.635215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895efb8a-65f3-4ae2-827e-617debde5b77 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.639945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.640819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.641249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.642071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.662515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1158/2291] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:28 2022] POST /identity/v3/users => generated 302 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:28.663584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.677574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.681561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.692405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b2b6b-b9f7-42c9-b135-4a194c17dccb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.707100 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1134/2292] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:28.708053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.708684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.712367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.713639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.714884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.747656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f153-4e13-40df-8661-15e7e936646a service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.755860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.759569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.760200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.760477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.760594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.760747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.765073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.770201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.789170 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1159/2293] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:28.798511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1135/2294] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:28.803082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.817950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f3658-5539-4ebc-9bdf-cfc391460eb9 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.824786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ae912-bf35-4864-b158-6cb9d3df56b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.825788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ae912-bf35-4864-b158-6cb9d3df56b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.826227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ae912-bf35-4864-b158-6cb9d3df56b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.826930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ae912-bf35-4864-b158-6cb9d3df56b6 admin admin] PATH_INFO: `/v3/projects/51f4fbbe06c3403ca745e73589256154/users/75484a214b604859b65ff05f3be67989/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.850127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ae912-bf35-4864-b158-6cb9d3df56b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f4fbbe06c3403ca745e73589256154, user_id=75484a214b604859b65ff05f3be67989, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.854381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ae912-bf35-4864-b158-6cb9d3df56b6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.881107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1160/2295] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:28 2022] PUT /identity/v3/projects/51f4fbbe06c3403ca745e73589256154/users/75484a214b604859b65ff05f3be67989/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:28.881729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.914373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de365e73-360c-4137-a4a9-581342814bee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.100652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110164a7-ce36-461f-9ff1-2b169c3000a8 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.102783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110164a7-ce36-461f-9ff1-2b169c3000a8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.103367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110164a7-ce36-461f-9ff1-2b169c3000a8 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.105322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.106110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.106519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.107709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.108876 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1161/2296] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:51:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:29.114508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.115830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110164a7-ce36-461f-9ff1-2b169c3000a8 None None] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:51:29.118556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-110164a7-ce36-461f-9ff1-2b169c3000a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:29.125791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.160387 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1136/2297] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:29.179325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:29.199460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd6a25-5d86-426b-8ab4-b7b88ef9c3b6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.205563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.206419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.206758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.207093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] PATH_INFO: `/v3/projects/51f4fbbe06c3403ca745e73589256154/users/75484a214b604859b65ff05f3be67989/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.209378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f4fbbe06c3403ca745e73589256154, user_id=75484a214b604859b65ff05f3be67989, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.213072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.261896 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1137/2298] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:29 2022] PUT /identity/v3/projects/51f4fbbe06c3403ca745e73589256154/users/75484a214b604859b65ff05f3be67989/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:29.261896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:29.263073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:29.312147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110164a7-ce36-461f-9ff1-2b169c3000a8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:29.359059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:29.416779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c7239-42c7-44a7-9a4f-2946f83edddb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.424659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.425311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.425613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.425907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] PATH_INFO: `/v3/projects/11c1a049afe54eecbf96b5d469374cda` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.430809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c1a049afe54eecbf96b5d469374cda)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.437217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110164a7-ce36-461f-9ff1-2b169c3000a8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.440454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.543631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abcf3e-116a-4e74-8d79-144876b97570 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ddcba1504da47d58fbaead731fa1200', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb972eda012a48b196dc9fa3b82aefe8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.544951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abcf3e-116a-4e74-8d79-144876b97570 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.545797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:29.546755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abcf3e-116a-4e74-8d79-144876b97570 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.547942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abcf3e-116a-4e74-8d79-144876b97570 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] PATH_INFO: `/v3/projects/b01f4e11e8674f35a6781fa05a0a727f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.550274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abcf3e-116a-4e74-8d79-144876b97570 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorizing `identity:get_project(project_id=b01f4e11e8674f35a6781fa05a0a727f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.561024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abcf3e-116a-4e74-8d79-144876b97570 tempest-QuotasAdminTestJSON-754300214 tempest-QuotasAdminTestJSON-754300214-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.567184 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1162/2299] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:51:29 2022] GET /identity/v3/projects/b01f4e11e8674f35a6781fa05a0a727f => generated 375 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:29.570529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2e1b-67e2-485e-bdd3-dda8b006c8e8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.570818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2e1b-67e2-485e-bdd3-dda8b006c8e8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.571089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2e1b-67e2-485e-bdd3-dda8b006c8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.575705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c1a049afe54eecbf96b5d469374cda', 'request_id': 'req-9573ceeb-fced-4767-aa44-106b4113c538'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:29.592936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c1a049afe54eecbf96b5d469374cda', 'request_id': 'req-9573ceeb-fced-4767-aa44-106b4113c538'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:29.622700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cffe2e1b-67e2-485e-bdd3-dda8b006c8e8 None None] MFA Rules not processed for user `75484a214b604859b65ff05f3be67989`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:29.624187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cffe2e1b-67e2-485e-bdd3-dda8b006c8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:29.632076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c1a049afe54eecbf96b5d469374cda', 'request_id': 'req-9573ceeb-fced-4767-aa44-106b4113c538'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:29.650717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] The token cache is being invalidate because project 11c1a049afe54eecbf96b5d469374cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:29.651080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:29.654109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9573ceeb-fced-4767-aa44-106b4113c538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:29.659096 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1138/2300] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:29 2022] DELETE /identity/v3/projects/11c1a049afe54eecbf96b5d469374cda => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:29.664937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c6fff-2d13-481d-aec3-2c76e8ab7d64 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.664937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c6fff-2d13-481d-aec3-2c76e8ab7d64 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.664937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c6fff-2d13-481d-aec3-2c76e8ab7d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.751185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-956c6fff-2d13-481d-aec3-2c76e8ab7d64 None None] MFA Rules not processed for user `5aae6d573509428db8d4d5db9d7b005c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:29.754862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-956c6fff-2d13-481d-aec3-2c76e8ab7d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:29.840667 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1163/2301] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:29 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:29.844747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.845244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.846367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.916569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] MFA Rules not processed for user `f1134231b4a545fdb136a5204cb4f289`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:29.918462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.009319 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1139/2302] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:30.009319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.009319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.009319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.053847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:30.062172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:30.063528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.122300 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1164/2303] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:29 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:30.123242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.127230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.225210 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1140/2304] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:30.226478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.228688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.302332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36960f3e-75c9-4b7f-bec9-a5c3edc39275 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:30.307786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:30.308515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.308866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.309146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] PATH_INFO: `/v3/projects/ea88200545ae4003b42b0d475f482c9d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.310536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea88200545ae4003b42b0d475f482c9d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:30.313756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:30.340158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:30.377123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7473fe-98d3-427e-b12f-99fa11a4eaa4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:30.410507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:30.411148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.411424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.412152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.418812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea88200545ae4003b42b0d475f482c9d', 'request_id': 'req-8242d118-547f-4427-91aa-c3877b5f0f06'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.423693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:30.426082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:30.432760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea88200545ae4003b42b0d475f482c9d', 'request_id': 'req-8242d118-547f-4427-91aa-c3877b5f0f06'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.455691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1141/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:30 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:30.455691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.455691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.460474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea88200545ae4003b42b0d475f482c9d', 'request_id': 'req-8242d118-547f-4427-91aa-c3877b5f0f06'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.484469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] The token cache is being invalidate because project ea88200545ae4003b42b0d475f482c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:30.484469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.486642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5fe73-1ee4-4b2f-ae1b-297c00dab990 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:30.488467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.491098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1165/2306] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:30 2022] DELETE /identity/v3/projects/ea88200545ae4003b42b0d475f482c9d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:30.492711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.495213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.610786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:30.611450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.611728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.611997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] PATH_INFO: `/v3/projects/5c06b859ea6d468bb45b4c6b9633d368` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.613525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c06b859ea6d468bb45b4c6b9633d368)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:30.615370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242d118-547f-4427-91aa-c3877b5f0f06 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:30.615826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:30.621279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cad36-a7d7-4cc1-93fa-b8fb9d7edc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:30.622019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cad36-a7d7-4cc1-93fa-b8fb9d7edc83 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.622653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cad36-a7d7-4cc1-93fa-b8fb9d7edc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.622992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cad36-a7d7-4cc1-93fa-b8fb9d7edc83 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.623826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cad36-a7d7-4cc1-93fa-b8fb9d7edc83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:30.626932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cad36-a7d7-4cc1-93fa-b8fb9d7edc83 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:30.674421 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1166/2307] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:30 2022] POST /identity/v3/projects => generated 395 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:30.674421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.674421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.674421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.687697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:30.694945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c06b859ea6d468bb45b4c6b9633d368', 'request_id': 'req-b5654e35-2d3f-40c7-8efe-3d668d797804'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.709332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c06b859ea6d468bb45b4c6b9633d368', 'request_id': 'req-b5654e35-2d3f-40c7-8efe-3d668d797804'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.715319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:30.717516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.729132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c06b859ea6d468bb45b4c6b9633d368', 'request_id': 'req-b5654e35-2d3f-40c7-8efe-3d668d797804'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.783590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] The token cache is being invalidate because project 5c06b859ea6d468bb45b4c6b9633d368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:30.783948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.784708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:30.791698 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1142/2308] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:30 2022] DELETE /identity/v3/projects/5c06b859ea6d468bb45b4c6b9633d368 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:30.792903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.796665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.872906 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1167/2309] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:30.874105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.876761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.943513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654e35-2d3f-40c7-8efe-3d668d797804 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:30.951112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:30.952553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.952905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.953266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] PATH_INFO: `/v3/users/672871e1c5d84e8b8b5eb8ccb66b9730` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.956107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=672871e1c5d84e8b8b5eb8ccb66b9730)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:30.958980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:30.973802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:30.982922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520e08-edf1-4b66-a970-593554f255d1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:30.992440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:30.993081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.993352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.993669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.995686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.044059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672871e1c5d84e8b8b5eb8ccb66b9730'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.044998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672871e1c5d84e8b8b5eb8ccb66b9730'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.056381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672871e1c5d84e8b8b5eb8ccb66b9730'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.061251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672871e1c5d84e8b8b5eb8ccb66b9730'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.078584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.080239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.085428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672871e1c5d84e8b8b5eb8ccb66b9730'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.093254 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1168/2310] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:31.094707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.096327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.112716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc3d05-d8ea-4b02-a2b5-93fbcf6e74b2 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.117964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.118987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.119267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.119540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] PATH_INFO: `/v3/users/2ce5fae031e34a228fd0375b92ba8d2b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.120789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce5fae031e34a228fd0375b92ba8d2b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.123351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.136330 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1143/2311] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:30 2022] DELETE /identity/v3/users/672871e1c5d84e8b8b5eb8ccb66b9730 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:31.137286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.139275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.197561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.217813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce5fae031e34a228fd0375b92ba8d2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.244800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce5fae031e34a228fd0375b92ba8d2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.265751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce5fae031e34a228fd0375b92ba8d2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.266517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb3f08-1b14-4674-9be3-7f9cc81484a6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.267031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce5fae031e34a228fd0375b92ba8d2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.267623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce5fae031e34a228fd0375b92ba8d2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.280148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.280807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.281100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.281443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.289209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.291212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1169/2312] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:31 2022] DELETE /identity/v3/users/2ce5fae031e34a228fd0375b92ba8d2b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:31.295981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.301764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.440333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3f84-4041-4622-b761-622c8779f69a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.470115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.470761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.471046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.471312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.471963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.473968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.495849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.497941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.520093 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1170/2313] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:31 2022] POST /identity/v3/users => generated 304 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:31.521184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.522893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.532197 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1144/2314] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:31.534316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.541249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.549491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5817a-392c-4055-b17c-f99e4a42e0f4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.566178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.566833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.567099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.567373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] PATH_INFO: `/v3/users/3db3d87bec8d48e3adb1f9e210152018` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.568913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db3d87bec8d48e3adb1f9e210152018)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.571136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.596279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.691072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef49d-db07-4f17-90fb-a4babce5d6da service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.696255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623869ca-0b55-4d85-ab9d-ba1c856df502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.696474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db3d87bec8d48e3adb1f9e210152018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.697113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623869ca-0b55-4d85-ab9d-ba1c856df502 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.697501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623869ca-0b55-4d85-ab9d-ba1c856df502 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.697948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623869ca-0b55-4d85-ab9d-ba1c856df502 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.699004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623869ca-0b55-4d85-ab9d-ba1c856df502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:31.699422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623869ca-0b55-4d85-ab9d-ba1c856df502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.701273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623869ca-0b55-4d85-ab9d-ba1c856df502 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.721901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db3d87bec8d48e3adb1f9e210152018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.731776 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1145/2315] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:31.735829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.736553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.737605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.746670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db3d87bec8d48e3adb1f9e210152018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.747340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db3d87bec8d48e3adb1f9e210152018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.747955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db3d87bec8d48e3adb1f9e210152018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:31.769190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:31.771655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.776590 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1171/2316] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:31 2022] DELETE /identity/v3/users/3db3d87bec8d48e3adb1f9e210152018 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:31.780752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.786840 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1146/2317] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:31.786840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.786840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.787196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.803793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364f3c4-ef78-4d5b-a0d1-4263bdcfe6de None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.826692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.928657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a29e57-beff-4016-9ab7-b5b611c4fb12 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.936207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.937213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.937351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.937579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] PATH_INFO: `/v3/users/3f911bb25f06441b9a23baa0bcc2f509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.941502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f911bb25f06441b9a23baa0bcc2f509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.943470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.946149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.946916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.947128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.947397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.948440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.949956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.973172 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1172/2318] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:31.973172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.982310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.008533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdce85-9997-4cfc-9ff4-062d315ed8ae admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.021313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.022043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.022246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.022518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.023494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.025084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.028782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f911bb25f06441b9a23baa0bcc2f509'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:32.030945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f911bb25f06441b9a23baa0bcc2f509'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:32.057148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f911bb25f06441b9a23baa0bcc2f509'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:32.057949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f911bb25f06441b9a23baa0bcc2f509'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:32.086093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f911bb25f06441b9a23baa0bcc2f509'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:32.086424 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1173/2319] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:31 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:32.087285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.094894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.124647 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1147/2320] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:31 2022] DELETE /identity/v3/users/3f911bb25f06441b9a23baa0bcc2f509 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:32.128286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.140738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.195056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7789dd6-bca9-4ffa-9811-861f77762427 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.262691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.348100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985872c2-c2f1-44ed-868d-9b5bfb2bfe31 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.362051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.362738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007243e-c130-4694-b899-9850b4049b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.362872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.363629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.363741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.364679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007243e-c130-4694-b899-9850b4049b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.366466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007243e-c130-4694-b899-9850b4049b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.367407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.367852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007243e-c130-4694-b899-9850b4049b87 admin admin] PATH_INFO: `/v3/projects/9264cb99cc4246b2a5fac162a65cc9c8/users/2bdc03ea28d940518fb8d67567c9f2bc/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.411234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.420670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007243e-c130-4694-b899-9850b4049b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9264cb99cc4246b2a5fac162a65cc9c8, user_id=2bdc03ea28d940518fb8d67567c9f2bc, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.421107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.427613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007243e-c130-4694-b899-9850b4049b87 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.450238 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1174/2321] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:32.451919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.455620 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1148/2322] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:32 2022] PUT /identity/v3/projects/9264cb99cc4246b2a5fac162a65cc9c8/users/2bdc03ea28d940518fb8d67567c9f2bc/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:32.457440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.466444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92fa38-11e3-4f7a-9992-45ec9992271f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.467461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92fa38-11e3-4f7a-9992-45ec9992271f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.469114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92fa38-11e3-4f7a-9992-45ec9992271f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.492968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd222e-22ea-480e-825c-0e1c0800d7fa service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.532556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ab92fa38-11e3-4f7a-9992-45ec9992271f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:32.536409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab92fa38-11e3-4f7a-9992-45ec9992271f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.634116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.675544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.676276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.676551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.676984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.677667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.679690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.716546 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1149/2323] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:32.718194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92fa38-11e3-4f7a-9992-45ec9992271f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.730264 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1175/2324] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:32 2022] POST /identity/v3/users => generated 303 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:32.732422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92fa38-11e3-4f7a-9992-45ec9992271f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.734212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.747734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.748531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.748897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.749290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.750087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.752910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.753196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36588a-dcfa-462e-8a56-1d140b94dbf8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.774556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.776580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.779126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.780170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.780996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.785181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.792679 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1150/2325] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:32.792679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.844663 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1176/2326] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:32.849640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.884490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3192b-99f8-4256-9d70-55899c907e4b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.901983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.902675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.903290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.905168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] PATH_INFO: `/v3/projects/9264cb99cc4246b2a5fac162a65cc9c8/users/2bdc03ea28d940518fb8d67567c9f2bc/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.909867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9264cb99cc4246b2a5fac162a65cc9c8, user_id=2bdc03ea28d940518fb8d67567c9f2bc, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.913237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.926756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b47c2-8c2a-4d7c-9e7f-84c505d1114d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.968892 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1177/2327] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:32 2022] PUT /identity/v3/projects/9264cb99cc4246b2a5fac162a65cc9c8/users/2bdc03ea28d940518fb8d67567c9f2bc/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:32.969970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.977691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.057626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.058524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.058925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.059398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.079147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.124748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13adce-d6fc-4edb-b791-fbe9a08171af admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.130971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.131697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.132019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.132339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] PATH_INFO: `/v3/projects/ee8337c545374fb79e72672eb365bb58` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.133789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8337c545374fb79e72672eb365bb58)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.136179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.183815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.189204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.225340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1151/2328] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:33.226415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.247487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8337c545374fb79e72672eb365bb58', 'request_id': 'req-1ee65097-6d4c-46b3-9380-61a4d8acd000'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.253423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621efac-bf3b-407c-ab13-6b5d1a03cd09 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.266533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8337c545374fb79e72672eb365bb58', 'request_id': 'req-1ee65097-6d4c-46b3-9380-61a4d8acd000'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.281113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8337c545374fb79e72672eb365bb58', 'request_id': 'req-1ee65097-6d4c-46b3-9380-61a4d8acd000'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.306067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] The token cache is being invalidate because project ee8337c545374fb79e72672eb365bb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:33.306355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.309845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.311770 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1178/2329] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:32 2022] DELETE /identity/v3/projects/ee8337c545374fb79e72672eb365bb58 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:33.316287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.321927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.381312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53b42f-5982-4001-a044-bff5c7fd6dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.382110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b42f-5982-4001-a044-bff5c7fd6dce admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.382442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b42f-5982-4001-a044-bff5c7fd6dce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.383048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b42f-5982-4001-a044-bff5c7fd6dce admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.383768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53b42f-5982-4001-a044-bff5c7fd6dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.386024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53b42f-5982-4001-a044-bff5c7fd6dce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.413510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1152/2330] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:33.414003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.414474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.415084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.427465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.450353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:33.452255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.463270 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1153/2331] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:33.464727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.469002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.476106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65097-6d4c-46b3-9380-61a4d8acd000 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.492996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ef566-b43a-4ef9-a2f7-de9ccba99c00 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.495811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.497018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.497311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.497620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] PATH_INFO: `/v3/projects/4938af1a380d4294aeac011134799031` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.501226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4938af1a380d4294aeac011134799031)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.501616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e81f08-0e91-49fa-b877-183f30a268f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.502311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81f08-0e91-49fa-b877-183f30a268f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.502584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81f08-0e91-49fa-b877-183f30a268f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.502850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81f08-0e91-49fa-b877-183f30a268f6 admin admin] PATH_INFO: `/v3/projects/8456c63a1d184cfea5d482549bccdabb/users/6d7f9995a47046bda06e450f2f57aea8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.508589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.545860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e81f08-0e91-49fa-b877-183f30a268f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8456c63a1d184cfea5d482549bccdabb, user_id=6d7f9995a47046bda06e450f2f57aea8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.552061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e81f08-0e91-49fa-b877-183f30a268f6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.588968 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1154/2332] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:33 2022] PUT /identity/v3/projects/8456c63a1d184cfea5d482549bccdabb/users/6d7f9995a47046bda06e450f2f57aea8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:33.593485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a85e-a3af-4cff-91b6-ce7049486af0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.593987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a85e-a3af-4cff-91b6-ce7049486af0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.594241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a85e-a3af-4cff-91b6-ce7049486af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.634083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4ab7a85e-a3af-4cff-91b6-ce7049486af0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:33.637425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ab7a85e-a3af-4cff-91b6-ce7049486af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.680180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4938af1a380d4294aeac011134799031', 'request_id': 'req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.741904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4938af1a380d4294aeac011134799031', 'request_id': 'req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.762690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4938af1a380d4294aeac011134799031', 'request_id': 'req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.801623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] The token cache is being invalidate because project 4938af1a380d4294aeac011134799031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:33.801965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.806649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dfc668dc-4ac9-4475-8f79-a0165fba4c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:33.816039 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1179/2333] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:33 2022] DELETE /identity/v3/projects/4938af1a380d4294aeac011134799031 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:33.818007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b412da-2094-43e6-ab22-6f156f8f32ad None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.818274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b412da-2094-43e6-ab22-6f156f8f32ad None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.818569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b412da-2094-43e6-ab22-6f156f8f32ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.838319 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1155/2334] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:33.842054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db66f0-5664-42b9-ab3c-a07462304cbc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.842264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db66f0-5664-42b9-ab3c-a07462304cbc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.842534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db66f0-5664-42b9-ab3c-a07462304cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.869318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d1b412da-2094-43e6-ab22-6f156f8f32ad None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:33.873333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1b412da-2094-43e6-ab22-6f156f8f32ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.897614 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1180/2335] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:33.898604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b412da-2094-43e6-ab22-6f156f8f32ad None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.911202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-77db66f0-5664-42b9-ab3c-a07462304cbc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:33.915111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-77db66f0-5664-42b9-ab3c-a07462304cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.919481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b412da-2094-43e6-ab22-6f156f8f32ad None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.934781 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1156/2336] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:33.942770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec9f23-d202-4a05-8162-13801a44c85e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.943056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec9f23-d202-4a05-8162-13801a44c85e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.943325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec9f23-d202-4a05-8162-13801a44c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.945571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.946476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.946895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.951621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.951621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.951621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.952129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.966117 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1181/2337] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.967313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.969544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:33.995551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-beec9f23-d202-4a05-8162-13801a44c85e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:33.999545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-beec9f23-d202-4a05-8162-13801a44c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:34.026370 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1157/2338] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:34.027619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec9f23-d202-4a05-8162-13801a44c85e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.038955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.046728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec9f23-d202-4a05-8162-13801a44c85e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.061000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.062122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.062122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.062329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.063069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.067859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.072146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f9d2f-72af-4320-81fd-d4a1c3b594ed admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.080407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.083085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.084267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.086494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.103900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.105908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.130455 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1182/2339] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:34.132271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.134184 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1158/2340] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:34 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:34.138567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.149569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867d81-d598-4699-bdf2-301dcb2c7c61 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.255627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.298354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.299671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.316775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b722072-dcc1-4655-b210-fe69f9b9e12c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.319868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.320473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.321642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:34.322189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.332101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.339822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.340652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.344471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.344918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.347133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.357494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.384000 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1183/2341] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:34.387471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.415449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bec835-45a8-43bf-9ae0-5e1037dee8f1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.424276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.424984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.425206 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1159/2342] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:34 2022] POST /identity/v3/users => generated 302 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:34.425751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.426082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.426790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.429615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.430250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.451255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746bb13-087d-4707-b815-9a44e44d3498 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.463486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.464120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.466192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.466780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.468165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.471275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.492341 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1184/2343] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:34 2022] POST /identity/v3/projects => generated 401 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:34.493345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.498685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:34.501769 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1160/2344] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:34.502908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.527062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75397cc8-a3df-442a-aba8-c18a47196a80 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.655856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ab9db-6c0c-45a0-b4d3-30e1c707c875 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.665824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.667018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.674768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.675234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] PATH_INFO: `/v3/projects/b01f4e11e8674f35a6781fa05a0a727f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.676663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.676806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01f4e11e8674f35a6781fa05a0a727f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.677295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.677641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.677928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.678562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.679606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.680764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.733438 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1161/2345] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:34 2022] POST /identity/v3/users => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:34.739216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.753135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e56348-fe46-43b4-9537-4669a8bf5567 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.759231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.760152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.760594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.761685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.762544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.764397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.779182 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1162/2346] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:34.780663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.794862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e8636-b6b3-4551-926b-0e6ce777586e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.828935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01f4e11e8674f35a6781fa05a0a727f', 'request_id': 'req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:34.841097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01f4e11e8674f35a6781fa05a0a727f', 'request_id': 'req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:34.849690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01f4e11e8674f35a6781fa05a0a727f', 'request_id': 'req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:34.852078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.867124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] The token cache is being invalidate because project b01f4e11e8674f35a6781fa05a0a727f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:34.867512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:34.868739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:34.871125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1185/2347] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:34 2022] DELETE /identity/v3/projects/b01f4e11e8674f35a6781fa05a0a727f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:34.871856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.874513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:34.895200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.896296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.896677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.897006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.898213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.900090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.927654 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1163/2348] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:34.928368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.932226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:34.975645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e985e6-13a2-4b66-b424-37ac4fcdeda8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.991679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.992379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.994456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.995584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] PATH_INFO: `/v3/users/6ddcba1504da47d58fbaead731fa1200` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.997104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddcba1504da47d58fbaead731fa1200)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.016066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.043996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.054521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaee7c8-87dd-41bd-a7a2-2d30fdec457b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.068044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.070644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.071082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.071548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] PATH_INFO: `/v3/projects/fb724c88a4d841489ed1a5c9364025a0/users/3d90b14b5f5948b0a915dfd56b41af29/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.156352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb724c88a4d841489ed1a5c9364025a0, user_id=3d90b14b5f5948b0a915dfd56b41af29, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.162054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddcba1504da47d58fbaead731fa1200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.162286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.164172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddcba1504da47d58fbaead731fa1200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.184453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddcba1504da47d58fbaead731fa1200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.185314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddcba1504da47d58fbaead731fa1200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.189252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1186/2349] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:34 2022] PUT /identity/v3/projects/fb724c88a4d841489ed1a5c9364025a0/users/3d90b14b5f5948b0a915dfd56b41af29/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:35.191896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.195166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:35.245158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddcba1504da47d58fbaead731fa1200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.301257 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1164/2350] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:34 2022] DELETE /identity/v3/users/6ddcba1504da47d58fbaead731fa1200 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:35.301257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.301600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:35.318528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674622af-35ad-4a37-a51c-395be9210092 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.436080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618360c-4dd4-44f2-8141-abaf507a0b6b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.444841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22003a-78d8-4364-ae9f-550670b22576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.445653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22003a-78d8-4364-ae9f-550670b22576 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.453619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22003a-78d8-4364-ae9f-550670b22576 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.453619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22003a-78d8-4364-ae9f-550670b22576 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.453619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22003a-78d8-4364-ae9f-550670b22576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.453619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22003a-78d8-4364-ae9f-550670b22576 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.462509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.463930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.464217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.465262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] PATH_INFO: `/v3/projects/9ddaad0529a24a0ca7afcfc96d79c945` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.468477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ddaad0529a24a0ca7afcfc96d79c945)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.470667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.476134 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1187/2351] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:35.477512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.477808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.479382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.561384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] MFA Rules not processed for user `3d90b14b5f5948b0a915dfd56b41af29`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:35.564281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:35.594525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.597258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ddaad0529a24a0ca7afcfc96d79c945', 'request_id': 'req-406ac48d-3904-4f39-86d1-a9f5a600c981'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.614038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ddaad0529a24a0ca7afcfc96d79c945', 'request_id': 'req-406ac48d-3904-4f39-86d1-a9f5a600c981'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.637687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ddaad0529a24a0ca7afcfc96d79c945', 'request_id': 'req-406ac48d-3904-4f39-86d1-a9f5a600c981'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.651538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] The token cache is being invalidate because project 9ddaad0529a24a0ca7afcfc96d79c945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:35.651883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.657490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.668566 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1165/2352] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:35 2022] DELETE /identity/v3/projects/9ddaad0529a24a0ca7afcfc96d79c945 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:35.670206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.678469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:35.730944 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1188/2353] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:35 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:35.732024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.733446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:35.753491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.837846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067d2d7-661c-4b0e-ba80-89494b2f490c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.854146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.856021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.856317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.858246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] PATH_INFO: `/v3/users/75484a214b604859b65ff05f3be67989` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.860806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=75484a214b604859b65ff05f3be67989)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.866423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.880621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac48d-3904-4f39-86d1-a9f5a600c981 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.904178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.905027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.905309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.905737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] PATH_INFO: `/v3/projects/1f305d163acf499c9808c5f11150a7cf/users/42f7c0fdad9346c58949928300021b07/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.942952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f305d163acf499c9808c5f11150a7cf, user_id=42f7c0fdad9346c58949928300021b07, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.951496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.959354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75484a214b604859b65ff05f3be67989'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.974024 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1166/2354] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:35 2022] PUT /identity/v3/projects/1f305d163acf499c9808c5f11150a7cf/users/42f7c0fdad9346c58949928300021b07/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:35.975018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.983926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:35.985007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75484a214b604859b65ff05f3be67989'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.996577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75484a214b604859b65ff05f3be67989'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.998285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75484a214b604859b65ff05f3be67989'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:35.999917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f94ec2ef-4ccf-416e-84d7-292f165952fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75484a214b604859b65ff05f3be67989'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:36.035790 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1189/2355] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:35 2022] DELETE /identity/v3/users/75484a214b604859b65ff05f3be67989 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:36.045479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.046242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.046621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.079927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:36.082211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:36.126504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345305e-1e4e-44d0-8a03-74972c596be6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.224887 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1190/2356] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:36.229623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.230433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.230767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.231466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.232179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.234447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.249080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.273178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.275068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:36.309601 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1167/2357] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:35 2022] POST /identity/v3/projects => generated 403 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:36.312240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.315125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:36.374105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.446709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e9188-df0f-4fcd-8db6-2cbc9600882f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.459267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a59b58-bac6-4dd0-933e-6c2c43aff46f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.478746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.479388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.479786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.480265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.481510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.483994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.521381 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1168/2358] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:36 2022] POST /identity/v3/users => generated 308 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:36.522467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.543920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcb171-93d4-44f2-8d55-b61f4b29a810 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.551469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.553020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.553329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.553662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.554521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.557257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.571794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.574960 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1169/2359] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:36.575175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.576109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.576318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.576417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.590342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a4be-9b3a-418a-a55f-9d5832451b5f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.609036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.609278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.611376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.612129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.612229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.612229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] PATH_INFO: `/v3/projects/0cebe9dbf7d049799b007300b7c1ce41/users/0cda1f7458294fcd8087ff7c65edbd6a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.630241 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1191/2360] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:36.644849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cebe9dbf7d049799b007300b7c1ce41, user_id=0cda1f7458294fcd8087ff7c65edbd6a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.648043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.681755 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1170/2361] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:36 2022] PUT /identity/v3/projects/0cebe9dbf7d049799b007300b7c1ce41/users/0cda1f7458294fcd8087ff7c65edbd6a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:36.682919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.702190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488c992-e580-4db4-bf57-4634030a6d62 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.794112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76874012-8da8-4148-b47e-7ef49b74751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.795192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76874012-8da8-4148-b47e-7ef49b74751e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.795707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76874012-8da8-4148-b47e-7ef49b74751e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.796166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76874012-8da8-4148-b47e-7ef49b74751e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.796987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76874012-8da8-4148-b47e-7ef49b74751e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.800031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76874012-8da8-4148-b47e-7ef49b74751e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.824847 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1192/2362] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:36.826187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.839286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec09f1-3ce0-4116-8d63-22cd7e5ff082 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.846372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.847118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.847534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.847807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] PATH_INFO: `/v3/projects/0cebe9dbf7d049799b007300b7c1ce41/users/0cda1f7458294fcd8087ff7c65edbd6a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.849923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cebe9dbf7d049799b007300b7c1ce41, user_id=0cda1f7458294fcd8087ff7c65edbd6a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.852674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.874972 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1171/2363] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:36 2022] PUT /identity/v3/projects/0cebe9dbf7d049799b007300b7c1ce41/users/0cda1f7458294fcd8087ff7c65edbd6a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:36.876719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0284-6295-4831-970f-a421613a5b36 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.877173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0284-6295-4831-970f-a421613a5b36 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.877250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da0284-6295-4831-970f-a421613a5b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.920392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.923825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f2da0284-6295-4831-970f-a421613a5b36 None None] MFA Rules not processed for user `0cda1f7458294fcd8087ff7c65edbd6a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:36.924027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:36.925138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f2da0284-6295-4831-970f-a421613a5b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:37.075884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1193/2364] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:36 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:37.078259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea19a8-414c-491f-9bc5-92c9c18b5d7e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.093232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.093972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.094249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.094517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] PATH_INFO: `/v3/projects/eb972eda012a48b196dc9fa3b82aefe8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.095835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb972eda012a48b196dc9fa3b82aefe8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.097904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.150308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:37.164718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb972eda012a48b196dc9fa3b82aefe8', 'request_id': 'req-82d7de50-1056-4949-b302-e159a781f7fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:37.174279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb972eda012a48b196dc9fa3b82aefe8', 'request_id': 'req-82d7de50-1056-4949-b302-e159a781f7fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:37.187401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb972eda012a48b196dc9fa3b82aefe8', 'request_id': 'req-82d7de50-1056-4949-b302-e159a781f7fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:37.201998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] The token cache is being invalidate because project eb972eda012a48b196dc9fa3b82aefe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:37.202387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:37.203286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82d7de50-1056-4949-b302-e159a781f7fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:37.211680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b1be1-3df0-4bbe-a010-889ffbce9c74 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.211971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b1be1-3df0-4bbe-a010-889ffbce9c74 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.212251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b1be1-3df0-4bbe-a010-889ffbce9c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.215524 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1172/2365] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:36 2022] DELETE /identity/v3/projects/eb972eda012a48b196dc9fa3b82aefe8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:37.247287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1b7b1be1-3df0-4bbe-a010-889ffbce9c74 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:37.248570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1b7b1be1-3df0-4bbe-a010-889ffbce9c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:37.259803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:37.337532 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1194/2366] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:37.347423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa8a8-8d1e-4bc6-980c-9d6741d575a7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.347909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa8a8-8d1e-4bc6-980c-9d6741d575a7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.348597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa8a8-8d1e-4bc6-980c-9d6741d575a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.375629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-885aa8a8-8d1e-4bc6-980c-9d6741d575a7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:37.377148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-885aa8a8-8d1e-4bc6-980c-9d6741d575a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:37.389817 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1173/2367] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:37.397336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b1be1-3df0-4bbe-a010-889ffbce9c74 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.415783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b1be1-3df0-4bbe-a010-889ffbce9c74 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.425259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.425966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.426248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.426568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.427291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:37.427647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.430417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.452452 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1195/2368] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:37.457382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aa8a8-8d1e-4bc6-980c-9d6741d575a7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.470750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aa8a8-8d1e-4bc6-980c-9d6741d575a7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.490048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.490739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.491271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.491603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.492465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.495394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.516510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.518536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:37.591449 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1174/2369] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:37 2022] POST /identity/v3/projects => generated 409 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:37.597662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.774286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c94da-8b5c-4414-b9c6-433e7bb5ed3d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.779243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152d1f1-f3fb-4b58-ba44-4923bc86193f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.779903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.780526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.780800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.781067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.781819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.787057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.821935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1175/2370] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:37 2022] POST /identity/v3/users => generated 305 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:37.828032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.832481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:37.859704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570badd0-f21e-4ab6-8cb8-69e0f8732de7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.874287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.875112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.875480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.876876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] PATH_INFO: `/v3/projects/51f4fbbe06c3403ca745e73589256154` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.878667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f4fbbe06c3403ca745e73589256154)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.884037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.953825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff2c5f-01b3-4881-a18f-5eac39b27615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.954453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2c5f-01b3-4881-a18f-5eac39b27615 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.954735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2c5f-01b3-4881-a18f-5eac39b27615 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.955585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2c5f-01b3-4881-a18f-5eac39b27615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.959552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d4ff2c5f-01b3-4881-a18f-5eac39b27615 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:37.971939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f4fbbe06c3403ca745e73589256154', 'request_id': 'req-7d266272-4113-4e22-b9fe-2c34e085b97e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:37.987366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f4fbbe06c3403ca745e73589256154', 'request_id': 'req-7d266272-4113-4e22-b9fe-2c34e085b97e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:37.999076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f4fbbe06c3403ca745e73589256154', 'request_id': 'req-7d266272-4113-4e22-b9fe-2c34e085b97e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:38.023065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] The token cache is being invalidate because project 51f4fbbe06c3403ca745e73589256154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:38.023436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:38.024492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:38.026391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1176/2371] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:37 2022] DELETE /identity/v3/projects/51f4fbbe06c3403ca745e73589256154 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:38.032346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.033782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.073779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=12,keystone:SELECT=82,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.092536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.179080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff2c5f-01b3-4881-a18f-5eac39b27615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.181551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff2c5f-01b3-4881-a18f-5eac39b27615 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.196048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266272-4113-4e22-b9fe-2c34e085b97e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.201910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.202772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.203576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.204464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.205340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.207886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.208597 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1196/2372] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:37 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:51:38.213536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d668ee-cd1d-4f5c-8b00-0277527f4024 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.214180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d668ee-cd1d-4f5c-8b00-0277527f4024 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.218549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d668ee-cd1d-4f5c-8b00-0277527f4024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.225200 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1177/2373] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:38.229869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.272195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-48d668ee-cd1d-4f5c-8b00-0277527f4024 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:38.274293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-48d668ee-cd1d-4f5c-8b00-0277527f4024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.275605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b64f7b-9a73-45cb-9310-12f4803dcb58 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.293676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1197/2374] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:38.294215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.295105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.295251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823636-69d6-475f-a198-dddf79ff4f65 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.295403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.295514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823636-69d6-475f-a198-dddf79ff4f65 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.295675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823636-69d6-475f-a198-dddf79ff4f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.299104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.300891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.302184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.334192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7a823636-69d6-475f-a198-dddf79ff4f65 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:38.335199 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1178/2375] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:38.335482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7a823636-69d6-475f-a198-dddf79ff4f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.337301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.362255 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1198/2376] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:38.363272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a823636-69d6-475f-a198-dddf79ff4f65 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.363742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d8258-76ac-486e-a36c-6ba724d86392 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.377383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab505963-0958-41ab-b49d-83fb0ba10f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.378302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505963-0958-41ab-b49d-83fb0ba10f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.378604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505963-0958-41ab-b49d-83fb0ba10f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.379069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab505963-0958-41ab-b49d-83fb0ba10f15 admin admin] PATH_INFO: `/v3/projects/71ae79b73bb548818c9fd2cf2e612918/users/f6d361c855374fba8b62baf73f160e27/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.381986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a823636-69d6-475f-a198-dddf79ff4f65 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.386689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.387304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.387586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.387852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.388507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:38.388811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.394275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.423128 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1199/2377] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:38.427142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab505963-0958-41ab-b49d-83fb0ba10f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ae79b73bb548818c9fd2cf2e612918, user_id=f6d361c855374fba8b62baf73f160e27, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.428141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.428453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab505963-0958-41ab-b49d-83fb0ba10f15 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.447070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8df0-fe7b-4801-b47d-8f2f7ec80b16 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.461248 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1179/2378] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:38 2022] PUT /identity/v3/projects/71ae79b73bb548818c9fd2cf2e612918/users/f6d361c855374fba8b62baf73f160e27/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:38.475606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d24a82-6b39-471f-a5e0-3514d5a2b48d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.476178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d24a82-6b39-471f-a5e0-3514d5a2b48d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.476178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d24a82-6b39-471f-a5e0-3514d5a2b48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.540558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f7d24a82-6b39-471f-a5e0-3514d5a2b48d None None] MFA Rules not processed for user `f6d361c855374fba8b62baf73f160e27`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:38.541726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f7d24a82-6b39-471f-a5e0-3514d5a2b48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.572771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.573400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.575006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.575286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.575945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.583926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.645076 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1200/2379] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/projects => generated 389 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:38.646036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.650064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.722920 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1180/2380] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:38.726773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d24a82-6b39-471f-a5e0-3514d5a2b48d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.754569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d24a82-6b39-471f-a5e0-3514d5a2b48d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.827746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247d7ee-e0f9-4f12-9eb4-224f69567e7b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.846240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.847025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.847322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.847602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.850024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.875134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.888660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.890410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.890694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.890965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.892117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.899014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.903647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.948443 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1181/2381] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/users => generated 295 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:38.949524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.970907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.976591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.984227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4617442-9b37-4f18-9656-01318ad2cefb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.997038 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1201/2382] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:38.997950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.999681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.000366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.000647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.000922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.001639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.003914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.041847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9ce4d-2a64-458b-839d-6d896ad9256d service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.057103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.057445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.057859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.058434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.065535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.071328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.090925 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1182/2383] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/projects => generated 409 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:39.100359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.110586 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1202/2384] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:39.112224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.141955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca493f-4a55-41c0-a5ec-8a0d3f7a759e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.233019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51ea26-60fa-470d-b985-e71da3b98f1c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.242479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.243326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.243610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.243882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.244628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.247321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.291796 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1183/2385] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:39 2022] POST /identity/v3/users => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:39.295559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.296195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.296474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.296715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.296854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.298083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.299736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.316194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b63ed-d3ac-402f-a035-922831a58764 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.322970 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1203/2386] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:39.325956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.332119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.333561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.336412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.337071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.338382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.342643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.349821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b51a14-5c23-4dd6-8ac8-a625ccad138f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.355592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a16b3-d18a-40a6-8e24-d01d0f388f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.356278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a16b3-d18a-40a6-8e24-d01d0f388f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.356620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a16b3-d18a-40a6-8e24-d01d0f388f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.356892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a16b3-d18a-40a6-8e24-d01d0f388f94 admin admin] PATH_INFO: `/v3/projects/4a80cf8bf5be4795a0be8a65a5b389e5/users/2fc322d5713f443d8351f65b132e703a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.377494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a16b3-d18a-40a6-8e24-d01d0f388f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a80cf8bf5be4795a0be8a65a5b389e5, user_id=2fc322d5713f443d8351f65b132e703a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.380930 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1184/2387] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:39.381317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a16b3-d18a-40a6-8e24-d01d0f388f94 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.401845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.411629 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1204/2388] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:39 2022] PUT /identity/v3/projects/4a80cf8bf5be4795a0be8a65a5b389e5/users/2fc322d5713f443d8351f65b132e703a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:39.416886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3806e-d5f3-4548-89c2-eb56d423919a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.417175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3806e-d5f3-4548-89c2-eb56d423919a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.417607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3806e-d5f3-4548-89c2-eb56d423919a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.431505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525d497-3fed-4447-bb09-b36806afdcb0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.467186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-08c3806e-d5f3-4548-89c2-eb56d423919a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:39.472049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-08c3806e-d5f3-4548-89c2-eb56d423919a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:39.573158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.591250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.591558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.591839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] PATH_INFO: `/v3/projects/175f243d1beb459ba4f43d08fe4e9e9b/users/7c2372326f184b4a9cb7a2cffa3457ca/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.640566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] RBAC: Authorizing `identity:create_grant(project_id=175f243d1beb459ba4f43d08fe4e9e9b, user_id=7c2372326f184b4a9cb7a2cffa3457ca, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.646655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.678075 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1205/2389] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:39.687850 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1185/2390] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:39 2022] PUT /identity/v3/projects/175f243d1beb459ba4f43d08fe4e9e9b/users/7c2372326f184b4a9cb7a2cffa3457ca/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:39.689236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3806e-d5f3-4548-89c2-eb56d423919a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.721238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3806e-d5f3-4548-89c2-eb56d423919a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.784192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:39.902281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.903251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.903538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.903807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.904474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.909641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.943152 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1206/2391] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:39.944499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.971564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822dd06-c450-453b-952b-e7eba0bd26db admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.989513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.996980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:39.997392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.998118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.998469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.998710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] PATH_INFO: `/v3/projects/175f243d1beb459ba4f43d08fe4e9e9b/users/7c2372326f184b4a9cb7a2cffa3457ca/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.011659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=175f243d1beb459ba4f43d08fe4e9e9b, user_id=7c2372326f184b4a9cb7a2cffa3457ca, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.014520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.047559 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1186/2392] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:39 2022] PUT /identity/v3/projects/175f243d1beb459ba4f43d08fe4e9e9b/users/7c2372326f184b4a9cb7a2cffa3457ca/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:40.047559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.051416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:40.083986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.136602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648f3a-3ea8-48c3-b1af-a209e200e90b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.191740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816ac96-124c-4e9c-9d26-97128ee08dbd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.204906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.205614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.205890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.206169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] PATH_INFO: `/v3/users/3d90b14b5f5948b0a915dfd56b41af29` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.217110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d90b14b5f5948b0a915dfd56b41af29)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.219169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.280708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7f6cc-1c25-4421-9ec1-f9c78d7b3f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.281324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f6cc-1c25-4421-9ec1-f9c78d7b3f7e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.287967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f6cc-1c25-4421-9ec1-f9c78d7b3f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.289084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f6cc-1c25-4421-9ec1-f9c78d7b3f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.308535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7f6cc-1c25-4421-9ec1-f9c78d7b3f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.310633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7f6cc-1c25-4421-9ec1-f9c78d7b3f7e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.318289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d90b14b5f5948b0a915dfd56b41af29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.318960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d90b14b5f5948b0a915dfd56b41af29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.328825 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1207/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:51:40.330357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.330678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.330995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.335194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d90b14b5f5948b0a915dfd56b41af29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.336035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d90b14b5f5948b0a915dfd56b41af29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.364915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-61d7a1a1-5ef5-4426-a01d-52f550c99a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d90b14b5f5948b0a915dfd56b41af29'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.397188 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1187/2394] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:40 2022] DELETE /identity/v3/users/3d90b14b5f5948b0a915dfd56b41af29 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:40.402621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.403299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.403534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.412197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] MFA Rules not processed for user `7c2372326f184b4a9cb7a2cffa3457ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:40.414452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:40.454667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:40.455987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:40.601520 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1208/2395] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:40 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:40.606497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.612261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:40.648752 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1188/2396] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:40.653270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.727449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.729993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:40.824029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.825107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f04a1-8311-43d5-96e3-fd5830b3df23 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.837576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.838245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.838554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.839811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] PATH_INFO: `/v3/users/f6d361c855374fba8b62baf73f160e27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.843148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d361c855374fba8b62baf73f160e27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.847463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.864853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f395a52-b343-4d96-988e-ab0140409766 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.869492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e243501-d320-4eaf-b2bb-a80b4893497e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.870692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e243501-d320-4eaf-b2bb-a80b4893497e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.871078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e243501-d320-4eaf-b2bb-a80b4893497e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.871416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e243501-d320-4eaf-b2bb-a80b4893497e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.885980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e243501-d320-4eaf-b2bb-a80b4893497e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.890781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e243501-d320-4eaf-b2bb-a80b4893497e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.906746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1189/2397] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:40.932015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d361c855374fba8b62baf73f160e27'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.949226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d361c855374fba8b62baf73f160e27'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.963887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d361c855374fba8b62baf73f160e27'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.966984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d361c855374fba8b62baf73f160e27'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.967757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d361c855374fba8b62baf73f160e27'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:40.988239 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1209/2398] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:40 2022] DELETE /identity/v3/users/f6d361c855374fba8b62baf73f160e27 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:40.989965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.990273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.990661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.019814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:41.021048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:41.122512 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1190/2399] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:41.144647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.147367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:41.250984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98170e3a-e3d8-4c93-ab09-13ee5f4312e1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.260226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.261808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.262098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.262700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.291726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.296989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.313602 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1210/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:42.193065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.194602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:42.219179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcab0d-d35f-416e-a360-130033536736 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.236023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.236803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.237148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.237487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] PATH_INFO: `/v3/projects/71ae79b73bb548818c9fd2cf2e612918` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.238761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ae79b73bb548818c9fd2cf2e612918)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.240748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.359185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ae79b73bb548818c9fd2cf2e612918', 'request_id': 'req-b9825f76-cf45-4c90-aa53-ff01d5d8a274'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.373414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ae79b73bb548818c9fd2cf2e612918', 'request_id': 'req-b9825f76-cf45-4c90-aa53-ff01d5d8a274'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.417049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ae79b73bb548818c9fd2cf2e612918', 'request_id': 'req-b9825f76-cf45-4c90-aa53-ff01d5d8a274'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.487477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] The token cache is being invalidate because project 71ae79b73bb548818c9fd2cf2e612918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:42.487794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.488641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.490543 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1191/2401] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:42 2022] DELETE /identity/v3/projects/71ae79b73bb548818c9fd2cf2e612918 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:42.495679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.525028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245182e-b549-48f3-bda9-7f7e43fdd3a9 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.621802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.739989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.741794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.742106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.742407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] PATH_INFO: `/v3/users/7c2372326f184b4a9cb7a2cffa3457ca` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.746743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2372326f184b4a9cb7a2cffa3457ca)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.749387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.826167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2372326f184b4a9cb7a2cffa3457ca'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.848439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2372326f184b4a9cb7a2cffa3457ca'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.861256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2372326f184b4a9cb7a2cffa3457ca'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.862558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2372326f184b4a9cb7a2cffa3457ca'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.863690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2372326f184b4a9cb7a2cffa3457ca'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:42.885054 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1211/2402] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:42 2022] DELETE /identity/v3/users/7c2372326f184b4a9cb7a2cffa3457ca => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:43.598354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.601577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:43.619338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9825f76-cf45-4c90-aa53-ff01d5d8a274 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.746026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.747745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.748434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.748914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] PATH_INFO: `/v3/projects/fb724c88a4d841489ed1a5c9364025a0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.750717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb724c88a4d841489ed1a5c9364025a0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.755088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.758504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.877985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb724c88a4d841489ed1a5c9364025a0', 'request_id': 'req-de66b44a-e508-4315-a44f-04fdea737210'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:43.898511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb724c88a4d841489ed1a5c9364025a0', 'request_id': 'req-de66b44a-e508-4315-a44f-04fdea737210'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:43.919558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb724c88a4d841489ed1a5c9364025a0', 'request_id': 'req-de66b44a-e508-4315-a44f-04fdea737210'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:43.939751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] The token cache is being invalidate because project fb724c88a4d841489ed1a5c9364025a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:43.941350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:43.944634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:43.947277 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1192/2403] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:43 2022] DELETE /identity/v3/projects/fb724c88a4d841489ed1a5c9364025a0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:43.951618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.954932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:43.991425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed72df-2bda-48b6-8957-be2a7a3d8bb4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.163057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.163725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.164019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.164264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] PATH_INFO: `/v3/users/0cda1f7458294fcd8087ff7c65edbd6a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.166011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cda1f7458294fcd8087ff7c65edbd6a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.168132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.264003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cda1f7458294fcd8087ff7c65edbd6a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.273634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:44.294250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cda1f7458294fcd8087ff7c65edbd6a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.322782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cda1f7458294fcd8087ff7c65edbd6a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.323552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cda1f7458294fcd8087ff7c65edbd6a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.324268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99053550-90a1-441c-bf35-9f34019a8d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cda1f7458294fcd8087ff7c65edbd6a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.366131 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1212/2404] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:43 2022] DELETE /identity/v3/users/0cda1f7458294fcd8087ff7c65edbd6a => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:44.581251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.583204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:44.622286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66b44a-e508-4315-a44f-04fdea737210 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.775959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.777097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.779319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.779884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] PATH_INFO: `/v3/projects/175f243d1beb459ba4f43d08fe4e9e9b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.781501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175f243d1beb459ba4f43d08fe4e9e9b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.784771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.849009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175f243d1beb459ba4f43d08fe4e9e9b', 'request_id': 'req-a8b9daeb-e053-460e-bc05-040e09b08412'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.856198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175f243d1beb459ba4f43d08fe4e9e9b', 'request_id': 'req-a8b9daeb-e053-460e-bc05-040e09b08412'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.872907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175f243d1beb459ba4f43d08fe4e9e9b', 'request_id': 'req-a8b9daeb-e053-460e-bc05-040e09b08412'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.890360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] The token cache is being invalidate because project 175f243d1beb459ba4f43d08fe4e9e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:44.891782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.893952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a8b9daeb-e053-460e-bc05-040e09b08412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:44.902523 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1193/2405] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:44 2022] DELETE /identity/v3/projects/175f243d1beb459ba4f43d08fe4e9e9b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:44.904562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8338573-a7e3-4cef-a65d-b6c0595b8a77 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.904946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8338573-a7e3-4cef-a65d-b6c0595b8a77 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.905338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8338573-a7e3-4cef-a65d-b6c0595b8a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.942592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c8338573-a7e3-4cef-a65d-b6c0595b8a77 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:44.944407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c8338573-a7e3-4cef-a65d-b6c0595b8a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:45.040194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1213/2406] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:45.048538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1c50a-e7aa-4c45-befa-65bfc334df5a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.048853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1c50a-e7aa-4c45-befa-65bfc334df5a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.049130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1c50a-e7aa-4c45-befa-65bfc334df5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.088096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-51e1c50a-e7aa-4c45-befa-65bfc334df5a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:45.089424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-51e1c50a-e7aa-4c45-befa-65bfc334df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:45.123063 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1194/2407] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:45.124481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8338573-a7e3-4cef-a65d-b6c0595b8a77 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.137844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8338573-a7e3-4cef-a65d-b6c0595b8a77 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.155186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.167565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.169963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.170649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.171601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:45.172156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.175666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.218211 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1214/2408] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:45.222862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1c50a-e7aa-4c45-befa-65bfc334df5a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.234182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1c50a-e7aa-4c45-befa-65bfc334df5a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.253500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.254166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.254443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.254708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.255363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.257796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.287854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:45.326486 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1195/2409] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:45 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:45.331966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.344607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c12b-fa94-40f4-ac55-7cccfc3e981f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.480187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.480835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.481115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.481389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.482258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.487436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.552888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.560812 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1215/2410] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:45 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:45.564730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4ad5d-292a-416d-b29e-a736667b8d00 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.594470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.597996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.597996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.597996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.597996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.598348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.627556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.634587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1196/2411] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:45.638429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379ec5-747c-4298-8846-462b06e50832 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.654244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8747d55-54fc-4f36-b298-be4dd5fdda26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.654880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8747d55-54fc-4f36-b298-be4dd5fdda26 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.655167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8747d55-54fc-4f36-b298-be4dd5fdda26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.655436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8747d55-54fc-4f36-b298-be4dd5fdda26 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.656099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8747d55-54fc-4f36-b298-be4dd5fdda26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.658920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8747d55-54fc-4f36-b298-be4dd5fdda26 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.689709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.695204 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1216/2412] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:45.702257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580b2e2-f221-47c3-b644-e1defb8925eb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.719580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.720466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.720890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.721338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] PATH_INFO: `/v3/projects/0eaaf70380604bbd9496ca1f77b8e4b0/users/13e22fbbe5164ea9995633bab30ad42e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.763751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eaaf70380604bbd9496ca1f77b8e4b0, user_id=13e22fbbe5164ea9995633bab30ad42e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.767849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.826142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ac4f6-a3c6-423d-93ae-4d2871266958 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.826459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ac4f6-a3c6-423d-93ae-4d2871266958 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.826687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ac4f6-a3c6-423d-93ae-4d2871266958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.830550 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1197/2413] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:45 2022] PUT /identity/v3/projects/0eaaf70380604bbd9496ca1f77b8e4b0/users/13e22fbbe5164ea9995633bab30ad42e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:45.864143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6e7ac4f6-a3c6-423d-93ae-4d2871266958 None None] MFA Rules not processed for user `13e22fbbe5164ea9995633bab30ad42e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:45.865895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6e7ac4f6-a3c6-423d-93ae-4d2871266958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:45.919108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:46.023955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1217/2414] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:46.285245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.287121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:46.437302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81befd-da2f-43f3-8f1a-29c0201220f0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:46.449372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.450366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.450641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.450910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.466574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:46.468143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:46.487748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1198/2415] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:46.588659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccdcbb-8f16-4b2b-8003-49ff5a0bf580 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.589083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccdcbb-8f16-4b2b-8003-49ff5a0bf580 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.589158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccdcbb-8f16-4b2b-8003-49ff5a0bf580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.618895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-34ccdcbb-8f16-4b2b-8003-49ff5a0bf580 None None] MFA Rules not processed for user `226a87a1f393420b8d19ec2390e9d063`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:46.629156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34ccdcbb-8f16-4b2b-8003-49ff5a0bf580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:46.862049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.865394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:46.930937 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1218/2416] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:47.081408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54044f2-7684-4428-82f3-f4f765e72374 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.094024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.094406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.094687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.094961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] PATH_INFO: `/v3/projects/0cebe9dbf7d049799b007300b7c1ce41` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.097127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cebe9dbf7d049799b007300b7c1ce41)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.101352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.203879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.206004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cebe9dbf7d049799b007300b7c1ce41', 'request_id': 'req-c92e2b48-6ed2-41f5-b6be-013cedcc3810'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:47.219168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cebe9dbf7d049799b007300b7c1ce41', 'request_id': 'req-c92e2b48-6ed2-41f5-b6be-013cedcc3810'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:47.241836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cebe9dbf7d049799b007300b7c1ce41', 'request_id': 'req-c92e2b48-6ed2-41f5-b6be-013cedcc3810'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:47.267210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] The token cache is being invalidate because project 0cebe9dbf7d049799b007300b7c1ce41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:47.267518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:47.268220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c92e2b48-6ed2-41f5-b6be-013cedcc3810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:47.270950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1199/2417] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:46 2022] DELETE /identity/v3/projects/0cebe9dbf7d049799b007300b7c1ce41 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:47.276694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056b2f-849d-4767-98b3-6a1aa3568af7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.278226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056b2f-849d-4767-98b3-6a1aa3568af7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.278653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056b2f-849d-4767-98b3-6a1aa3568af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.323980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-84056b2f-849d-4767-98b3-6a1aa3568af7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:47.328935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-84056b2f-849d-4767-98b3-6a1aa3568af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:47.447037 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1219/2418] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:47.454689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6324b-508e-4203-9f7a-5b52b9f6e978 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.455765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6324b-508e-4203-9f7a-5b52b9f6e978 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.455765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6324b-508e-4203-9f7a-5b52b9f6e978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.493332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2aa6324b-508e-4203-9f7a-5b52b9f6e978 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:47.496989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2aa6324b-508e-4203-9f7a-5b52b9f6e978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:47.512006 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1200/2419] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:47.512992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84056b2f-849d-4767-98b3-6a1aa3568af7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.525530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84056b2f-849d-4767-98b3-6a1aa3568af7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.533996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.534865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.535291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.535704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.536694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:47.537301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.539264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.555375 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1220/2420] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:47.559934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6324b-508e-4203-9f7a-5b52b9f6e978 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.579718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6324b-508e-4203-9f7a-5b52b9f6e978 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.585006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.585696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.586177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.586177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.586793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.588704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.666640 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1201/2421] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:47 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:47.673890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.685011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d5cbe-e1c4-499f-9d14-8f89f8130d66 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.839388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.850668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.851505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.851932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.852343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.853284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.856352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.900348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1221/2422] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:47 2022] POST /identity/v3/users => generated 296 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:47.904841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.919171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b5d95-5a31-4cdc-b3cd-79fb2689b3ef admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.926663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.927477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.928038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.933883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.934959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.940050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.965312 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1202/2423] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:47.969139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.980965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f202d73-4936-4666-9cb1-5c81801d793b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.986654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdec884-d061-41d2-830a-3d948ca04d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.987288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdec884-d061-41d2-830a-3d948ca04d1f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.988006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdec884-d061-41d2-830a-3d948ca04d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.988165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdec884-d061-41d2-830a-3d948ca04d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.988837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdec884-d061-41d2-830a-3d948ca04d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.990680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdec884-d061-41d2-830a-3d948ca04d1f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.007772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1222/2424] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:48.009280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.021383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147fb1a-3dae-493b-9a87-d5bb2e5bf0bc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.045552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.045552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.045552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.045928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] PATH_INFO: `/v3/projects/ec5c7bf7759d4ac8b73a1365c1f3fc9f/users/29cff8178d0d44389c9bb8aa0cfbcd3c/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.073952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5c7bf7759d4ac8b73a1365c1f3fc9f, user_id=29cff8178d0d44389c9bb8aa0cfbcd3c, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.078446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.106772 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1203/2425] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:48 2022] PUT /identity/v3/projects/ec5c7bf7759d4ac8b73a1365c1f3fc9f/users/29cff8178d0d44389c9bb8aa0cfbcd3c/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:48.117171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb902-adfb-4590-847c-e3d0aecb8a9f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.117478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb902-adfb-4590-847c-e3d0aecb8a9f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.118080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb902-adfb-4590-847c-e3d0aecb8a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.189083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-295eb902-adfb-4590-847c-e3d0aecb8a9f None None] MFA Rules not processed for user `29cff8178d0d44389c9bb8aa0cfbcd3c`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:48.191054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-295eb902-adfb-4590-847c-e3d0aecb8a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:48.286919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1223/2426] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:48.291804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.518475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1816b1-fe17-422e-bd98-2c0beb11b3d7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.523399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.524004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.524547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.532263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.533024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.536205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.610143 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1204/2427] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:48 2022] POST /identity/v3/projects => generated 393 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:48.614996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295eb902-adfb-4590-847c-e3d0aecb8a9f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.627547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295eb902-adfb-4590-847c-e3d0aecb8a9f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.884474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.885159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.885368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.886510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.887219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.889188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.959797 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1224/2428] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:48 2022] POST /identity/v3/users => generated 303 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:48.967573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.988164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d859bda-0c13-4e7a-addb-a96a74a7af7d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.003275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.005097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.006015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.007721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.009967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.014940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.064624 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1205/2429] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:49.069713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.086679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e6ef0-ae63-4e6b-af4d-036be334ab68 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.116666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.119936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.120522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.120953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] PATH_INFO: `/v3/projects/5385b725784c41cdb8dcc0485e8abac8/users/712bb5fd076549369ca67951d6e0cbae/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.157817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5385b725784c41cdb8dcc0485e8abac8, user_id=712bb5fd076549369ca67951d6e0cbae, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.160911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.183244 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1225/2430] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:49 2022] PUT /identity/v3/projects/5385b725784c41cdb8dcc0485e8abac8/users/712bb5fd076549369ca67951d6e0cbae/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:49.187769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.210229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.217610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d86b1-7242-4977-9e4c-12a8d6052a55 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.218294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:49.308800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:49.324080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:49.377672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.379485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.379769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.380124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.380801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.385037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.394953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5b59d-b595-4e22-8237-4f654c0b850f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.407905 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1206/2431] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:49.408982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.414598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.415715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.416223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.416744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] PATH_INFO: `/v3/users/13e22fbbe5164ea9995633bab30ad42e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.418439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e22fbbe5164ea9995633bab30ad42e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.432731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.435703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7231968-8210-47fb-bcd0-b307d62178cf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.453876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf12e7-5a0a-4612-81f3-ca715252cdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.454580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf12e7-5a0a-4612-81f3-ca715252cdb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.454859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf12e7-5a0a-4612-81f3-ca715252cdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.455133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf12e7-5a0a-4612-81f3-ca715252cdb6 admin admin] PATH_INFO: `/v3/projects/5385b725784c41cdb8dcc0485e8abac8/users/712bb5fd076549369ca67951d6e0cbae/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.459987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf12e7-5a0a-4612-81f3-ca715252cdb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5385b725784c41cdb8dcc0485e8abac8, user_id=712bb5fd076549369ca67951d6e0cbae, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.466813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf12e7-5a0a-4612-81f3-ca715252cdb6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.507199 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1207/2432] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:49 2022] PUT /identity/v3/projects/5385b725784c41cdb8dcc0485e8abac8/users/712bb5fd076549369ca67951d6e0cbae/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:49.511904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.512199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.512464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.551042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e22fbbe5164ea9995633bab30ad42e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:49.582317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] MFA Rules not processed for user `712bb5fd076549369ca67951d6e0cbae`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:49.583557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:49.603079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e22fbbe5164ea9995633bab30ad42e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:49.625800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e22fbbe5164ea9995633bab30ad42e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:49.626954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e22fbbe5164ea9995633bab30ad42e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:49.627892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a730704-93af-4412-9589-3f5f218beefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e22fbbe5164ea9995633bab30ad42e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:49.658480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1226/2433] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:49 2022] DELETE /identity/v3/users/13e22fbbe5164ea9995633bab30ad42e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:49.670598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.678504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.679096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.735573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:49.742316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:49.834065 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1208/2434] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:49.847398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.849101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:49.934435 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1227/2435] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:49.979252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.981023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:50.056978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c112ced-8e00-45b0-a4a7-9110e74807a2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:50.072264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:50.072936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.073217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.074987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] PATH_INFO: `/v3/users/29cff8178d0d44389c9bb8aa0cfbcd3c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.078099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29cff8178d0d44389c9bb8aa0cfbcd3c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:50.083345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:50.157659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571644ab-6931-45c7-b010-de109ac6a9b1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:50.165433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb37ea8-53a7-4455-a46c-9ba4a8d32a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:50.166118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb37ea8-53a7-4455-a46c-9ba4a8d32a58 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.166402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb37ea8-53a7-4455-a46c-9ba4a8d32a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.166669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb37ea8-53a7-4455-a46c-9ba4a8d32a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.177283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:50.202527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb37ea8-53a7-4455-a46c-9ba4a8d32a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:50.204283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb37ea8-53a7-4455-a46c-9ba4a8d32a58 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:50.229580 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1228/2436] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:50.454797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e937ea-20cd-42ae-933b-b2b312a99f95 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.460182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e937ea-20cd-42ae-933b-b2b312a99f95 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.460559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e937ea-20cd-42ae-933b-b2b312a99f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.524562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29cff8178d0d44389c9bb8aa0cfbcd3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:50.528872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29cff8178d0d44389c9bb8aa0cfbcd3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:50.544109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:50.547669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-10e937ea-20cd-42ae-933b-b2b312a99f95 None None] MFA Rules not processed for user `2bdc03ea28d940518fb8d67567c9f2bc`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:50.554136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29cff8178d0d44389c9bb8aa0cfbcd3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:50.555132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29cff8178d0d44389c9bb8aa0cfbcd3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:50.556849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10e937ea-20cd-42ae-933b-b2b312a99f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:50.610068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9cb51fe1-6f8d-4127-9e58-24f392261324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29cff8178d0d44389c9bb8aa0cfbcd3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:50.658919 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1209/2437] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:49 2022] DELETE /identity/v3/users/29cff8178d0d44389c9bb8aa0cfbcd3c => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:50.663552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.664814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.667319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.731682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:50.733551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:50.799443 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1229/2438] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:50 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:50.801148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.803932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.804346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.845473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] MFA Rules not processed for user `f1134231b4a545fdb136a5204cb4f289`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:50.847860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:50.878840 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1210/2439] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:50.950605 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1230/2440] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:50 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:50.962506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:50.964395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:50.965223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:50.967729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:51.086916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ffd82-4766-44ef-bc90-d0dce7aa84e8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:51.098312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:51.099915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.101487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.102019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.126635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:51.128417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:51.137926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b20f90-0861-45d4-aa20-ee0c0bbca6b8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:51.153920 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1211/2441] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:51.169429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dba577-b639-403c-b330-5e82268d3b1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:51.170690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba577-b639-403c-b330-5e82268d3b1e service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.170979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba577-b639-403c-b330-5e82268d3b1e service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.171253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba577-b639-403c-b330-5e82268d3b1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.174759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f1dba577-b639-403c-b330-5e82268d3b1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:51.203078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dba577-b639-403c-b330-5e82268d3b1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:51.204088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dba577-b639-403c-b330-5e82268d3b1e service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:51.218115 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1231/2442] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:50 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:52.135188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.136787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:52.154443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461127-2a78-486b-a649-a81015916e20 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.161755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.162531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.162812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.163136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] PATH_INFO: `/v3/projects/0eaaf70380604bbd9496ca1f77b8e4b0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.166012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eaaf70380604bbd9496ca1f77b8e4b0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.168181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.300460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eaaf70380604bbd9496ca1f77b8e4b0', 'request_id': 'req-4d939166-26c3-4f1f-ad53-6419de5ce634'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:52.310642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eaaf70380604bbd9496ca1f77b8e4b0', 'request_id': 'req-4d939166-26c3-4f1f-ad53-6419de5ce634'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:52.333222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eaaf70380604bbd9496ca1f77b8e4b0', 'request_id': 'req-4d939166-26c3-4f1f-ad53-6419de5ce634'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:52.354323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:52.367297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] The token cache is being invalidate because project 0eaaf70380604bbd9496ca1f77b8e4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:52.367850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:52.374715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d939166-26c3-4f1f-ad53-6419de5ce634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:52.377219 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1212/2443] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:52 2022] DELETE /identity/v3/projects/0eaaf70380604bbd9496ca1f77b8e4b0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:52.382530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ffa76-e4e1-4c64-bfdd-35b379955c09 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.383132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ffa76-e4e1-4c64-bfdd-35b379955c09 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.383272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ffa76-e4e1-4c64-bfdd-35b379955c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.436846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ca2ffa76-e4e1-4c64-bfdd-35b379955c09 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:52.438459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ca2ffa76-e4e1-4c64-bfdd-35b379955c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:52.449993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:52.562715 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1232/2444] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:52.643237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a734dc3-8faa-474f-9b92-badc8b63d562 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.644301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a734dc3-8faa-474f-9b92-badc8b63d562 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.644683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a734dc3-8faa-474f-9b92-badc8b63d562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.707293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1a734dc3-8faa-474f-9b92-badc8b63d562 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:52.708445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1a734dc3-8faa-474f-9b92-badc8b63d562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:52.729382 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1213/2445] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:52.730781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ffa76-e4e1-4c64-bfdd-35b379955c09 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.745991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ffa76-e4e1-4c64-bfdd-35b379955c09 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.765891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.769111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.769443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.771622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.772362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:52.775349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.779211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.797356 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1233/2446] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:52.798531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a734dc3-8faa-474f-9b92-badc8b63d562 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.815814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.816365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a734dc3-8faa-474f-9b92-badc8b63d562 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.820235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:52.823168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.823433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.823740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.824013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.824811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.826825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.875882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a32115-0d82-4dae-8046-cf2ae20426b6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.889745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.890384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.890691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.891366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] PATH_INFO: `/v3/projects/ec5c7bf7759d4ac8b73a1365c1f3fc9f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.892712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5c7bf7759d4ac8b73a1365c1f3fc9f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.895052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.902665 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1214/2447] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:52 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:52.907029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.014934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5c7bf7759d4ac8b73a1365c1f3fc9f', 'request_id': 'req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.048527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5c7bf7759d4ac8b73a1365c1f3fc9f', 'request_id': 'req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.061917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5c7bf7759d4ac8b73a1365c1f3fc9f', 'request_id': 'req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.091187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd443aca-bc75-42f2-a3cb-e7571e537ebc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.093603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] The token cache is being invalidate because project ec5c7bf7759d4ac8b73a1365c1f3fc9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:53.093908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.094683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.096790 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1234/2448] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:52 2022] DELETE /identity/v3/projects/ec5c7bf7759d4ac8b73a1365c1f3fc9f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:53.102453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.108650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:53.239780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.241198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.241785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.242273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.243137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.253584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.298421 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1215/2449] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:52 2022] POST /identity/v3/users => generated 302 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:53.302594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.307046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:53.351683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85e3e-9cc1-4adc-bcee-d62f3679dbf2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.369163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.370136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.370635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.371079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.371978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.375463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.384644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37f197-2a29-4919-ad5c-2876e3ddfdd6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.401988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.405703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.406102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1216/2450] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:53.409544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.409942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] PATH_INFO: `/v3/users/712bb5fd076549369ca67951d6e0cbae` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.410747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.414943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=712bb5fd076549369ca67951d6e0cbae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.425204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.433428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45a2f1-ac95-419e-a1bb-879e401d1384 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.453923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.454862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.455287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.455731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.457114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.475089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.499124 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1217/2451] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:53.500818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.516468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:53.546477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca35b88-9644-4a43-a235-1bbe49a1d838 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.593746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f45240-12fb-485f-8671-b90a85a390fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.596079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45240-12fb-485f-8671-b90a85a390fa admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.600896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45240-12fb-485f-8671-b90a85a390fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.601432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f45240-12fb-485f-8671-b90a85a390fa admin admin] PATH_INFO: `/v3/projects/6589c8be0f944eb7bb726df3d468d2f8/users/5b6c0aef786d4fd7ba8c5bd00a097fe6/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.624956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712bb5fd076549369ca67951d6e0cbae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.651128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f45240-12fb-485f-8671-b90a85a390fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6589c8be0f944eb7bb726df3d468d2f8, user_id=5b6c0aef786d4fd7ba8c5bd00a097fe6, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.658271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712bb5fd076549369ca67951d6e0cbae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.664207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f45240-12fb-485f-8671-b90a85a390fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.686914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712bb5fd076549369ca67951d6e0cbae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.687772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712bb5fd076549369ca67951d6e0cbae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.688492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712bb5fd076549369ca67951d6e0cbae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:53.726011 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1218/2452] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:53 2022] PUT /identity/v3/projects/6589c8be0f944eb7bb726df3d468d2f8/users/5b6c0aef786d4fd7ba8c5bd00a097fe6/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:53.729737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.730337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.730736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.741752 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1235/2453] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:51:53 2022] DELETE /identity/v3/users/712bb5fd076549369ca67951d6e0cbae => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:53.784111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:53.785715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:53.846596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.018018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1219/2454] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:54.204025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.206437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:54.424753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bb58b-6d07-487e-b6dc-81e96bdfdfe3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.432219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa1c7b-c7cc-4f1f-aef2-5cfc66df3225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.433423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1c7b-c7cc-4f1f-aef2-5cfc66df3225 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.433963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1c7b-c7cc-4f1f-aef2-5cfc66df3225 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.434704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1c7b-c7cc-4f1f-aef2-5cfc66df3225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.467068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa1c7b-c7cc-4f1f-aef2-5cfc66df3225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.470183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa1c7b-c7cc-4f1f-aef2-5cfc66df3225 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.496236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1236/2455] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:51:56.938459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:56.940585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:56.962543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae254e0-5d61-49e8-badc-fbdd378dfca7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.968134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.969292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.969742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.970151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] PATH_INFO: `/v3/projects/5385b725784c41cdb8dcc0485e8abac8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.971657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5385b725784c41cdb8dcc0485e8abac8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.973756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.044875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5385b725784c41cdb8dcc0485e8abac8', 'request_id': 'req-17999303-55a6-46d5-8dc2-4f34b0a820c8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:57.054568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5385b725784c41cdb8dcc0485e8abac8', 'request_id': 'req-17999303-55a6-46d5-8dc2-4f34b0a820c8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:57.068137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5385b725784c41cdb8dcc0485e8abac8', 'request_id': 'req-17999303-55a6-46d5-8dc2-4f34b0a820c8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:57.083269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] The token cache is being invalidate because project 5385b725784c41cdb8dcc0485e8abac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:51:57.083865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:57.084691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17999303-55a6-46d5-8dc2-4f34b0a820c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:51:57.088458 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1220/2456] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:51:56 2022] DELETE /identity/v3/projects/5385b725784c41cdb8dcc0485e8abac8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:57.096648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283aaa8-54e6-4d37-aab7-a23d926af807 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.097158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283aaa8-54e6-4d37-aab7-a23d926af807 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.097376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283aaa8-54e6-4d37-aab7-a23d926af807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.130213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1283aaa8-54e6-4d37-aab7-a23d926af807 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:57.132632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1283aaa8-54e6-4d37-aab7-a23d926af807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:57.236217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.236496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.237150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.323679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] MFA Rules not processed for user `42f7c0fdad9346c58949928300021b07`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:57.325530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:57.349020 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1237/2457] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:51:57.366471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b2ceb-f913-4e7a-80ec-3fea3fdc9de9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.366928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b2ceb-f913-4e7a-80ec-3fea3fdc9de9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.367267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b2ceb-f913-4e7a-80ec-3fea3fdc9de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.457396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-644b2ceb-f913-4e7a-80ec-3fea3fdc9de9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:57.459758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-644b2ceb-f913-4e7a-80ec-3fea3fdc9de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:57.496216 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1238/2458] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:57.505185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b2ceb-f913-4e7a-80ec-3fea3fdc9de9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.505408 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1221/2459] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:57.510249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.510249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:57.537774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb786e85-1259-4a5f-98a7-dd245e6cf357 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.549706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b2ceb-f913-4e7a-80ec-3fea3fdc9de9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.555110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.556497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.557023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.557761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.558794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.565414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.583467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.584116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.584405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.584677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.585366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:57.585702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.594729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.632821 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1239/2460] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:57.638003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.644693 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1222/2461] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:57 2022] POST /identity/v3/projects => generated 403 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:57.648027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.673643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f04cce-e3b9-4521-a713-da25bf848672 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.692253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:57.740439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:57.804198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.805677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.806213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.808935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.810208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.813544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.844282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef93e2f-f405-4045-9431-9f51c9f1c276 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.854667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.855494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.857735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.860050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.862720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.867221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.880035 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1240/2462] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:57 2022] POST /identity/v3/projects => generated 375 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:57.886178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.926116 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1223/2463] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:57 2022] POST /identity/v3/users => generated 308 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:57.932088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.983926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ecef8-75a6-4872-b4e8-7cef178abb12 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.076071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4e2b6-44a1-41ce-8682-bf8692991330 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.099838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.100504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.100744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.101016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.101717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.103654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.127927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.139389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.141270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.141270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.141270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.142882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.156510 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1241/2464] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:57 2022] POST /identity/v3/users => generated 288 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:58.162592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:58.167459 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1224/2465] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:51:58.168523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:58.196213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1051-b94d-4e56-af66-136d980552c6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.202381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.203061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.203352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.203628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] PATH_INFO: `/v3/projects/6175b06907754fa0b579311bcf7a5d96/users/535f4800b67b46a3a6fd2d431ca88860/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.206918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9c5c8-e481-4c04-890d-1f632249af12 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.226500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.227400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.227400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.227789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.233602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.235730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6175b06907754fa0b579311bcf7a5d96, user_id=535f4800b67b46a3a6fd2d431ca88860, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.240546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.243620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.295413 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1242/2466] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:58.296423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:58.301488 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1225/2467] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:58 2022] PUT /identity/v3/projects/6175b06907754fa0b579311bcf7a5d96/users/535f4800b67b46a3a6fd2d431ca88860/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:58.302549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:58.324715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d6ff9-7074-40e3-8c57-88b6443e8381 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.456658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.458180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.458624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.459056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.459893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.466499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.478045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b00c7-937b-4d7e-9a46-44e8d2126b09 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.481885 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1243/2468] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:58.486133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.487100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.487405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:58.487548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.488056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.489150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.490926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.512231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568e02-c7ed-49ab-89df-95d1204ebd45 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.520491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1226/2469] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:58.522329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:58.530199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.531168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.532095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.532533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] PATH_INFO: `/v3/projects/aca5ff8c1a6143888f70ebdcde126b33/users/56357b6a7102410790a0307eb3f13091/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.548333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fdf1b-774f-4a4d-9f66-160d653910bd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.556924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f588ffb-dc18-4d00-be0a-d063f234e73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.559252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588ffb-dc18-4d00-be0a-d063f234e73c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.560613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588ffb-dc18-4d00-be0a-d063f234e73c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.560903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f588ffb-dc18-4d00-be0a-d063f234e73c admin admin] PATH_INFO: `/v3/projects/6175b06907754fa0b579311bcf7a5d96/users/535f4800b67b46a3a6fd2d431ca88860/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.567468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f588ffb-dc18-4d00-be0a-d063f234e73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6175b06907754fa0b579311bcf7a5d96, user_id=535f4800b67b46a3a6fd2d431ca88860, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.576072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f588ffb-dc18-4d00-be0a-d063f234e73c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.576559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca5ff8c1a6143888f70ebdcde126b33, user_id=56357b6a7102410790a0307eb3f13091, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.580577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:58.582593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.602082 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1227/2470] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:58 2022] PUT /identity/v3/projects/6175b06907754fa0b579311bcf7a5d96/users/535f4800b67b46a3a6fd2d431ca88860/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:58.615193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:58.632317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1244/2471] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:58 2022] PUT /identity/v3/projects/aca5ff8c1a6143888f70ebdcde126b33/users/56357b6a7102410790a0307eb3f13091/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:58.636824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0bb59-172b-4371-93bb-fa214e43fca1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.637131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0bb59-172b-4371-93bb-fa214e43fca1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.637412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0bb59-172b-4371-93bb-fa214e43fca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.718679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-99a0bb59-172b-4371-93bb-fa214e43fca1 None None] MFA Rules not processed for user `56357b6a7102410790a0307eb3f13091`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:58.719939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-99a0bb59-172b-4371-93bb-fa214e43fca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:58.879218 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1228/2472] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:51:58.884064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.055766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897e7f5-cbf9-430b-acad-e043b7af21d1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.074327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.075253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.075541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.075846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.076463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.083995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.177623 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1245/2473] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:51:58 2022] POST /identity/v3/projects => generated 373 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:59.181638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0bb59-172b-4371-93bb-fa214e43fca1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.196508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0bb59-172b-4371-93bb-fa214e43fca1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.326555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.326957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.328378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.329006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.330499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.335813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.382777 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1229/2474] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:51:59 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:59.386772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.404294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d701-5b34-4c3c-893b-065032c88fc7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.419954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.420602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.420884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.421247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.421941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.427003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.464006 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1246/2475] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:59.469278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.499884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bf65a-50d4-4ec9-91cf-88b3dd80a350 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.514298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.520847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.523008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.523818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] PATH_INFO: `/v3/projects/4befb6a27ba347ffa709bdf2f65e0a36/users/4ad9e32e3c074b82a46dccd767f7ce8d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.561933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4befb6a27ba347ffa709bdf2f65e0a36, user_id=4ad9e32e3c074b82a46dccd767f7ce8d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.574692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.609050 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1230/2476] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:59 2022] PUT /identity/v3/projects/4befb6a27ba347ffa709bdf2f65e0a36/users/4ad9e32e3c074b82a46dccd767f7ce8d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:59.613294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.628610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb9d83-97ca-4bb7-a54f-c6c2f7a4fabb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.764653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679fcf-a35f-4c37-a466-db26091a9113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.765248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679fcf-a35f-4c37-a466-db26091a9113 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.765635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679fcf-a35f-4c37-a466-db26091a9113 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.766648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679fcf-a35f-4c37-a466-db26091a9113 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.767468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679fcf-a35f-4c37-a466-db26091a9113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.770259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679fcf-a35f-4c37-a466-db26091a9113 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.823211 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1247/2477] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:51:59.824664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.840744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27acd4-8d65-449c-9e0c-a002789a2102 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.852051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.852806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.853084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.855668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] PATH_INFO: `/v3/projects/4befb6a27ba347ffa709bdf2f65e0a36/users/4ad9e32e3c074b82a46dccd767f7ce8d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.861586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4befb6a27ba347ffa709bdf2f65e0a36, user_id=4ad9e32e3c074b82a46dccd767f7ce8d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.869631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.894983 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1231/2478] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:51:59 2022] PUT /identity/v3/projects/4befb6a27ba347ffa709bdf2f65e0a36/users/4ad9e32e3c074b82a46dccd767f7ce8d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:59.914014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934c485-ec96-4562-bd96-7cdae62b8a13 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.919075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934c485-ec96-4562-bd96-7cdae62b8a13 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.919075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934c485-ec96-4562-bd96-7cdae62b8a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.983055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b934c485-ec96-4562-bd96-7cdae62b8a13 None None] MFA Rules not processed for user `4ad9e32e3c074b82a46dccd767f7ce8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:59.985120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b934c485-ec96-4562-bd96-7cdae62b8a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:00.121627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.150919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1248/2479] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:51:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:00.160032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.178831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136ad62-a757-449d-a5c1-c8836f63fdb1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.185067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8055c-c376-46ef-a38c-ccf87eae13b3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.186088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8055c-c376-46ef-a38c-ccf87eae13b3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.186513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8055c-c376-46ef-a38c-ccf87eae13b3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.186927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8055c-c376-46ef-a38c-ccf87eae13b3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.187721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8055c-c376-46ef-a38c-ccf87eae13b3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:00.188271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8055c-c376-46ef-a38c-ccf87eae13b3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.190182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8055c-c376-46ef-a38c-ccf87eae13b3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.208031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1232/2480] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:00.212674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934c485-ec96-4562-bd96-7cdae62b8a13 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.228433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934c485-ec96-4562-bd96-7cdae62b8a13 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.258880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.259662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.259871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.260083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.260746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.273803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.316602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.317300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.318052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.356060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1249/2481] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:00 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:00.359844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.386166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] MFA Rules not processed for user `6d7f9995a47046bda06e450f2f57aea8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:00.387518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:00.414827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.504560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fefb6-9505-4315-9a76-fd937c4cbacd tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.512161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.513258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.513678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.516405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.519567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:00.519567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.522288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.538342 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1250/2482] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:00.551659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.554760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:00.567567 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1233/2483] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:52:00.568799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.571869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:00.677185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba1000-e9c6-4977-8c67-45acd5450ccc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.684377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.684997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.685396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.685871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.686528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.688560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.709967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e41dbe-a659-4792-8afe-2dd48c2e758c tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.718691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1f2b4-aacb-40b6-b348-b2d8b9ea7c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.719336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f2b4-aacb-40b6-b348-b2d8b9ea7c53 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.719607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f2b4-aacb-40b6-b348-b2d8b9ea7c53 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.719872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f2b4-aacb-40b6-b348-b2d8b9ea7c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.727522 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1234/2484] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:00 2022] POST /identity/v3/projects => generated 405 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:00.731600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1f2b4-aacb-40b6-b348-b2d8b9ea7c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.732998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.734735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1f2b4-aacb-40b6-b348-b2d8b9ea7c53 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.748387 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1251/2485] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:00.789772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0639b-3b82-4c9c-a928-68bdf215dd58 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.789979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0639b-3b82-4c9c-a928-68bdf215dd58 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.790237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0639b-3b82-4c9c-a928-68bdf215dd58 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.792321 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1252/2486] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:00.799515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0639b-3b82-4c9c-a928-68bdf215dd58 None None] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:52:00.895118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e1edd-b77e-448a-af7e-93a8156bc69a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.903210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.911797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.912144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.912485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.921961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.923870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.941027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.966579 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1235/2487] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:00 2022] POST /identity/v3/users => generated 309 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:00.970490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.976030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0639b-3b82-4c9c-a928-68bdf215dd58 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.991908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcd6b6-6aa7-4ae0-8335-ac398573be0e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.999462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.000555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.000555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.001295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.001667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.010943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.026748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1236/2488] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:01.048799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.092452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb3800-07fb-4ad7-b2d4-230e116ffa5a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.103965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad37bfc-3adf-40e7-a109-8773c9a99609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.105654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad37bfc-3adf-40e7-a109-8773c9a99609 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.106106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad37bfc-3adf-40e7-a109-8773c9a99609 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.106550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad37bfc-3adf-40e7-a109-8773c9a99609 admin admin] PATH_INFO: `/v3/projects/9e7293acae7349e5994d719b758425a1/users/87bbc00e8087432d985ac8a43e06e54c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.118095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0639b-3b82-4c9c-a928-68bdf215dd58 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.131092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.131796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.132077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.132343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.135231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:get_project(project_id=66ad9f5ab0304cd8bbd551accb4212b8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.136445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.138908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.143360 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1253/2489] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8 => generated 369 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:01.147302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad37bfc-3adf-40e7-a109-8773c9a99609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7293acae7349e5994d719b758425a1, user_id=87bbc00e8087432d985ac8a43e06e54c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.162552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad37bfc-3adf-40e7-a109-8773c9a99609 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.207147 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1237/2490] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:01 2022] PUT /identity/v3/projects/9e7293acae7349e5994d719b758425a1/users/87bbc00e8087432d985ac8a43e06e54c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:01.210843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.233263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d777ae-80e2-4394-b248-319e62ea1c4b tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.372050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.372786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.373157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.373567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.374320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.380274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.391484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9d650-f15e-4bf7-a06e-0770736351a2 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.392207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9d650-f15e-4bf7-a06e-0770736351a2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.392322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9d650-f15e-4bf7-a06e-0770736351a2 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.393990 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1238/2491] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:01.403649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9d650-f15e-4bf7-a06e-0770736351a2 None None] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:52:01.404162 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1254/2492] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:01.408890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.439855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69498ab6-b53a-4d47-8447-4a6350b0eb83 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.453666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.454408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.454754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.455824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] PATH_INFO: `/v3/projects/9e7293acae7349e5994d719b758425a1/users/87bbc00e8087432d985ac8a43e06e54c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.462335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7293acae7349e5994d719b758425a1, user_id=87bbc00e8087432d985ac8a43e06e54c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.466992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.489602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9d650-f15e-4bf7-a06e-0770736351a2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.506445 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1255/2493] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:01 2022] PUT /identity/v3/projects/9e7293acae7349e5994d719b758425a1/users/87bbc00e8087432d985ac8a43e06e54c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:01.573930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9d650-f15e-4bf7-a06e-0770736351a2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.655769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.656497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.656885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.657207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.658479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:get_project(project_id=66ad9f5ab0304cd8bbd551accb4212b8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.660707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.679687 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1239/2494] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8 => generated 369 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:01.724895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.741442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6771-75b2-45d9-9925-3a1bff7dab4b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.746473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.748560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.748891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.749111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.749818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:01.750298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.752372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.766842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1256/2495] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:01.772159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.795377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6f1e7-8f7c-4896-8c7e-5bf71b683bf5 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.804590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf03107-b543-4a95-a469-7dd9f41dbce7 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.805351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03107-b543-4a95-a469-7dd9f41dbce7 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.811529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03107-b543-4a95-a469-7dd9f41dbce7 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.812955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03107-b543-4a95-a469-7dd9f41dbce7 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.814205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf03107-b543-4a95-a469-7dd9f41dbce7 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.816396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf03107-b543-4a95-a469-7dd9f41dbce7 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.860212 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1240/2496] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:01 2022] POST /identity/v3/users => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:01.867755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.892901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce5ec3-2748-462a-a7fe-e771b6ad9ab0 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.909160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.909827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.910107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.910428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.911093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:01.911423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.913044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.930857 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1257/2497] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:02.278416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b9c9-47c8-48fd-bb16-cbd6ce7199e7 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.279000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b9c9-47c8-48fd-bb16-cbd6ce7199e7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.279000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b9c9-47c8-48fd-bb16-cbd6ce7199e7 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.281681 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1241/2498] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:52:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:02.289260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:52:02.292343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:02.398163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.462619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.475743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76b382-5117-43f9-97d6-77f80100bee3 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.482811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.483609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.484025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.484772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.486952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:get_project(project_id=66ad9f5ab0304cd8bbd551accb4212b8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.490718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.494480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1258/2499] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8 => generated 369 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:02.779745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081c29e-e883-4902-b70f-6a9d32ab72e0 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.779745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081c29e-e883-4902-b70f-6a9d32ab72e0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.779996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081c29e-e883-4902-b70f-6a9d32ab72e0 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.781395 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1242/2500] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:02.790112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:52:02.804275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.814442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6d01-5510-4325-9062-d89999ef6b66 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.819288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.819933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.820211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.820481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.821643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:get_project(project_id=66ad9f5ab0304cd8bbd551accb4212b8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.824004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.826076 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1259/2501] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8 => generated 369 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:02.857565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081c29e-e883-4902-b70f-6a9d32ab72e0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.870760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081c29e-e883-4902-b70f-6a9d32ab72e0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.875511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.876156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.876441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.876716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/users/4066cd60f9df4c21887969d380704e48` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.893742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:delete_user(user_id=4066cd60f9df4c21887969d380704e48)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.895869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.897745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.963624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4066cd60f9df4c21887969d380704e48'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:02.965312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4066cd60f9df4c21887969d380704e48'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:02.984487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4066cd60f9df4c21887969d380704e48'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:02.985875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4066cd60f9df4c21887969d380704e48'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:03.025800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4066cd60f9df4c21887969d380704e48'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:03.083106 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1243/2502] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:02 2022] DELETE /identity/v3/users/4066cd60f9df4c21887969d380704e48 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:03.084543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.086489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:03.108670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95379157-0f8a-4119-b522-96cdc0500fec tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.244073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad9e32e3c074b82a46dccd767f7ce8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4befb6a27ba347ffa709bdf2f65e0a36', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.245353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.246065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.246674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] PATH_INFO: `/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.249984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorizing `identity:delete_project(project_id=66ad9f5ab0304cd8bbd551accb4212b8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.255065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.473844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ad9f5ab0304cd8bbd551accb4212b8', 'request_id': 'req-88ba972c-a7cb-4560-8668-c2c008c69866'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:03.492874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ad9f5ab0304cd8bbd551accb4212b8', 'request_id': 'req-88ba972c-a7cb-4560-8668-c2c008c69866'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:03.519651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ad9f5ab0304cd8bbd551accb4212b8', 'request_id': 'req-88ba972c-a7cb-4560-8668-c2c008c69866'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:03.540914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] The token cache is being invalidate because project 66ad9f5ab0304cd8bbd551accb4212b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:03.542683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:03.545337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-88ba972c-a7cb-4560-8668-c2c008c69866 tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:03.548435 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1260/2503] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:03 2022] DELETE /identity/v3/projects/66ad9f5ab0304cd8bbd551accb4212b8 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:03.554090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.557685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:03.713931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cb60f-a562-423b-86ee-1921eea9de5e tempest-QuotasAdminTestV257-775824531 tempest-QuotasAdminTestV257-775824531-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.866749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.869102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.869486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.869839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] PATH_INFO: `/v3/users/56357b6a7102410790a0307eb3f13091` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.871467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56357b6a7102410790a0307eb3f13091)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.873675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.957773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:04.019920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56357b6a7102410790a0307eb3f13091'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:04.021182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56357b6a7102410790a0307eb3f13091'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:04.038806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56357b6a7102410790a0307eb3f13091'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:04.040540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56357b6a7102410790a0307eb3f13091'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:04.085869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56357b6a7102410790a0307eb3f13091'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:04.138341 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1244/2504] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:03 2022] DELETE /identity/v3/users/56357b6a7102410790a0307eb3f13091 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:04.149963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0047d2-8b87-4d23-9e86-f783c3e7bffd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.150249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0047d2-8b87-4d23-9e86-f783c3e7bffd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.150529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0047d2-8b87-4d23-9e86-f783c3e7bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.185848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4e0047d2-8b87-4d23-9e86-f783c3e7bffd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:04.187166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4e0047d2-8b87-4d23-9e86-f783c3e7bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:04.265810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:04.290389 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1261/2505] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:04.338885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:04.340661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:04.459047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c1722-0070-4160-a202-14d4346e42d2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:04.468281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:04.468932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.469185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.469451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.490373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:04.491861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:04.512910 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1245/2506] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:04.605309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b5f10-bfd9-44c9-85e7-70ae42c1c9bd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.605842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b5f10-bfd9-44c9-85e7-70ae42c1c9bd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.605941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b5f10-bfd9-44c9-85e7-70ae42c1c9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.664840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4a0b5f10-bfd9-44c9-85e7-70ae42c1c9bd None None] MFA Rules not processed for user `2fc322d5713f443d8351f65b132e703a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:04.666113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4a0b5f10-bfd9-44c9-85e7-70ae42c1c9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:04.836681 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1262/2507] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:04.847665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:04.849675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:04.951791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ed039-846c-49f0-a468-44db9bcfd609 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:04.959081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63a0c8-b5b8-46ff-9938-023a344f6267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:04.959617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63a0c8-b5b8-46ff-9938-023a344f6267 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.960436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63a0c8-b5b8-46ff-9938-023a344f6267 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.960719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63a0c8-b5b8-46ff-9938-023a344f6267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.977702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63a0c8-b5b8-46ff-9938-023a344f6267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:04.979480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63a0c8-b5b8-46ff-9938-023a344f6267 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:04.991083 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1246/2508] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:05.923990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600076ae-0260-4006-8b10-986e5ad1fa14 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.924552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600076ae-0260-4006-8b10-986e5ad1fa14 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.925187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600076ae-0260-4006-8b10-986e5ad1fa14 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.927283 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1263/2509] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:52:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:05.964289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.964903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.964903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.004923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:06.005922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:06.077427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:06.099524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600076ae-0260-4006-8b10-986e5ad1fa14 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.103320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-600076ae-0260-4006-8b10-986e5ad1fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:06.143984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600076ae-0260-4006-8b10-986e5ad1fa14 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.162991 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1247/2510] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:52:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:52:06.167193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.167907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.168370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.168642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] PATH_INFO: `/v3/users/5aae6d573509428db8d4d5db9d7b005c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.173132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aae6d573509428db8d4d5db9d7b005c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.181718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.195529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.197012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:06.222029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777b443-100d-46b3-bc1c-e8de091bc429 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.246583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.247629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.248121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.248723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.285184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.288142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aae6d573509428db8d4d5db9d7b005c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:06.291188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.317195 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1248/2511] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:52:06.330403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aae6d573509428db8d4d5db9d7b005c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:06.352303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aae6d573509428db8d4d5db9d7b005c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:06.353760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aae6d573509428db8d4d5db9d7b005c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:06.354597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aae6d573509428db8d4d5db9d7b005c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:06.387331 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1264/2512] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:06 2022] DELETE /identity/v3/users/5aae6d573509428db8d4d5db9d7b005c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:07.406110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:07.407978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:07.437114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629709c-82cd-4b99-bd84-a4143be762ae service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:07.585658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:07.586404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:07.588743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:07.589711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] PATH_INFO: `/v3/projects/aca5ff8c1a6143888f70ebdcde126b33` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:07.592215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca5ff8c1a6143888f70ebdcde126b33)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:07.597315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:07.732932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca5ff8c1a6143888f70ebdcde126b33', 'request_id': 'req-9657b708-881e-4065-9e7b-d718912541b2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:07.742122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca5ff8c1a6143888f70ebdcde126b33', 'request_id': 'req-9657b708-881e-4065-9e7b-d718912541b2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:07.766031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca5ff8c1a6143888f70ebdcde126b33', 'request_id': 'req-9657b708-881e-4065-9e7b-d718912541b2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:07.798653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] The token cache is being invalidate because project aca5ff8c1a6143888f70ebdcde126b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:07.799749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:07.802469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:07.806717 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1249/2513] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:07 2022] DELETE /identity/v3/projects/aca5ff8c1a6143888f70ebdcde126b33 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:07.811095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:07.813744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:07.868177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17374c02-edfe-4a83-9294-51e5df5bb4d3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:08.046684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:08.057110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:08.060601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.061606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.062281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] PATH_INFO: `/v3/users/4ad9e32e3c074b82a46dccd767f7ce8d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.065208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad9e32e3c074b82a46dccd767f7ce8d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:08.076933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:08.194261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad9e32e3c074b82a46dccd767f7ce8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.249386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad9e32e3c074b82a46dccd767f7ce8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.263856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:08.267741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:08.288002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657b708-881e-4065-9e7b-d718912541b2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:08.294708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad9e32e3c074b82a46dccd767f7ce8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.295510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad9e32e3c074b82a46dccd767f7ce8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.296420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0f21a092-2b4f-4def-a94f-cee2d45d7ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad9e32e3c074b82a46dccd767f7ce8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.296865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:08.297913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.298441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.299100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] PATH_INFO: `/v3/projects/06f2b7e4e205458b8cf6f19a43b77296` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.301017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f2b7e4e205458b8cf6f19a43b77296)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:08.309582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:08.338035 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1265/2514] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:07 2022] DELETE /identity/v3/users/4ad9e32e3c074b82a46dccd767f7ce8d => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:08.390179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:08.405936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f2b7e4e205458b8cf6f19a43b77296', 'request_id': 'req-c334a0e1-2f20-4972-a685-58c01989b7f1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.422906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f2b7e4e205458b8cf6f19a43b77296', 'request_id': 'req-c334a0e1-2f20-4972-a685-58c01989b7f1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.440389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f2b7e4e205458b8cf6f19a43b77296', 'request_id': 'req-c334a0e1-2f20-4972-a685-58c01989b7f1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.466422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] The token cache is being invalidate because project 06f2b7e4e205458b8cf6f19a43b77296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:08.466975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.467937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c334a0e1-2f20-4972-a685-58c01989b7f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:08.470414 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1250/2515] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:08 2022] DELETE /identity/v3/projects/06f2b7e4e205458b8cf6f19a43b77296 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:08.475521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f92a1-5f38-4909-b803-8e5ba952c989 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.475811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f92a1-5f38-4909-b803-8e5ba952c989 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.476089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f92a1-5f38-4909-b803-8e5ba952c989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.526952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5f4f92a1-5f38-4909-b803-8e5ba952c989 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:08.528570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f4f92a1-5f38-4909-b803-8e5ba952c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:08.696899 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1266/2516] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:08.704760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eba5d-4047-426c-b6d2-841403f2bbb9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.705252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eba5d-4047-426c-b6d2-841403f2bbb9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.705558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eba5d-4047-426c-b6d2-841403f2bbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.747096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-190eba5d-4047-426c-b6d2-841403f2bbb9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:08.748356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-190eba5d-4047-426c-b6d2-841403f2bbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:08.773378 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1251/2517] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:08.774607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f92a1-5f38-4909-b803-8e5ba952c989 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:08.790854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f92a1-5f38-4909-b803-8e5ba952c989 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:08.803566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:08.804216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.804495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.804833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.805429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:08.805753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:08.811027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:08.853767 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1267/2518] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:08.858028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190eba5d-4047-426c-b6d2-841403f2bbb9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:08.869219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190eba5d-4047-426c-b6d2-841403f2bbb9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:08.882509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:08.882896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.883178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.883447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.884406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:08.886445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:08.958267 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1252/2519] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:08 2022] POST /identity/v3/projects => generated 411 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:08.962185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:08.974769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74327c-eae2-4553-8c83-28d434fd02c5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:09.116943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:09.117654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.117943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.118221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.118891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:09.120973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:09.155039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:09.181535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1268/2520] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:08 2022] POST /identity/v3/users => generated 306 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:09.185989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:09.201042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076241f-868b-481b-97a0-671b6d8444fa admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:09.207823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:09.212198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.212660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.213387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.214706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:09.216959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:09.240985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1253/2521] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:09.244735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:09.275107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b29bb-ae9e-436f-a924-edbd235a0470 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:09.289563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00043ab5-4e89-455a-9bbc-fddb7e735b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:09.290224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00043ab5-4e89-455a-9bbc-fddb7e735b32 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.290550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00043ab5-4e89-455a-9bbc-fddb7e735b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.290831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00043ab5-4e89-455a-9bbc-fddb7e735b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.291505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00043ab5-4e89-455a-9bbc-fddb7e735b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:09.312997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00043ab5-4e89-455a-9bbc-fddb7e735b32 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:09.358300 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1269/2522] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:09.362673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:09.377826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d3874-6c19-4692-94be-3abd53b330e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:09.391376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:09.392056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.392498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.392765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] PATH_INFO: `/v3/projects/e566b2c69fec4ec784a13e4034eaef72/users/feb0a86572434db48c987279d67cd4e0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.432327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e566b2c69fec4ec784a13e4034eaef72, user_id=feb0a86572434db48c987279d67cd4e0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:09.435342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:09.482282 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1254/2523] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:09 2022] PUT /identity/v3/projects/e566b2c69fec4ec784a13e4034eaef72/users/feb0a86572434db48c987279d67cd4e0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:09.487768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910eec-aeb5-4038-a694-775860719797 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.489391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910eec-aeb5-4038-a694-775860719797 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.490680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910eec-aeb5-4038-a694-775860719797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.529783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-40910eec-aeb5-4038-a694-775860719797 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:09.535223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-40910eec-aeb5-4038-a694-775860719797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:09.739382 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1270/2524] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:52:10.121692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.123468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:10.143793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e4c61-c207-4f09-82be-03886aa553ca admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.166114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.166793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.167065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.172014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] PATH_INFO: `/v3/projects/4befb6a27ba347ffa709bdf2f65e0a36` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.175766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4befb6a27ba347ffa709bdf2f65e0a36)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.179563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.250110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40910eec-aeb5-4038-a694-775860719797 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.257815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-40910eec-aeb5-4038-a694-775860719797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:10.351241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4befb6a27ba347ffa709bdf2f65e0a36', 'request_id': 'req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:10.371514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4befb6a27ba347ffa709bdf2f65e0a36', 'request_id': 'req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:10.389265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4befb6a27ba347ffa709bdf2f65e0a36', 'request_id': 'req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:10.420218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] The token cache is being invalidate because project 4befb6a27ba347ffa709bdf2f65e0a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:10.420854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:10.421832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b0cdab36-90a7-4d07-9f68-e20b8cdd1265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:10.425848 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1255/2525] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:10 2022] DELETE /identity/v3/projects/4befb6a27ba347ffa709bdf2f65e0a36 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:10.432975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbfe1b-ac37-4eba-b578-8c75ed6b7584 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.433898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbfe1b-ac37-4eba-b578-8c75ed6b7584 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.434334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbfe1b-ac37-4eba-b578-8c75ed6b7584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.534288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40910eec-aeb5-4038-a694-775860719797 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.558188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-17bbfe1b-ac37-4eba-b578-8c75ed6b7584 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:10.560338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17bbfe1b-ac37-4eba-b578-8c75ed6b7584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:10.616906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:10.655036 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1256/2526] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:10.666413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abc4c9-d6b1-44ea-a460-da5e10db1669 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.668683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abc4c9-d6b1-44ea-a460-da5e10db1669 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.670074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abc4c9-d6b1-44ea-a460-da5e10db1669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.742041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f7abc4c9-d6b1-44ea-a460-da5e10db1669 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:10.758831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f7abc4c9-d6b1-44ea-a460-da5e10db1669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:10.774879 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1257/2527] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:10.782132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abc4c9-d6b1-44ea-a460-da5e10db1669 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.801605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abc4c9-d6b1-44ea-a460-da5e10db1669 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.807388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.808588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.809015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.809421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.810336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:10.810801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.814358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.814670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.818759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.823563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.826339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.846469 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1258/2528] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:10.848868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:10.853041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.870671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b471e6-22a5-4239-9e92-40279f3005b2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.888442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.889343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.890799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.891299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.892241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.894471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.938900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.940452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.955371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1259/2529] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:10 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:10.960217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.998463 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1271/2530] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 750 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:11.219867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb4ebe-515f-4d87-8f0f-d19fe1039844 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.232292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.233174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.233699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.234131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.234943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.238927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.297902 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1260/2531] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:10 2022] POST /identity/v3/users => generated 287 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:11.299577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.316026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37768150-9828-4896-83d6-32685f9d62a1 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.335350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.340656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.341015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.341301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.342047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.343748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.377734 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1272/2532] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:11.382392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.395985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927057f9-5455-4eac-a392-da3fce71b1d0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.410805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e7f64-9b96-485d-a08f-33f92e9f8004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.411454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e7f64-9b96-485d-a08f-33f92e9f8004 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.411774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e7f64-9b96-485d-a08f-33f92e9f8004 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.412039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e7f64-9b96-485d-a08f-33f92e9f8004 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.412693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e7f64-9b96-485d-a08f-33f92e9f8004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.419214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e7f64-9b96-485d-a08f-33f92e9f8004 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.448150 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1261/2533] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:11.453254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.481251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19a4e-95f4-416b-b027-de1c13da818e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.497940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.501200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.502565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.505734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] PATH_INFO: `/v3/projects/d53ead9bed9640a88ca01af3a10626c5/users/a38f77df31e648f99763a2f785b6fb4a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.556134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53ead9bed9640a88ca01af3a10626c5, user_id=a38f77df31e648f99763a2f785b6fb4a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.560294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.608222 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1273/2534] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:11 2022] PUT /identity/v3/projects/d53ead9bed9640a88ca01af3a10626c5/users/a38f77df31e648f99763a2f785b6fb4a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:11.622720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf53-8c1c-43e3-8761-a64da5fa1df9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.622962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf53-8c1c-43e3-8761-a64da5fa1df9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.623281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf53-8c1c-43e3-8761-a64da5fa1df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.708556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-df0bdf53-8c1c-43e3-8761-a64da5fa1df9 None None] MFA Rules not processed for user `a38f77df31e648f99763a2f785b6fb4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:11.711760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df0bdf53-8c1c-43e3-8761-a64da5fa1df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:11.909249 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1262/2535] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:11.910871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.923908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:12.114539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5377a2-c265-4652-8a5f-de0c76875e91 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.124360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.125000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.125283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.125599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.126561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.128205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.202440 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1274/2536] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:11 2022] POST /identity/v3/projects => generated 371 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:12.203713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bdf53-8c1c-43e3-8761-a64da5fa1df9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.240598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bdf53-8c1c-43e3-8761-a64da5fa1df9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.261512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:12.406071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.406784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.407072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.407347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.408636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.415241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.471088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.483679 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1263/2537] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:12 2022] POST /identity/v3/users => generated 292 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:12.484183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329a2a-7a80-4c02-812e-6048e647cc08 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.499092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.499827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.500332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.500818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.501823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.503668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.532374 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1275/2538] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:52:12.537699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.559765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab62fa3-76b4-4182-b10a-ab223e6a5b04 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.584770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.586310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.586594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.586939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] PATH_INFO: `/v3/projects/e508b3ed81164348acbc6976874b3152/users/1a75e6fc6b2e4f469d74af2674d26e0e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.620178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e508b3ed81164348acbc6976874b3152, user_id=1a75e6fc6b2e4f469d74af2674d26e0e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.623196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.659758 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1264/2539] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:12 2022] PUT /identity/v3/projects/e508b3ed81164348acbc6976874b3152/users/1a75e6fc6b2e4f469d74af2674d26e0e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:12.659907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.679716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af7b5e-a05b-4a0a-a29b-5be7caa9f417 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.811791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7776336-b715-4ab3-8f63-097d41765e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.812434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7776336-b715-4ab3-8f63-097d41765e87 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.812771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7776336-b715-4ab3-8f63-097d41765e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.813381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7776336-b715-4ab3-8f63-097d41765e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.814135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7776336-b715-4ab3-8f63-097d41765e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.816007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7776336-b715-4ab3-8f63-097d41765e87 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.844427 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1276/2540] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:52:12.850784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.904065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1f450-070c-41aa-90ee-29abddbfafd3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.921813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d0581-f296-4f53-b0ce-83d885dafbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.922864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d0581-f296-4f53-b0ce-83d885dafbe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.923207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d0581-f296-4f53-b0ce-83d885dafbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.923481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d0581-f296-4f53-b0ce-83d885dafbe4 admin admin] PATH_INFO: `/v3/projects/e508b3ed81164348acbc6976874b3152/users/1a75e6fc6b2e4f469d74af2674d26e0e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.926232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d0581-f296-4f53-b0ce-83d885dafbe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e508b3ed81164348acbc6976874b3152, user_id=1a75e6fc6b2e4f469d74af2674d26e0e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.929393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d0581-f296-4f53-b0ce-83d885dafbe4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.986163 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1265/2541] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:12 2022] PUT /identity/v3/projects/e508b3ed81164348acbc6976874b3152/users/1a75e6fc6b2e4f469d74af2674d26e0e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:12.990687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.991047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.991425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.081845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] MFA Rules not processed for user `1a75e6fc6b2e4f469d74af2674d26e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:13.083086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:13.331823 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1277/2542] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:13.341938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec60446-1ff7-4efc-848f-4293d1781d78 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.342275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec60446-1ff7-4efc-848f-4293d1781d78 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.342527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec60446-1ff7-4efc-848f-4293d1781d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.391882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8ec60446-1ff7-4efc-848f-4293d1781d78 None None] MFA Rules not processed for user `a38f77df31e648f99763a2f785b6fb4a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:13.394430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8ec60446-1ff7-4efc-848f-4293d1781d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:13.595876 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1266/2543] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:13.609958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.611773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:13.618307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:13.791717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a950e-445b-4df3-86ba-bd8c8e5e1333 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.813381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10f3bc-1a3a-46f5-a497-c2b35bb88e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.814290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10f3bc-1a3a-46f5-a497-c2b35bb88e39 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.815512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10f3bc-1a3a-46f5-a497-c2b35bb88e39 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.816668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10f3bc-1a3a-46f5-a497-c2b35bb88e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.831802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10f3bc-1a3a-46f5-a497-c2b35bb88e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.833495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10f3bc-1a3a-46f5-a497-c2b35bb88e39 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.846205 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1278/2544] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:14.456464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70b043-338f-4bd6-896c-aebf592a1c6b None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:14.456753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70b043-338f-4bd6-896c-aebf592a1c6b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:14.457024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70b043-338f-4bd6-896c-aebf592a1c6b None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:14.459003 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1267/2545] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:14.477937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e7ca-ce58-4312-8625-38aefd52a31b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:14.478130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e7ca-ce58-4312-8625-38aefd52a31b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:14.478886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e7ca-ce58-4312-8625-38aefd52a31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:14.525917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6d31e7ca-ce58-4312-8625-38aefd52a31b None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:14.527445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6d31e7ca-ce58-4312-8625-38aefd52a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:14.604167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1279/2546] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:52:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:52:15.026475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70b043-338f-4bd6-896c-aebf592a1c6b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.030938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fe70b043-338f-4bd6-896c-aebf592a1c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:15.226792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70b043-338f-4bd6-896c-aebf592a1c6b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:15.235647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:15.236338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.236795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.237399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.266036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:15.269834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:15.296454 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1268/2547] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:18.320530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414ebf6-ba1a-40e9-9784-ad530d959e8f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:18.321854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414ebf6-ba1a-40e9-9784-ad530d959e8f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:18.322340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414ebf6-ba1a-40e9-9784-ad530d959e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:18.429747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6414ebf6-ba1a-40e9-9784-ad530d959e8f None None] MFA Rules not processed for user `535f4800b67b46a3a6fd2d431ca88860`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:18.434720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6414ebf6-ba1a-40e9-9784-ad530d959e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:18.620083 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1280/2548] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:18 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:19.367203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:19.380409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422f4b-83f2-43b8-91cf-be3c8eca302c service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:19.387481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c85760-ebcc-4883-b39e-63d25583a328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:19.388387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c85760-ebcc-4883-b39e-63d25583a328 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.388819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c85760-ebcc-4883-b39e-63d25583a328 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.389238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c85760-ebcc-4883-b39e-63d25583a328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.393821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70c85760-ebcc-4883-b39e-63d25583a328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:19.425890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:19.539235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c85760-ebcc-4883-b39e-63d25583a328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:19.542313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c85760-ebcc-4883-b39e-63d25583a328 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:19.557177 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1269/2549] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:19.619073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414ebf6-ba1a-40e9-9784-ad530d959e8f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:19.654774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414ebf6-ba1a-40e9-9784-ad530d959e8f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:19.670142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:19.671450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.674481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.676130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.711361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:19.713145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:19.759685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1281/2550] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:19 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:20.340665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.341173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.342210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.496369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] MFA Rules not processed for user `5b6c0aef786d4fd7ba8c5bd00a097fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:20.500871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:20.687839 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1270/2551] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:21.131670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.155143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31295c1-ac36-4b89-9b99-6c8de4a7212c service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.173354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c8a7d-d638-4c11-a558-3b1d6cca63b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.176347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8a7d-d638-4c11-a558-3b1d6cca63b7 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.176749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8a7d-d638-4c11-a558-3b1d6cca63b7 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.177636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8a7d-d638-4c11-a558-3b1d6cca63b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.211997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c8a7d-d638-4c11-a558-3b1d6cca63b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.216238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c8a7d-d638-4c11-a558-3b1d6cca63b7 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.241229 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1282/2552] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:21.349727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.351768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:21.378761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85773d2a-b1bd-48ef-b69b-5a5be217f959 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.385300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.386597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.387081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.387863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.389971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:21.564810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.566806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.608974 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1271/2553] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:22.518264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7c180-4d12-40b8-9ed3-e712b17e9433 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:22.518744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7c180-4d12-40b8-9ed3-e712b17e9433 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:22.518824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7c180-4d12-40b8-9ed3-e712b17e9433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:22.570036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a2d7c180-4d12-40b8-9ed3-e712b17e9433 None None] MFA Rules not processed for user `42f7c0fdad9346c58949928300021b07`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:22.571333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a2d7c180-4d12-40b8-9ed3-e712b17e9433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:22.596169 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1283/2554] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:24.685495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:24.687788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:24.736010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:24.762453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa745-7d8f-4993-965a-b822a4d08caf service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:24.768515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:24.769408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.769873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.770291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] PATH_INFO: `/v3/users/2fc322d5713f443d8351f65b132e703a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.772643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc322d5713f443d8351f65b132e703a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:24.775046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:24.834925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc322d5713f443d8351f65b132e703a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:24.835857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc322d5713f443d8351f65b132e703a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:24.868819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc322d5713f443d8351f65b132e703a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:24.869535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc322d5713f443d8351f65b132e703a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:24.895809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1b6d82c-d3f2-4dde-bdf0-d6bb071b521e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc322d5713f443d8351f65b132e703a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:24.953528 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1272/2555] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:24 2022] DELETE /identity/v3/users/2fc322d5713f443d8351f65b132e703a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:25.189727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7c180-4d12-40b8-9ed3-e712b17e9433 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.210285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7c180-4d12-40b8-9ed3-e712b17e9433 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.294726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:25.331828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.332440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.332894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.333174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] PATH_INFO: `/v3/projects/4a80cf8bf5be4795a0be8a65a5b389e5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.334522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a80cf8bf5be4795a0be8a65a5b389e5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.337028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.405593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a80cf8bf5be4795a0be8a65a5b389e5', 'request_id': 'req-c072c499-c41c-4e7a-a4ca-94095effdc5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:25.442759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a80cf8bf5be4795a0be8a65a5b389e5', 'request_id': 'req-c072c499-c41c-4e7a-a4ca-94095effdc5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:25.474953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a80cf8bf5be4795a0be8a65a5b389e5', 'request_id': 'req-c072c499-c41c-4e7a-a4ca-94095effdc5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:25.513591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] The token cache is being invalidate because project 4a80cf8bf5be4795a0be8a65a5b389e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:25.514633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:25.516532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c072c499-c41c-4e7a-a4ca-94095effdc5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:25.521260 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1284/2556] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:25 2022] DELETE /identity/v3/projects/4a80cf8bf5be4795a0be8a65a5b389e5 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:25.527214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98493cb9-5e04-459f-8969-064dca5fcfe0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.528227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98493cb9-5e04-459f-8969-064dca5fcfe0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.528731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98493cb9-5e04-459f-8969-064dca5fcfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.595244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-98493cb9-5e04-459f-8969-064dca5fcfe0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:25.597091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98493cb9-5e04-459f-8969-064dca5fcfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:25.787929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1273/2557] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:25.795825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92728c-e715-4f08-9966-5a7b420adf3e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.796115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92728c-e715-4f08-9966-5a7b420adf3e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.796397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92728c-e715-4f08-9966-5a7b420adf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.850372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bd92728c-e715-4f08-9966-5a7b420adf3e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:25.851667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bd92728c-e715-4f08-9966-5a7b420adf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:25.879062 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1285/2558] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:25.886223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98493cb9-5e04-459f-8969-064dca5fcfe0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.908821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98493cb9-5e04-459f-8969-064dca5fcfe0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.928540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.929968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.930416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.930838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.931666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:25.932129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.935611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.960076 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1274/2559] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:25.964950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92728c-e715-4f08-9966-5a7b420adf3e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.976187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92728c-e715-4f08-9966-5a7b420adf3e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:26.005676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:26.006234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.006517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.006791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.007449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:26.009398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:26.090746 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1286/2560] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:25 2022] POST /identity/v3/projects => generated 361 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:26.094717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:26.116581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073154-920b-4678-a565-63e0608530ea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:26.187307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:26.213185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:26.214531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.215407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.215883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.217313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:26.226333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:26.270729 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1275/2561] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:26 2022] POST /identity/v3/users => generated 281 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:26.274869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:26.297786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8e10c-3554-4987-b422-dcefe71dd914 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:26.303513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c964800-d75c-4968-a57e-e01e7d8d6f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:26.304275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c964800-d75c-4968-a57e-e01e7d8d6f95 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.305263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c964800-d75c-4968-a57e-e01e7d8d6f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.305447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c964800-d75c-4968-a57e-e01e7d8d6f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.306242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c964800-d75c-4968-a57e-e01e7d8d6f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:26.313878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c964800-d75c-4968-a57e-e01e7d8d6f95 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:26.338871 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1287/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:26.343180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:26.357214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485c09e-0e1b-4e58-8bc1-cf68d02584af admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:26.395579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:26.396409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.396729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.397541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.398029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:26.400781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:26.415221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603e9c5-829d-4b1d-b01f-5b65189aec0f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.415507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603e9c5-829d-4b1d-b01f-5b65189aec0f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.416558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603e9c5-829d-4b1d-b01f-5b65189aec0f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.420349 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1288/2563] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:52:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:26.428670 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1276/2564] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:52:26.430882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de48a-8b15-40fe-8f51-3f9d13bc98df None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.431735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de48a-8b15-40fe-8f51-3f9d13bc98df None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.432182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de48a-8b15-40fe-8f51-3f9d13bc98df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.446073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:26.482643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393546e-6348-49a5-b8a6-badff320db97 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:26.490408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-260de48a-8b15-40fe-8f51-3f9d13bc98df None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:26.496231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-260de48a-8b15-40fe-8f51-3f9d13bc98df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:26.499086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d29eb-df01-4a1d-a4ec-55cb857d2c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:26.500953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d29eb-df01-4a1d-a4ec-55cb857d2c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.501231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d29eb-df01-4a1d-a4ec-55cb857d2c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.501530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d29eb-df01-4a1d-a4ec-55cb857d2c25 admin admin] PATH_INFO: `/v3/projects/f0c08745e585417fbeb26f23fed560b5/users/51c36d21a3fa4834bb707d2a1f2ef7ab/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.547360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d29eb-df01-4a1d-a4ec-55cb857d2c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c08745e585417fbeb26f23fed560b5, user_id=51c36d21a3fa4834bb707d2a1f2ef7ab, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:26.560023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d29eb-df01-4a1d-a4ec-55cb857d2c25 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:26.595970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1277/2565] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:26 2022] PUT /identity/v3/projects/f0c08745e585417fbeb26f23fed560b5/users/51c36d21a3fa4834bb707d2a1f2ef7ab/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:26.597932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.598219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.601472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.647293 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1289/2566] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:26.651147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.651534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.657360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.659838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] MFA Rules not processed for user `87bbc00e8087432d985ac8a43e06e54c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:26.661231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:26.736017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] MFA Rules not processed for user `51c36d21a3fa4834bb707d2a1f2ef7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:26.739669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:26.765742 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1278/2567] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:26.766854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:26.771134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:26.817796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:26.900750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311387c-ee23-413b-945c-3113c3731c7d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:26.907295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:26.907937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:26.908213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:26.908660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:26.921666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:26.924905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:26.931308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1290/2568] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:26.939590 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1279/2569] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:27.293140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:27.295782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:27.485431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e11e-76d3-4a3b-9307-fb26611cd36d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:27.509932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8589d4-79ee-411d-9c24-acc40b49e261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:27.511693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8589d4-79ee-411d-9c24-acc40b49e261 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.515092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8589d4-79ee-411d-9c24-acc40b49e261 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.515588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8589d4-79ee-411d-9c24-acc40b49e261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.551110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8589d4-79ee-411d-9c24-acc40b49e261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:27.553376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8589d4-79ee-411d-9c24-acc40b49e261 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:27.583847 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1291/2570] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:27.690372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:27.815971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:27.929838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d48841-69d2-47bc-8dc6-6ebe934e5353 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:27.941328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:27.941786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.942070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.942341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] PATH_INFO: `/v3/users/51c36d21a3fa4834bb707d2a1f2ef7ab` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.943665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c36d21a3fa4834bb707d2a1f2ef7ab)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:27.945986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:28.056019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c36d21a3fa4834bb707d2a1f2ef7ab'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:28.057707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c36d21a3fa4834bb707d2a1f2ef7ab'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:28.096445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c36d21a3fa4834bb707d2a1f2ef7ab'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:28.097759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c36d21a3fa4834bb707d2a1f2ef7ab'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:28.151246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c36d21a3fa4834bb707d2a1f2ef7ab'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:28.187967 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1280/2571] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:27 2022] DELETE /identity/v3/users/51c36d21a3fa4834bb707d2a1f2ef7ab => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:28.189096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0665-6ba1-4c8f-9ba7-a846e622c8e3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.189397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0665-6ba1-4c8f-9ba7-a846e622c8e3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.189768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0665-6ba1-4c8f-9ba7-a846e622c8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.236715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5cef0665-6ba1-4c8f-9ba7-a846e622c8e3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:28.238082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5cef0665-6ba1-4c8f-9ba7-a846e622c8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:28.370231 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1292/2572] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:28.388069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:28.390162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:28.570860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e92d-52eb-4197-b543-c6d36059ff90 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:28.575861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:28.576722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.577829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.578126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.592814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:28.594953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:28.608591 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1281/2573] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:52:28.624267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.625316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.625995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.655153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:28.656349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:28.738379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:28.806227 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1293/2574] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:52:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:28.829042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:28.839140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1a4b7-8f66-47d0-b28f-ed2a9fc3958f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:28.848779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541783b-2390-4f73-8c49-cdabecf290be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:28.851863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541783b-2390-4f73-8c49-cdabecf290be service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.854368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541783b-2390-4f73-8c49-cdabecf290be service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.854809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541783b-2390-4f73-8c49-cdabecf290be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.886045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541783b-2390-4f73-8c49-cdabecf290be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:28.887669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541783b-2390-4f73-8c49-cdabecf290be service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:28.918471 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1282/2575] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:29.552208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:29.554430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:29.582106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c9ae-8e57-4aae-85e7-a655c5534981 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:29.594759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:29.596131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:29.596589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:29.597011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] PATH_INFO: `/v3/projects/f0c08745e585417fbeb26f23fed560b5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:29.598497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c08745e585417fbeb26f23fed560b5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:29.600917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:29.734019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c08745e585417fbeb26f23fed560b5', 'request_id': 'req-60e262ad-d52e-410b-b93a-ca4d2dc42e59'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:29.748526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c08745e585417fbeb26f23fed560b5', 'request_id': 'req-60e262ad-d52e-410b-b93a-ca4d2dc42e59'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:29.785653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c08745e585417fbeb26f23fed560b5', 'request_id': 'req-60e262ad-d52e-410b-b93a-ca4d2dc42e59'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:29.811084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] The token cache is being invalidate because project f0c08745e585417fbeb26f23fed560b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:29.811711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:29.830073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60e262ad-d52e-410b-b93a-ca4d2dc42e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:29.836596 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1294/2576] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:29 2022] DELETE /identity/v3/projects/f0c08745e585417fbeb26f23fed560b5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:29.841134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b9113-570c-4274-8cad-f4d37d60d796 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:29.841724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b9113-570c-4274-8cad-f4d37d60d796 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:29.841876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b9113-570c-4274-8cad-f4d37d60d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:29.882580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e64b9113-570c-4274-8cad-f4d37d60d796 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:29.887142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e64b9113-570c-4274-8cad-f4d37d60d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:29.962564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:30.048027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e2e5e-c51a-42c3-9048-1b1d1ca87f1e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.048384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e2e5e-c51a-42c3-9048-1b1d1ca87f1e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.048578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e2e5e-c51a-42c3-9048-1b1d1ca87f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.058558 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1283/2577] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:30.084723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f15e2e5e-c51a-42c3-9048-1b1d1ca87f1e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:30.086004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f15e2e5e-c51a-42c3-9048-1b1d1ca87f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:30.106190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b9113-570c-4274-8cad-f4d37d60d796 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.107185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1295/2578] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:30.117839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b9113-570c-4274-8cad-f4d37d60d796 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.123208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.124478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.124927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.125378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.126230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:30.126698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.128932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.172030 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1284/2579] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:30.174431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e2e5e-c51a-42c3-9048-1b1d1ca87f1e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.185760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e2e5e-c51a-42c3-9048-1b1d1ca87f1e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.223719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.224362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.224663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.224942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.225678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.227811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.329185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1296/2580] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:30 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:30.332359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.356026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdc9ed-50e4-4b89-8a1e-3326f27981ce admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.535697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.536497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.537274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.537762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.538524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.540724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.616208 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1285/2581] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:30 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:30.620788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.636690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860f4ae-d6c5-4899-9dde-d65b47b0210e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.646199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.647103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.647559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.647992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.649007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.650924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.685427 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1297/2582] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:30.686245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.704283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ce32-b2d9-409c-acf1-f14001f567f2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.713758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663481a5-864a-454e-b040-572704f49608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.720046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663481a5-864a-454e-b040-572704f49608 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.720628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663481a5-864a-454e-b040-572704f49608 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.721051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663481a5-864a-454e-b040-572704f49608 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.726014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663481a5-864a-454e-b040-572704f49608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.729004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663481a5-864a-454e-b040-572704f49608 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.770494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1286/2583] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:52:30.776870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.799239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7834d3-8604-4fb9-9c3f-ff5c1499d7ed admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.805401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.806275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.806689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.807030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] PATH_INFO: `/v3/projects/26d60144477f4a5192d581350d678979/users/6f7681de64c248268da4d78124f59b54/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.843180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d60144477f4a5192d581350d678979, user_id=6f7681de64c248268da4d78124f59b54, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.851598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.905772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1298/2584] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:30 2022] PUT /identity/v3/projects/26d60144477f4a5192d581350d678979/users/6f7681de64c248268da4d78124f59b54/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:30.911344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b40c5f-1a12-457e-88ce-369adca0372e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.911823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b40c5f-1a12-457e-88ce-369adca0372e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.912007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b40c5f-1a12-457e-88ce-369adca0372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.949836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a0b40c5f-1a12-457e-88ce-369adca0372e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:30.951209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a0b40c5f-1a12-457e-88ce-369adca0372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:31.054973 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1287/2585] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:52:31.078306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:31.080256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:31.166774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc974b8-a00d-443b-832f-3d036a182e63 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:31.171226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03270d03-3418-46a5-9d6b-13de0728a1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:31.171849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03270d03-3418-46a5-9d6b-13de0728a1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.172125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03270d03-3418-46a5-9d6b-13de0728a1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.172392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03270d03-3418-46a5-9d6b-13de0728a1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.183819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03270d03-3418-46a5-9d6b-13de0728a1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:31.185574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03270d03-3418-46a5-9d6b-13de0728a1a3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:31.205646 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1299/2586] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:31.856688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.857250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.857412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.915422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] MFA Rules not processed for user `feb0a86572434db48c987279d67cd4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:31.918410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:32.001542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:32.133715 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1288/2587] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:31 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:32.140519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2e65-ed70-4a87-91a7-416947a780a6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.140979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2e65-ed70-4a87-91a7-416947a780a6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.141073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2e65-ed70-4a87-91a7-416947a780a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.149741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:32.182764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-169e2e65-ed70-4a87-91a7-416947a780a6 None None] MFA Rules not processed for user `feb0a86572434db48c987279d67cd4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:32.189293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-169e2e65-ed70-4a87-91a7-416947a780a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:32.214261 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1300/2588] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:32.217934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:32.219443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:32.357541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bda71e-6a9e-4fbd-b897-b87871e07f7e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:32.364503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9adff-dc7e-403c-ab9c-762f603c7c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:32.365336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9adff-dc7e-403c-ab9c-762f603c7c1b service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.365700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9adff-dc7e-403c-ab9c-762f603c7c1b service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.366140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9adff-dc7e-403c-ab9c-762f603c7c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.375232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9adff-dc7e-403c-ab9c-762f603c7c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:32.376987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9adff-dc7e-403c-ab9c-762f603c7c1b service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:32.388371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1289/2589] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:32 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:34.094098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542669ca-03fc-46fb-95c4-119bcc705df0 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.094625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542669ca-03fc-46fb-95c4-119bcc705df0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.094929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542669ca-03fc-46fb-95c4-119bcc705df0 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.097239 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1301/2590] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:52:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:34.133713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08176c-2a14-46b3-b21f-0f41c0e5f0a6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.134778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08176c-2a14-46b3-b21f-0f41c0e5f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.135192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08176c-2a14-46b3-b21f-0f41c0e5f0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.157370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ba08176c-2a14-46b3-b21f-0f41c0e5f0a6 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:34.158723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ba08176c-2a14-46b3-b21f-0f41c0e5f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:34.167969 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1290/2591] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:52:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:34.783596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542669ca-03fc-46fb-95c4-119bcc705df0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:34.786386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-542669ca-03fc-46fb-95c4-119bcc705df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:34.835727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542669ca-03fc-46fb-95c4-119bcc705df0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:34.855316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:34.855976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.856260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.856526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.878965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:34.884151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:34.930810 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1302/2592] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:44.152235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=50 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:44.905391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:46.609407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461aa9-661f-435d-abb4-c206fa2da7c6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:46.609934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461aa9-661f-435d-abb4-c206fa2da7c6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:46.610016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461aa9-661f-435d-abb4-c206fa2da7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:46.693605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-41461aa9-661f-435d-abb4-c206fa2da7c6 None None] MFA Rules not processed for user `6f7681de64c248268da4d78124f59b54`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:46.695661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-41461aa9-661f-435d-abb4-c206fa2da7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:46.845408 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1291/2593] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:46.853403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:46.870736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093065-b19b-4f2a-bb60-4ed168456421 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:46.904690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a717286-4b6b-4c3d-a2e8-b1e3b7c5a59b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:46.915319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717286-4b6b-4c3d-a2e8-b1e3b7c5a59b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:46.915526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717286-4b6b-4c3d-a2e8-b1e3b7c5a59b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:46.915805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717286-4b6b-4c3d-a2e8-b1e3b7c5a59b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:46.941157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a717286-4b6b-4c3d-a2e8-b1e3b7c5a59b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:46.942933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a717286-4b6b-4c3d-a2e8-b1e3b7c5a59b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:46.983796 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1303/2594] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:47.719773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582937b8-a9df-4842-a740-bcb27f083440 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:47.720333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582937b8-a9df-4842-a740-bcb27f083440 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:47.720522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582937b8-a9df-4842-a740-bcb27f083440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:47.744968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-582937b8-a9df-4842-a740-bcb27f083440 None None] MFA Rules not processed for user `f1134231b4a545fdb136a5204cb4f289`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:47.746819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-582937b8-a9df-4842-a740-bcb27f083440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:47.909661 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1292/2595] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:49.820665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb90a8-003a-4853-a063-79d70a0062a2 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:49.822027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb90a8-003a-4853-a063-79d70a0062a2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:49.822503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb90a8-003a-4853-a063-79d70a0062a2 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:49.825262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1304/2596] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:52:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:49.831837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:49.832815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:49.833297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:49.913271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:49.916574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:49.942065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb90a8-003a-4853-a063-79d70a0062a2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:49.944279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fafb90a8-003a-4853-a063-79d70a0062a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:50.016518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb90a8-003a-4853-a063-79d70a0062a2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:50.026429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:50.027079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:50.028353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:50.028632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] PATH_INFO: `/v3/users/5b6c0aef786d4fd7ba8c5bd00a097fe6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:50.033562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6c0aef786d4fd7ba8c5bd00a097fe6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:50.036503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:50.134810 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1293/2597] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:50.140915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6c0aef786d4fd7ba8c5bd00a097fe6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:50.147111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:50.149098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:50.175640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b232c8-2b62-4487-a45d-c16f2afeccbe None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:50.176844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6c0aef786d4fd7ba8c5bd00a097fe6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:50.188476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc96817-2c51-4342-8467-d20a4a631c4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:50.189388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc96817-2c51-4342-8467-d20a4a631c4e service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:50.189860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc96817-2c51-4342-8467-d20a4a631c4e service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:50.190182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc96817-2c51-4342-8467-d20a4a631c4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:50.194325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6c0aef786d4fd7ba8c5bd00a097fe6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:50.194685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6c0aef786d4fd7ba8c5bd00a097fe6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:50.195585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6c0aef786d4fd7ba8c5bd00a097fe6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:50.220728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc96817-2c51-4342-8467-d20a4a631c4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:50.230731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc96817-2c51-4342-8467-d20a4a631c4e service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:50.232211 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1305/2598] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:49 2022] DELETE /identity/v3/users/5b6c0aef786d4fd7ba8c5bd00a097fe6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:52:50.247199 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1294/2599] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:52:50.795100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:50.796962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:51.101159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9c009-1bc9-4d67-a575-f076f2b38718 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:51.166966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:51.167651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:51.167957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:51.168286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] PATH_INFO: `/v3/projects/6589c8be0f944eb7bb726df3d468d2f8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:51.177215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6589c8be0f944eb7bb726df3d468d2f8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:51.187567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:51.315685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6589c8be0f944eb7bb726df3d468d2f8', 'request_id': 'req-764f6241-8124-4204-8600-4a99a3995b0b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:51.333395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6589c8be0f944eb7bb726df3d468d2f8', 'request_id': 'req-764f6241-8124-4204-8600-4a99a3995b0b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:51.344355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:51.349252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6589c8be0f944eb7bb726df3d468d2f8', 'request_id': 'req-764f6241-8124-4204-8600-4a99a3995b0b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:51.390539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] The token cache is being invalidate because project 6589c8be0f944eb7bb726df3d468d2f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:51.390845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:51.391576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-764f6241-8124-4204-8600-4a99a3995b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:51.393534 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1306/2600] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:50 2022] DELETE /identity/v3/projects/6589c8be0f944eb7bb726df3d468d2f8 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:51.407819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277b8a-2663-4366-a3e8-e4aa516b22b2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:51.408730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277b8a-2663-4366-a3e8-e4aa516b22b2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:51.409391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277b8a-2663-4366-a3e8-e4aa516b22b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:51.473097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1a277b8a-2663-4366-a3e8-e4aa516b22b2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:51.476496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1a277b8a-2663-4366-a3e8-e4aa516b22b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:51.530644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:51.689291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1295/2601] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:51.697274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c732f-8ccd-45f5-b7f6-4108057dfbc2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:51.697487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c732f-8ccd-45f5-b7f6-4108057dfbc2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:51.697830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c732f-8ccd-45f5-b7f6-4108057dfbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:51.775712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-990c732f-8ccd-45f5-b7f6-4108057dfbc2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:51.777329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-990c732f-8ccd-45f5-b7f6-4108057dfbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:51.822331 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1307/2602] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:51.827255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277b8a-2663-4366-a3e8-e4aa516b22b2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:51.843445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277b8a-2663-4366-a3e8-e4aa516b22b2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:51.867033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:51.867677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:51.867951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:51.868218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:51.868878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:51.869184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:51.876737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:51.912845 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1296/2603] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:51.919042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c732f-8ccd-45f5-b7f6-4108057dfbc2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:51.930131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c732f-8ccd-45f5-b7f6-4108057dfbc2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:51.956235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:51.957196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:51.957837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:51.958291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:51.959126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:51.966043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:52.064711 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1308/2604] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:51 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:52.066152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:52.082146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4c361-f6d6-49b9-870f-be812cd99479 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:52.221556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:52.223109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:52.223545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:52.225250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:52.226635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:52.234567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:52.236863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:52.239208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:52.275899 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1297/2605] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:52 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:52.280000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:52.301980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211fb94-3175-4f56-99f3-6e315d919a9f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:52.309394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:52.310320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:52.310787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:52.311360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:52.312164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:52.314100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:52.329495 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1298/2606] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:52:52.334175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:52.349931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fbef3-1eda-44b8-a527-f5a14d715f3a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:52.357895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:52.359129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:52.360440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:52.360923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:52.364055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:52.366546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:52.380511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1299/2607] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:52.384719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:52.386847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c6638-36e3-455f-90c5-d48ad13379c7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:52.400821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c0c38-7154-40a8-b1a1-b1fee7260a34 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:52.405114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:52.406092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:52.407240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:52.407240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:52.409699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:52.417060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6ac26-861f-4375-9527-811c791b8d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:52.420141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6ac26-861f-4375-9527-811c791b8d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:52.420616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6ac26-861f-4375-9527-811c791b8d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:52.421100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6ac26-861f-4375-9527-811c791b8d37 admin admin] PATH_INFO: `/v3/projects/5d29b99a6bd9481f9042d9c7a9451c49/users/495973792ca942bca2c6f65d545346e9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:52.448324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6ac26-861f-4375-9527-811c791b8d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d29b99a6bd9481f9042d9c7a9451c49, user_id=495973792ca942bca2c6f65d545346e9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:52.463126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6ac26-861f-4375-9527-811c791b8d37 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:52.499286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1300/2608] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:52 2022] PUT /identity/v3/projects/5d29b99a6bd9481f9042d9c7a9451c49/users/495973792ca942bca2c6f65d545346e9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:52.512915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:52.513407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:52.513838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:52.553576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:52.556278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:52.585113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:52.586981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:52.594266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1309/2609] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:52.664899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:52.711153 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1301/2610] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:52.747010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:52.748843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:52.870611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dabe9-ef53-4b61-9b57-9d3ba2af2639 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:52.884343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:52.968665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ecfdf-39b4-4e1c-88d2-e381de9f77bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:52.969659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ecfdf-39b4-4e1c-88d2-e381de9f77bf service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:52.970214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ecfdf-39b4-4e1c-88d2-e381de9f77bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:52.970703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ecfdf-39b4-4e1c-88d2-e381de9f77bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:52.983103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ecfdf-39b4-4e1c-88d2-e381de9f77bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:52.984947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ecfdf-39b4-4e1c-88d2-e381de9f77bf service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:52.997723 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1310/2611] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:52:54.282515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:54.284232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:54.305785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08379c-6bc3-4f1a-952d-06dd0f43d50f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:54.310970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:54.311728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:54.312101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:54.312466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] PATH_INFO: `/v3/users/6d7f9995a47046bda06e450f2f57aea8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:54.314931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d7f9995a47046bda06e450f2f57aea8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:54.317026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:54.391171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d7f9995a47046bda06e450f2f57aea8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:54.392010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d7f9995a47046bda06e450f2f57aea8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:54.407536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d7f9995a47046bda06e450f2f57aea8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:54.409194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d7f9995a47046bda06e450f2f57aea8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:54.441940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d7f9995a47046bda06e450f2f57aea8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:54.461921 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1302/2612] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:54 2022] DELETE /identity/v3/users/6d7f9995a47046bda06e450f2f57aea8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:54.851649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eae15a-f73b-4892-9d6a-9e0526c6fed2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:54.853070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eae15a-f73b-4892-9d6a-9e0526c6fed2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:54.853567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eae15a-f73b-4892-9d6a-9e0526c6fed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:54.904364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a3eae15a-f73b-4892-9d6a-9e0526c6fed2 None None] MFA Rules not processed for user `6f7681de64c248268da4d78124f59b54`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:54.906342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a3eae15a-f73b-4892-9d6a-9e0526c6fed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:55.005609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1311/2613] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:55.139564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:55.144621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:55.348470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3999c5-43a0-4bfe-9fab-ffbfee8eeb50 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:55.368254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:55.369192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:55.370437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:55.370872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] PATH_INFO: `/v3/projects/8456c63a1d184cfea5d482549bccdabb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:55.373333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8456c63a1d184cfea5d482549bccdabb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:55.376911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:55.459409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8456c63a1d184cfea5d482549bccdabb', 'request_id': 'req-473d30cf-edad-411b-8706-3cbc47c54411'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.467359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8456c63a1d184cfea5d482549bccdabb', 'request_id': 'req-473d30cf-edad-411b-8706-3cbc47c54411'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.483022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8456c63a1d184cfea5d482549bccdabb', 'request_id': 'req-473d30cf-edad-411b-8706-3cbc47c54411'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.486679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:55.502589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] The token cache is being invalidate because project 8456c63a1d184cfea5d482549bccdabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:55.503453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.505802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.512253 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1303/2614] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:55 2022] DELETE /identity/v3/projects/8456c63a1d184cfea5d482549bccdabb => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:55.513390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eae15a-f73b-4892-9d6a-9e0526c6fed2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:55.536034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eae15a-f73b-4892-9d6a-9e0526c6fed2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:55.645301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:55.646267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:55.646694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:55.647126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] PATH_INFO: `/v3/users/87bbc00e8087432d985ac8a43e06e54c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:55.648595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] RBAC: Authorizing `identity:delete_user(user_id=87bbc00e8087432d985ac8a43e06e54c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:55.650731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:55.758508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87bbc00e8087432d985ac8a43e06e54c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.780135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:55.799013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87bbc00e8087432d985ac8a43e06e54c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.813284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87bbc00e8087432d985ac8a43e06e54c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.815280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87bbc00e8087432d985ac8a43e06e54c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.816103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4342889-f9fd-4839-b134-751c4be2b41e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87bbc00e8087432d985ac8a43e06e54c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:55.841779 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1312/2615] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:52:55 2022] DELETE /identity/v3/users/87bbc00e8087432d985ac8a43e06e54c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:56.345221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:56.349190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:56.377061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d30cf-edad-411b-8706-3cbc47c54411 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:56.557124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:56.557890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:56.558166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:56.558437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] PATH_INFO: `/v3/projects/9e7293acae7349e5994d719b758425a1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:56.562740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7293acae7349e5994d719b758425a1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:56.565307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:56.673381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7293acae7349e5994d719b758425a1', 'request_id': 'req-9bb0e650-a413-4f1d-8537-c9e548c594b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:56.700669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7293acae7349e5994d719b758425a1', 'request_id': 'req-9bb0e650-a413-4f1d-8537-c9e548c594b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:56.716397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7293acae7349e5994d719b758425a1', 'request_id': 'req-9bb0e650-a413-4f1d-8537-c9e548c594b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:56.766664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] The token cache is being invalidate because project 9e7293acae7349e5994d719b758425a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:52:56.767165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:56.768299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9bb0e650-a413-4f1d-8537-c9e548c594b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:52:56.774453 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1304/2616] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:52:56 2022] DELETE /identity/v3/projects/9e7293acae7349e5994d719b758425a1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:56.777278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a890a51-cf4f-445b-b1ed-9f1d7de2c7a3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:56.778157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a890a51-cf4f-445b-b1ed-9f1d7de2c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:56.778629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a890a51-cf4f-445b-b1ed-9f1d7de2c7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:56.856646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2a890a51-cf4f-445b-b1ed-9f1d7de2c7a3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:56.857928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2a890a51-cf4f-445b-b1ed-9f1d7de2c7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:56.935396 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1313/2617] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:56.942831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa054-4a16-4907-ae36-a81987654a5d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:56.943125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa054-4a16-4907-ae36-a81987654a5d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:56.943396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa054-4a16-4907-ae36-a81987654a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:56.986133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-21baa054-4a16-4907-ae36-a81987654a5d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:56.987609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21baa054-4a16-4907-ae36-a81987654a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:57.010516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1305/2618] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:57.014752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a890a51-cf4f-445b-b1ed-9f1d7de2c7a3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.029315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a890a51-cf4f-445b-b1ed-9f1d7de2c7a3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.034692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.035311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.035589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.035857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.036716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:57.037131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.038955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.068057 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1314/2619] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:57.072427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21baa054-4a16-4907-ae36-a81987654a5d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.084113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21baa054-4a16-4907-ae36-a81987654a5d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.092721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.094246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.094931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.095605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.097248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.101242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.201282 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1306/2620] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:52:57 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:57.205319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.223219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94952da5-0fb6-4ed6-b730-058f84be1456 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.399039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.400603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.405237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.407215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.408240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.413091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.478042 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1315/2621] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:57 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:57.482653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.496058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee00b-c919-4a30-a6f2-9a066e2c4667 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.500669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.501790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.502163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.502525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.503370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.505113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.521617 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1307/2622] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:57.525926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.548332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a57c9-e7b7-4b1e-aae4-3704d53b1c94 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.555040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fafb50-3685-43b9-9cb8-59d353fb02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.555807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fafb50-3685-43b9-9cb8-59d353fb02bb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.556184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fafb50-3685-43b9-9cb8-59d353fb02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.556620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fafb50-3685-43b9-9cb8-59d353fb02bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.557373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fafb50-3685-43b9-9cb8-59d353fb02bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.559218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fafb50-3685-43b9-9cb8-59d353fb02bb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.608150 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1316/2623] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:52:57.612228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.630554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd893-f102-4a0e-8934-c3fa2ebec0e2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.638232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.639423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.639951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.640549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] PATH_INFO: `/v3/projects/f6b44bc44e7c4a998bfd99e7d11e99fc/users/9c2b746b7c194596808a7b0c736f3b53/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.676056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b44bc44e7c4a998bfd99e7d11e99fc, user_id=9c2b746b7c194596808a7b0c736f3b53, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.680083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.714634 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1308/2624] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:52:57 2022] PUT /identity/v3/projects/f6b44bc44e7c4a998bfd99e7d11e99fc/users/9c2b746b7c194596808a7b0c736f3b53/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:57.725233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a06f8-e7fe-4d78-ba76-52d2423abd12 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.725731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a06f8-e7fe-4d78-ba76-52d2423abd12 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.726007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a06f8-e7fe-4d78-ba76-52d2423abd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.758224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-077a06f8-e7fe-4d78-ba76-52d2423abd12 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:57.760080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-077a06f8-e7fe-4d78-ba76-52d2423abd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:57.804394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:57.836263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1317/2625] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:52:57.906992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.908826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:57.935640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:57.985731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afee94-a526-4f0e-a96f-b58ca6170ded admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:58.014241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8778ac-de26-47b2-8583-af24a322aed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:58.018596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8778ac-de26-47b2-8583-af24a322aed2 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:58.019087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8778ac-de26-47b2-8583-af24a322aed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:58.019564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8778ac-de26-47b2-8583-af24a322aed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:58.051768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8778ac-de26-47b2-8583-af24a322aed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:58.053614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8778ac-de26-47b2-8583-af24a322aed2 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:58.082496 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1309/2626] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:00.066142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:00.066811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:00.066811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:00.099082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] MFA Rules not processed for user `226a87a1f393420b8d19ec2390e9d063`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:00.100640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:00.202246 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1318/2627] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:53:07.157852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58131640-e2a7-4fc4-b139-65fd6e2d8bf0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:07.158770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58131640-e2a7-4fc4-b139-65fd6e2d8bf0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:07.159187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58131640-e2a7-4fc4-b139-65fd6e2d8bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:07.220759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-58131640-e2a7-4fc4-b139-65fd6e2d8bf0 None None] MFA Rules not processed for user `495973792ca942bca2c6f65d545346e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:07.224597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58131640-e2a7-4fc4-b139-65fd6e2d8bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:07.334566 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1310/2628] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:07.730955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:07.732703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:07.925430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b388d8a-6c81-4eb3-8f94-85ea80bbb4e1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:07.949509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbc7db-228b-4414-8893-2494102f341d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:07.950819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbc7db-228b-4414-8893-2494102f341d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:07.951380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbc7db-228b-4414-8893-2494102f341d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:07.954438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbc7db-228b-4414-8893-2494102f341d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:07.976189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbc7db-228b-4414-8893-2494102f341d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:07.979184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbc7db-228b-4414-8893-2494102f341d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:08.002416 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1319/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:11.241187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a4fb-f9c7-4abe-8ec4-8ddf8fc60a89 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:11.246556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a4fb-f9c7-4abe-8ec4-8ddf8fc60a89 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:11.249615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5a4fb-f9c7-4abe-8ec4-8ddf8fc60a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:11.311977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f0f5a4fb-f9c7-4abe-8ec4-8ddf8fc60a89 None None] MFA Rules not processed for user `495973792ca942bca2c6f65d545346e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:11.313580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f0f5a4fb-f9c7-4abe-8ec4-8ddf8fc60a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:11.349279 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1311/2630] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:12.746752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fb838-2316-4473-b8c4-2037682b1d41 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:12.747384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fb838-2316-4473-b8c4-2037682b1d41 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:12.747533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fb838-2316-4473-b8c4-2037682b1d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:12.795705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c24fb838-2316-4473-b8c4-2037682b1d41 None None] MFA Rules not processed for user `9c2b746b7c194596808a7b0c736f3b53`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:12.796961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c24fb838-2316-4473-b8c4-2037682b1d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:12.890224 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1320/2631] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:12.891910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5a4fb-f9c7-4abe-8ec4-8ddf8fc60a89 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:12.908159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5a4fb-f9c7-4abe-8ec4-8ddf8fc60a89 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:12.918629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:12.919362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:12.921627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:12.922244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:12.923349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:12.928462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:12.986811 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1312/2632] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:53:12 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:12.990995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fb838-2316-4473-b8c4-2037682b1d41 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:12.997718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:13.004278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fb838-2316-4473-b8c4-2037682b1d41 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.101952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391874-492b-49bb-b529-81199701a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.102897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391874-492b-49bb-b529-81199701a754 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.103335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391874-492b-49bb-b529-81199701a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.103755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391874-492b-49bb-b529-81199701a754 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.105076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50391874-492b-49bb-b529-81199701a754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.110545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50391874-492b-49bb-b529-81199701a754 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.162139 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1321/2633] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:53:12 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:13.165947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:13.178234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dff59-bc44-423b-9626-51f244fdee58 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.188457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.189345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.190079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.190564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.191732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.193662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.212916 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1313/2634] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:53:13.217242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391874-492b-49bb-b529-81199701a754 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:13.229209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391874-492b-49bb-b529-81199701a754 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.234849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.236244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.236244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.236466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] PATH_INFO: `/v3/projects/c3928c4d218a4173b46e6ca792c337d1/users/eda2d41b016543d98d09539949cc1e72/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.286498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3928c4d218a4173b46e6ca792c337d1, user_id=eda2d41b016543d98d09539949cc1e72, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.296516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.369402 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1322/2635] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:53:13 2022] PUT /identity/v3/projects/c3928c4d218a4173b46e6ca792c337d1/users/eda2d41b016543d98d09539949cc1e72/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:13.370953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:13.387753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239f670-432c-4f47-a12e-65f18563bb56 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.487330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.487975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.488256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.488526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.489291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.491103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.511716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1314/2636] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:53:13.512873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:13.542153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f168e-881b-4f52-ba2f-57d63e1eca62 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.549894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.551376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.551756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.552187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] PATH_INFO: `/v3/projects/c3928c4d218a4173b46e6ca792c337d1/users/eda2d41b016543d98d09539949cc1e72/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.559966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3928c4d218a4173b46e6ca792c337d1, user_id=eda2d41b016543d98d09539949cc1e72, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.562988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.584285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1323/2637] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:53:13 2022] PUT /identity/v3/projects/c3928c4d218a4173b46e6ca792c337d1/users/eda2d41b016543d98d09539949cc1e72/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:53:15.295352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:15.298710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:15.323051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:15.480521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa67ad7-4d53-4df5-a8a9-3b13d795f7c0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:15.575261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692859a0-5ca0-42cc-b0fe-3d73a51cc021 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:15.576074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692859a0-5ca0-42cc-b0fe-3d73a51cc021 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:15.577025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692859a0-5ca0-42cc-b0fe-3d73a51cc021 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:15.577617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692859a0-5ca0-42cc-b0fe-3d73a51cc021 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:15.580262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-692859a0-5ca0-42cc-b0fe-3d73a51cc021 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:15.596677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692859a0-5ca0-42cc-b0fe-3d73a51cc021 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:15.598950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692859a0-5ca0-42cc-b0fe-3d73a51cc021 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:15.624893 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1315/2638] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:15.800415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:15.821768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733157f8-8c31-426b-bd8f-40c44aaca8c6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:15.835177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e97d6-a129-4c2c-a14a-d06c145d3fbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:15.836107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e97d6-a129-4c2c-a14a-d06c145d3fbd service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:15.837022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e97d6-a129-4c2c-a14a-d06c145d3fbd service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:15.837452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e97d6-a129-4c2c-a14a-d06c145d3fbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:15.839764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d5e97d6-a129-4c2c-a14a-d06c145d3fbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:15.870595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e97d6-a129-4c2c-a14a-d06c145d3fbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:15.872609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e97d6-a129-4c2c-a14a-d06c145d3fbd service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:15.906593 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1324/2639] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:25.604663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09bafe-7050-4924-8846-feda5599ee01 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:25.605385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09bafe-7050-4924-8846-feda5599ee01 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:25.605385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09bafe-7050-4924-8846-feda5599ee01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:25.617562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=55 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:25.660348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cf09bafe-7050-4924-8846-feda5599ee01 None None] MFA Rules not processed for user `eda2d41b016543d98d09539949cc1e72`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:25.662803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cf09bafe-7050-4924-8846-feda5599ee01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:25.872048 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1316/2640] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:25.884505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:25.885015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:25.885015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:25.921292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] MFA Rules not processed for user `9c2b746b7c194596808a7b0c736f3b53`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:25.923685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:26.087356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17126dc9-1836-4b88-993d-f4ec91d7a112 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:26.087634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17126dc9-1836-4b88-993d-f4ec91d7a112 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:26.087905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17126dc9-1836-4b88-993d-f4ec91d7a112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:26.142420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-17126dc9-1836-4b88-993d-f4ec91d7a112 None None] MFA Rules not processed for user `42f7c0fdad9346c58949928300021b07`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:26.142420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17126dc9-1836-4b88-993d-f4ec91d7a112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:26.155062 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1325/2641] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:26.340419 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1317/2642] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:53:26.680301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:26.681874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:26.788859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:26.859470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e8591-b8b4-4e44-a328-ec853731402b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:26.877171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9bb98-a14e-4181-b6bf-6f9e8149c398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:26.878326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9bb98-a14e-4181-b6bf-6f9e8149c398 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:26.879877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9bb98-a14e-4181-b6bf-6f9e8149c398 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:26.879877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9bb98-a14e-4181-b6bf-6f9e8149c398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:26.898143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9bb98-a14e-4181-b6bf-6f9e8149c398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:26.899822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9bb98-a14e-4181-b6bf-6f9e8149c398 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:26.922076 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1326/2643] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:27.312647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1868-a3bd-469e-8ee6-434245003e8a None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:27.313170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1868-a3bd-469e-8ee6-434245003e8a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:27.313170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1868-a3bd-469e-8ee6-434245003e8a None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:27.315525 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1318/2644] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 17 20:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:27.322329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:27.322768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:27.323190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:27.362137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:27.363792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:27.472749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1327/2645] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:53:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:27.666070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f1868-a3bd-469e-8ee6-434245003e8a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:27.703409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f1868-a3bd-469e-8ee6-434245003e8a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:27.712856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c6548-1f99-41e4-b846-1f19b12402e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:27.713519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c6548-1f99-41e4-b846-1f19b12402e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:27.714068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c6548-1f99-41e4-b846-1f19b12402e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:27.714342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c6548-1f99-41e4-b846-1f19b12402e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:27.733904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c6548-1f99-41e4-b846-1f19b12402e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:27.735577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c6548-1f99-41e4-b846-1f19b12402e3 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:27.755171 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1319/2646] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:31.598585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:31.600139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:31.816530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438e317-aa3e-4d58-a22a-6664cf5d876a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:31.851991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:31.861190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:31.868040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:31.869149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] PATH_INFO: `/v3/users/a38f77df31e648f99763a2f785b6fb4a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:31.878236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38f77df31e648f99763a2f785b6fb4a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:31.880633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:32.100708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38f77df31e648f99763a2f785b6fb4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:32.132429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38f77df31e648f99763a2f785b6fb4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:32.148180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38f77df31e648f99763a2f785b6fb4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:32.150387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38f77df31e648f99763a2f785b6fb4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:32.151470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38f77df31e648f99763a2f785b6fb4a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:32.168295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:32.181891 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1328/2647] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:53:31 2022] DELETE /identity/v3/users/a38f77df31e648f99763a2f785b6fb4a => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:32.187077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:32.187666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:32.188259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:32.295752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:32.295752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:32.585115 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1320/2648] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:53:32.596161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:32.597689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:32.763650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e08c7-a8f8-4877-83d8-7925d0e460be admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:32.775275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:32.776054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:32.776372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:32.776613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:32.792282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:32.794121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:32.819942 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1329/2649] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:33.603199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:33.606830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:33.655786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c4901-30a3-4e97-8cdc-43846755f1ed None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:33.663735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:33.665028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:33.665770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:33.666315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] PATH_INFO: `/v3/projects/d53ead9bed9640a88ca01af3a10626c5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:33.668596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53ead9bed9640a88ca01af3a10626c5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:33.673896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:33.722965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:33.809752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53ead9bed9640a88ca01af3a10626c5', 'request_id': 'req-b9ae0db2-74b6-4523-927b-d075480866c3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:33.821322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53ead9bed9640a88ca01af3a10626c5', 'request_id': 'req-b9ae0db2-74b6-4523-927b-d075480866c3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:33.846042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53ead9bed9640a88ca01af3a10626c5', 'request_id': 'req-b9ae0db2-74b6-4523-927b-d075480866c3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:33.866901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] The token cache is being invalidate because project d53ead9bed9640a88ca01af3a10626c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:53:33.867260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:33.871294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:33.873041 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1321/2650] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:53:33 2022] DELETE /identity/v3/projects/d53ead9bed9640a88ca01af3a10626c5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:33.880930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:33.903928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dea67-25bf-4d55-b6a3-2bfe4bd9b984 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:34.006067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:34.006938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:34.007208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:34.007508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] PATH_INFO: `/v3/users/1a75e6fc6b2e4f469d74af2674d26e0e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:34.009429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a75e6fc6b2e4f469d74af2674d26e0e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:34.016489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:34.122426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a75e6fc6b2e4f469d74af2674d26e0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:34.139662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a75e6fc6b2e4f469d74af2674d26e0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:34.157908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a75e6fc6b2e4f469d74af2674d26e0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:34.158516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a75e6fc6b2e4f469d74af2674d26e0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:34.159096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-492f17cf-5f06-441e-9512-9388ff3912d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a75e6fc6b2e4f469d74af2674d26e0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:34.194747 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1330/2651] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:53:33 2022] DELETE /identity/v3/users/1a75e6fc6b2e4f469d74af2674d26e0e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:35.782717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:35.785827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:35.814495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae0db2-74b6-4523-927b-d075480866c3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:35.945768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:35.947235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:35.948114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:35.948804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] PATH_INFO: `/v3/projects/e508b3ed81164348acbc6976874b3152` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:35.950647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e508b3ed81164348acbc6976874b3152)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:35.953736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:36.083368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e508b3ed81164348acbc6976874b3152', 'request_id': 'req-bac86544-ea90-433e-a283-004f3140187d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:36.093808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e508b3ed81164348acbc6976874b3152', 'request_id': 'req-bac86544-ea90-433e-a283-004f3140187d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:36.112052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e508b3ed81164348acbc6976874b3152', 'request_id': 'req-bac86544-ea90-433e-a283-004f3140187d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:36.140808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] The token cache is being invalidate because project e508b3ed81164348acbc6976874b3152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:53:36.141188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:36.142072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bac86544-ea90-433e-a283-004f3140187d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:36.144325 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1322/2652] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:53:35 2022] DELETE /identity/v3/projects/e508b3ed81164348acbc6976874b3152 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:53:36.152764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5de1-6836-469b-9f5f-60c1e2cb951f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.153663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5de1-6836-469b-9f5f-60c1e2cb951f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.154121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5de1-6836-469b-9f5f-60c1e2cb951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.188777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-64af5de1-6836-469b-9f5f-60c1e2cb951f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:36.190355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-64af5de1-6836-469b-9f5f-60c1e2cb951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:36.224549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:36.379461 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1331/2653] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:36.388687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e60f8-22f7-41e1-a226-f5474624b8cd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.388910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e60f8-22f7-41e1-a226-f5474624b8cd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.389195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e60f8-22f7-41e1-a226-f5474624b8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.439179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a73e60f8-22f7-41e1-a226-f5474624b8cd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:36.440843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a73e60f8-22f7-41e1-a226-f5474624b8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:36.455275 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1323/2654] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:36.460058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af5de1-6836-469b-9f5f-60c1e2cb951f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:36.477532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af5de1-6836-469b-9f5f-60c1e2cb951f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:36.484946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:36.486021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.486503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.487116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.488022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:36.488535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:36.491070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:36.516185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e60f8-22f7-41e1-a226-f5474624b8cd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:36.523102 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1332/2655] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:36.538032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e60f8-22f7-41e1-a226-f5474624b8cd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:36.548543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:36.549437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.550370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.550850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.554700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:36.557186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:36.616776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:36.621819 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1324/2656] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:53:36 2022] POST /identity/v3/projects => generated 397 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:53:36.632307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed5c8b-2f5c-421e-8419-fa8d6e7f353b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:36.737629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:36.738557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.739038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.739620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.740469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:36.744158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:36.781030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1333/2657] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:53:36 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:36.781831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:36.788577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:36.795824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f4f6-9d73-4956-b437-8be93e3cbe0e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:36.811426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:36.812096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.812464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.812760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.813548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:36.816674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:36.829681 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1325/2658] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:53:36.834414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:36.848590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69a46b-b71f-4405-aa97-09dcb1817ee8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:36.854988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f060f6a-9b19-4ba4-af9b-d005c7698704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:36.856077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060f6a-9b19-4ba4-af9b-d005c7698704 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.856434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060f6a-9b19-4ba4-af9b-d005c7698704 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.856761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060f6a-9b19-4ba4-af9b-d005c7698704 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.857883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f060f6a-9b19-4ba4-af9b-d005c7698704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:36.859678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f060f6a-9b19-4ba4-af9b-d005c7698704 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:36.880529 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1334/2659] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:53:36.880731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:36.897181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95215bee-6b76-43bf-b8cb-db8b5ba02dc5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:36.910929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:36.911709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.912065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.912416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] PATH_INFO: `/v3/projects/6b843f5d09c94cf3916f65e815d6d7a1/users/aaa9e5c7b7dd4a74935de34327145697/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.935077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b843f5d09c94cf3916f65e815d6d7a1, user_id=aaa9e5c7b7dd4a74935de34327145697, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:36.938737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:36.963723 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1326/2660] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:53:36 2022] PUT /identity/v3/projects/6b843f5d09c94cf3916f65e815d6d7a1/users/aaa9e5c7b7dd4a74935de34327145697/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:36.970122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c9deb-a397-43b2-a70c-0f45ee353e67 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:36.970430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c9deb-a397-43b2-a70c-0f45ee353e67 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:36.970820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c9deb-a397-43b2-a70c-0f45ee353e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:36.999257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7a3c9deb-a397-43b2-a70c-0f45ee353e67 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:37.001164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7a3c9deb-a397-43b2-a70c-0f45ee353e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:37.105743 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1335/2661] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:37.124335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:37.126656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:37.233141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd94151-465d-4aaa-9419-bdee530d217d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:37.237151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107e3a7-7011-48d7-b436-e14b6fec7454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:37.237813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e3a7-7011-48d7-b436-e14b6fec7454 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:37.238232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e3a7-7011-48d7-b436-e14b6fec7454 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:37.238463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107e3a7-7011-48d7-b436-e14b6fec7454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:37.255485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e3a7-7011-48d7-b436-e14b6fec7454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:37.257002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107e3a7-7011-48d7-b436-e14b6fec7454 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:37.270732 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1327/2662] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:47.091827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:47.265392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:49.036776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:49.037298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:49.037566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:49.083013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] MFA Rules not processed for user `aaa9e5c7b7dd4a74935de34327145697`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:49.084718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:49.208526 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1336/2663] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:49.219118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a96c2-1588-4d4e-9743-b0634bdf468e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:49.219670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a96c2-1588-4d4e-9743-b0634bdf468e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:49.219670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a96c2-1588-4d4e-9743-b0634bdf468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:49.296598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f86a96c2-1588-4d4e-9743-b0634bdf468e None None] MFA Rules not processed for user `aaa9e5c7b7dd4a74935de34327145697`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:49.298308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f86a96c2-1588-4d4e-9743-b0634bdf468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:49.317742 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1328/2664] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:49.325911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:49.327513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:49.450197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750e9bf-64f9-43a1-a391-a4ac2dfd964f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:49.456247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:49.456988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:49.457332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:49.457712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:49.467101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:49.469091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:49.480071 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1337/2665] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:50.116866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4ec0-d007-4f9b-bc84-61c1b209a909 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:50.117149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4ec0-d007-4f9b-bc84-61c1b209a909 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:50.117423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a4ec0-d007-4f9b-bc84-61c1b209a909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:50.140954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-392a4ec0-d007-4f9b-bc84-61c1b209a909 None None] MFA Rules not processed for user `f1134231b4a545fdb136a5204cb4f289`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:50.142335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-392a4ec0-d007-4f9b-bc84-61c1b209a909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:50.228986 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1329/2666] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:53:52.359724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:52.361888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:52.404577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb6d2-4ea3-4935-a9aa-80d4122e514c service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:52.440823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:52.446363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:52.447902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:52.449791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] PATH_INFO: `/v3/users/feb0a86572434db48c987279d67cd4e0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:52.453260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb0a86572434db48c987279d67cd4e0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:52.455290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:52.548538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb0a86572434db48c987279d67cd4e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:52.569622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb0a86572434db48c987279d67cd4e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:52.574617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:52.580901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb0a86572434db48c987279d67cd4e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:52.581077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb0a86572434db48c987279d67cd4e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:52.582064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9a054aec-1d2f-4ec7-98da-d2b8ef07f164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb0a86572434db48c987279d67cd4e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:52.603191 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1338/2667] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:53:52 2022] DELETE /identity/v3/users/feb0a86572434db48c987279d67cd4e0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:53.534287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4ec0-d007-4f9b-bc84-61c1b209a909 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:53.546374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a4ec0-d007-4f9b-bc84-61c1b209a909 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:53.696320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:53.700132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:53.701167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:53.701704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] PATH_INFO: `/v3/projects/e566b2c69fec4ec784a13e4034eaef72` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:53.706916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e566b2c69fec4ec784a13e4034eaef72)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:53.713665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:53.810809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e566b2c69fec4ec784a13e4034eaef72', 'request_id': 'req-556e00c0-18b3-4872-bf42-e57cff59a94e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:53.818768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e566b2c69fec4ec784a13e4034eaef72', 'request_id': 'req-556e00c0-18b3-4872-bf42-e57cff59a94e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:53.830543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e566b2c69fec4ec784a13e4034eaef72', 'request_id': 'req-556e00c0-18b3-4872-bf42-e57cff59a94e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:53.846229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] The token cache is being invalidate because project e566b2c69fec4ec784a13e4034eaef72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:53:53.847164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:53.848605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-556e00c0-18b3-4872-bf42-e57cff59a94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:53:53.851556 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1330/2668] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:53:53 2022] DELETE /identity/v3/projects/e566b2c69fec4ec784a13e4034eaef72 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:53.856558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812c71b-d2c8-4d63-98ab-f87b64d19b32 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:53.857167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812c71b-d2c8-4d63-98ab-f87b64d19b32 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:53.857167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812c71b-d2c8-4d63-98ab-f87b64d19b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:53.886059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d812c71b-d2c8-4d63-98ab-f87b64d19b32 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:53.887454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d812c71b-d2c8-4d63-98ab-f87b64d19b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:53.968926 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1339/2669] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:53.970791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91bbd8-9411-4a07-8887-5347ef736e79 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:53.971080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91bbd8-9411-4a07-8887-5347ef736e79 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:53.971356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91bbd8-9411-4a07-8887-5347ef736e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.000940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-df91bbd8-9411-4a07-8887-5347ef736e79 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:54.003272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df91bbd8-9411-4a07-8887-5347ef736e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:54.012524 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1331/2670] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:54.016497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812c71b-d2c8-4d63-98ab-f87b64d19b32 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:54.031060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812c71b-d2c8-4d63-98ab-f87b64d19b32 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:54.037837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:54.038542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.039018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.039648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.040385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:54.040788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:54.042939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:54.086641 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1340/2671] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:54.088174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91bbd8-9411-4a07-8887-5347ef736e79 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:54.106101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:54.109663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:54.129914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91bbd8-9411-4a07-8887-5347ef736e79 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:54.174884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:54.176546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.176932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.177305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.178228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:54.180477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:54.272883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:54.297772 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1332/2672] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:53:54 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:54.299560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:54.315118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ced0-2a11-4f41-a612-13385281500e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:54.544289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:54.545132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.545406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.545699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.550823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:54.558531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d1584-5556-4a78-9c71-8ec88527d37c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:54.573266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:54.578458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:54.579334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.581027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.581603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.582462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:54.584874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:54.650506 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1333/2673] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:53:54 2022] POST /identity/v3/users => generated 290 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:54.655548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:54.683428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049e644-420c-4bcf-8529-714412a4394b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:54.703508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:54.705860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.706252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.706602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.714452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:54.722027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:54.742434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:54.745192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:54.752509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1334/2674] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:53:54.754840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:54.780165 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1341/2675] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:53:54.785519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fba3f1-6e53-4522-b53d-da0ebb960b8c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:54.803335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572adbd-0f65-436d-9496-aeee52c5891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:54.804162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572adbd-0f65-436d-9496-aeee52c5891b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.804611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572adbd-0f65-436d-9496-aeee52c5891b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.804974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572adbd-0f65-436d-9496-aeee52c5891b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.805740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572adbd-0f65-436d-9496-aeee52c5891b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:54.807544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572adbd-0f65-436d-9496-aeee52c5891b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:54.838944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:54.848820 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1335/2676] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:53:54.857723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b610c6-82f1-4c6d-977b-7ffd850496e3 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:54.874313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:54.875333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.875793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.876243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] PATH_INFO: `/v3/projects/db3f74611f1840c282454eb53c4063d5/users/79370e0ba5674f9ea8c09ad7aa1828fe/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.894006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3f74611f1840c282454eb53c4063d5, user_id=79370e0ba5674f9ea8c09ad7aa1828fe, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:54.899447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:54.930152 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1342/2677] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:53:54 2022] PUT /identity/v3/projects/db3f74611f1840c282454eb53c4063d5/users/79370e0ba5674f9ea8c09ad7aa1828fe/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:53:54.940487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb85ee5-fa6a-470f-a5ee-87776d16d4ec None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:54.941012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb85ee5-fa6a-470f-a5ee-87776d16d4ec None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:54.941430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb85ee5-fa6a-470f-a5ee-87776d16d4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:54.989518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fdb85ee5-fa6a-470f-a5ee-87776d16d4ec None None] MFA Rules not processed for user `79370e0ba5674f9ea8c09ad7aa1828fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:54.990639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fdb85ee5-fa6a-470f-a5ee-87776d16d4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:55.103766 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1336/2678] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:53:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:55.117731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:55.119918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:55.250319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14400cc-2250-4f43-8b1b-2fda76402839 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:55.266993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:55.267638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:55.267911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:55.268183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:55.294455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:55.295961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:55.310225 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1343/2679] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:53:55.563062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd971-685f-44e7-8fe4-002740409f82 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:55.563956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd971-685f-44e7-8fe4-002740409f82 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:55.564224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bd971-685f-44e7-8fe4-002740409f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:55.584755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e84bd971-685f-44e7-8fe4-002740409f82 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:55.586045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e84bd971-685f-44e7-8fe4-002740409f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:55.647555 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1337/2680] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:53:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:55.724255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:55.725884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:55.804169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d50f7-e3bf-4ec9-a123-6682ffb700a4 service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:55.809373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:55.810087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:55.810426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:55.810892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:55.822689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:55.824309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:55.835578 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1344/2681] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:53:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:53:56.513137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe999f-607b-4f1b-b368-474ddea2a731 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:56.513544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe999f-607b-4f1b-b368-474ddea2a731 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:56.514020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe999f-607b-4f1b-b368-474ddea2a731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:56.525040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:56.540285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-29fe999f-607b-4f1b-b368-474ddea2a731 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:56.542918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-29fe999f-607b-4f1b-b368-474ddea2a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:56.675420 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1338/2682] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:53:56.696293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:56.698020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:56.713212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:56.777344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b252b0-4559-44a2-9a61-6ba6b8fe58a9 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:56.783346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:56.783928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:56.784209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:56.784526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:56.801346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:56.802882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:56.814943 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1345/2683] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:54:03.595758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2d5ba-1534-4e55-9efd-13e8f69488eb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:03.596349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2d5ba-1534-4e55-9efd-13e8f69488eb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:03.596349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2d5ba-1534-4e55-9efd-13e8f69488eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:03.641555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9ac2d5ba-1534-4e55-9efd-13e8f69488eb None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:03.643134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9ac2d5ba-1534-4e55-9efd-13e8f69488eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:03.652302 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1339/2684] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 20:54:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:03.670739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:03.690564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e26e6-83fb-4ce2-9c1a-bbe251818915 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:03.691800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f5c51-2cef-4452-8600-bceec548862b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:03.692619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5c51-2cef-4452-8600-bceec548862b service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:03.693065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5c51-2cef-4452-8600-bceec548862b service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:03.693526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f5c51-2cef-4452-8600-bceec548862b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:03.707690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f5c51-2cef-4452-8600-bceec548862b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:03.709581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f5c51-2cef-4452-8600-bceec548862b service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:03.729720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1346/2685] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:54:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:54:03.925009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2d5ba-1534-4e55-9efd-13e8f69488eb None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:04.024878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2d5ba-1534-4e55-9efd-13e8f69488eb None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:04.030886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:04.031639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:04.032015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:04.032378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] PATH_INFO: `/v3/users/79370e0ba5674f9ea8c09ad7aa1828fe` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:04.037387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79370e0ba5674f9ea8c09ad7aa1828fe)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:04.039493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:04.150509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79370e0ba5674f9ea8c09ad7aa1828fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:04.151794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79370e0ba5674f9ea8c09ad7aa1828fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:04.175508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79370e0ba5674f9ea8c09ad7aa1828fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:04.176279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79370e0ba5674f9ea8c09ad7aa1828fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:04.192574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79370e0ba5674f9ea8c09ad7aa1828fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:04.210543 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1340/2686] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:54:03 2022] DELETE /identity/v3/users/79370e0ba5674f9ea8c09ad7aa1828fe => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:04.216067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:04.216439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:04.217559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:04.251560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:04.254540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:04.329990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1347/2687] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:54:04.347779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:04.349211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:04.367884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:04.436121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2c78f-37fb-4a5d-a528-a3a2ba48113b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:04.440353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4153cc8-1c6b-4af6-99c5-4142787ef401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:04.440965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4153cc8-1c6b-4af6-99c5-4142787ef401 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:04.441562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4153cc8-1c6b-4af6-99c5-4142787ef401 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:04.443586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4153cc8-1c6b-4af6-99c5-4142787ef401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:04.453390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4153cc8-1c6b-4af6-99c5-4142787ef401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:04.454878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4153cc8-1c6b-4af6-99c5-4142787ef401 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:04.466407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1341/2688] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:54:05.901322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:05.902803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:05.914661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e9450-fb89-449f-831a-3a7e97245ca8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:05.919493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:05.920135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:05.920413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:05.920681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] PATH_INFO: `/v3/projects/db3f74611f1840c282454eb53c4063d5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:05.925501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3f74611f1840c282454eb53c4063d5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:05.925501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:05.989975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3f74611f1840c282454eb53c4063d5', 'request_id': 'req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:06.003918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3f74611f1840c282454eb53c4063d5', 'request_id': 'req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:06.012257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3f74611f1840c282454eb53c4063d5', 'request_id': 'req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:06.029284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] The token cache is being invalidate because project db3f74611f1840c282454eb53c4063d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:54:06.029709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:06.030436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb31939b-1bc3-4e4d-bc4e-f70b35f012cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:06.036186 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1348/2689] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:54:05 2022] DELETE /identity/v3/projects/db3f74611f1840c282454eb53c4063d5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:06.037355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf97962-1a8f-4be9-be58-a90631e8d343 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.037910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf97962-1a8f-4be9-be58-a90631e8d343 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.037910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf97962-1a8f-4be9-be58-a90631e8d343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.069482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-edf97962-1a8f-4be9-be58-a90631e8d343 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:06.070919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-edf97962-1a8f-4be9-be58-a90631e8d343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:06.140331 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1342/2690] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:54:06.148822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e51f4-920d-4ed8-8f33-cee03138864c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.149801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e51f4-920d-4ed8-8f33-cee03138864c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.149801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e51f4-920d-4ed8-8f33-cee03138864c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.171648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-312e51f4-920d-4ed8-8f33-cee03138864c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:06.172976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-312e51f4-920d-4ed8-8f33-cee03138864c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:06.181692 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1349/2691] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:54:06.187039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf97962-1a8f-4be9-be58-a90631e8d343 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:06.201620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf97962-1a8f-4be9-be58-a90631e8d343 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:06.207829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:06.208598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.209251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.209687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.210930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:06.211318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:06.213378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:06.229011 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1343/2692] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:54:06.234303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e51f4-920d-4ed8-8f33-cee03138864c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:06.245830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e51f4-920d-4ed8-8f33-cee03138864c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:06.251630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:06.252285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.252568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.252843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.253513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:06.255522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:06.280756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:06.291797 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1350/2693] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:54:06 2022] POST /identity/v3/projects => generated 397 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:54:06.295658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:06.307645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cac0ed-a22d-4076-8346-a7ec5204426c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:06.385582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:06.386372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.386630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.387425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.388224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:06.390728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:06.421310 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1344/2694] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:54:06 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:54:06.425696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:06.439263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6fd5a-f2d5-4b03-b8da-d90ff2231ee6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:06.447687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:06.448376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.448664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.448952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.449657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:06.451332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:06.462598 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1351/2695] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:54:06.466742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:06.477589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2a4be-fe5b-4830-835e-2859dac71b3e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:06.483826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20b112-c876-491e-89b8-d02889319778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:06.484451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20b112-c876-491e-89b8-d02889319778 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.484734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20b112-c876-491e-89b8-d02889319778 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.485002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20b112-c876-491e-89b8-d02889319778 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.485688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20b112-c876-491e-89b8-d02889319778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:06.488436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20b112-c876-491e-89b8-d02889319778 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:06.504202 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1345/2696] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:54:06.509239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:06.538157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f43f5f-6909-4891-8c06-b14aae6746da admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:06.550947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:06.551989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.552463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.553133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] PATH_INFO: `/v3/projects/343bbd535db44f4a97ea0db400d0b30e/users/4b89ffdc23fa44baa5bdae8f98751f4c/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.581038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343bbd535db44f4a97ea0db400d0b30e, user_id=4b89ffdc23fa44baa5bdae8f98751f4c, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:06.583809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:06.605984 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1352/2697] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:54:06 2022] PUT /identity/v3/projects/343bbd535db44f4a97ea0db400d0b30e/users/4b89ffdc23fa44baa5bdae8f98751f4c/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:06.617973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa52d-9f04-4538-a3bd-0e2036bd1b65 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.618562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa52d-9f04-4538-a3bd-0e2036bd1b65 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.619199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa52d-9f04-4538-a3bd-0e2036bd1b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.652468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-404fa52d-9f04-4538-a3bd-0e2036bd1b65 None None] MFA Rules not processed for user `4b89ffdc23fa44baa5bdae8f98751f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:06.654428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-404fa52d-9f04-4538-a3bd-0e2036bd1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:06.658568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:06.732351 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1346/2698] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:06.746302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:06.748154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:06.829987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925de42-d3c2-4f46-afd9-2f27e040c384 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:06.835760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c179b8-ae7b-46f8-820a-5fcdbc54812b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:06.836484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c179b8-ae7b-46f8-820a-5fcdbc54812b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:06.836969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c179b8-ae7b-46f8-820a-5fcdbc54812b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:06.838324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c179b8-ae7b-46f8-820a-5fcdbc54812b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:06.847705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c179b8-ae7b-46f8-820a-5fcdbc54812b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:06.849391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c179b8-ae7b-46f8-820a-5fcdbc54812b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:06.859610 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1353/2699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:54:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:54:07.379781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fa52d-9f04-4538-a3bd-0e2036bd1b65 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:07.461203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fa52d-9f04-4538-a3bd-0e2036bd1b65 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:07.466505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:07.467153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:07.467430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:07.467698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] PATH_INFO: `/v3/users/4b89ffdc23fa44baa5bdae8f98751f4c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:07.469173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b89ffdc23fa44baa5bdae8f98751f4c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:07.470849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:07.535476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b89ffdc23fa44baa5bdae8f98751f4c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:07.544611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b89ffdc23fa44baa5bdae8f98751f4c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:07.557624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b89ffdc23fa44baa5bdae8f98751f4c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:07.558172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b89ffdc23fa44baa5bdae8f98751f4c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:07.575119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b89ffdc23fa44baa5bdae8f98751f4c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:07.589166 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1347/2700] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:54:07 2022] DELETE /identity/v3/users/4b89ffdc23fa44baa5bdae8f98751f4c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:07.593777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:07.594288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:07.594895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:07.642695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:07.644379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:07.718950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1354/2701] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:54:07.728529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:07.730083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:07.758520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:07.790238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e7992-2460-4e2a-9c32-f825834d5c78 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:07.793994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f26eea-e285-4946-8a3b-fe287e72f73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:07.794592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f26eea-e285-4946-8a3b-fe287e72f73b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:07.794862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f26eea-e285-4946-8a3b-fe287e72f73b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:07.795124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f26eea-e285-4946-8a3b-fe287e72f73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:07.804173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f26eea-e285-4946-8a3b-fe287e72f73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:07.805636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f26eea-e285-4946-8a3b-fe287e72f73b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:07.815156 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1348/2702] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:54:08.152584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:08.154194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:08.166785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef62cf-4bf4-431b-af7d-2b8c9d55d5e7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:08.172625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:08.174296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.174728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.175141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] PATH_INFO: `/v3/projects/343bbd535db44f4a97ea0db400d0b30e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.176477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=343bbd535db44f4a97ea0db400d0b30e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:08.178622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:08.258156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:08.264389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343bbd535db44f4a97ea0db400d0b30e', 'request_id': 'req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:08.278869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343bbd535db44f4a97ea0db400d0b30e', 'request_id': 'req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:08.285945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343bbd535db44f4a97ea0db400d0b30e', 'request_id': 'req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:08.297911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] The token cache is being invalidate because project 343bbd535db44f4a97ea0db400d0b30e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:54:08.298346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:08.299101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb2b9fb1-a95c-4fc8-80a7-527e6b164e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:08.300638 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1355/2703] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:54:08 2022] DELETE /identity/v3/projects/343bbd535db44f4a97ea0db400d0b30e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:08.307643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb5678-a61c-4096-a603-10a68a5811af None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.308186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb5678-a61c-4096-a603-10a68a5811af None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.308516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb5678-a61c-4096-a603-10a68a5811af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.330470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9bfb5678-a61c-4096-a603-10a68a5811af None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:08.331744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9bfb5678-a61c-4096-a603-10a68a5811af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:08.413743 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1349/2704] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:08.415016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b00c72-4b40-41ca-b38b-3907636b85ee None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.415314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b00c72-4b40-41ca-b38b-3907636b85ee None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.415582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b00c72-4b40-41ca-b38b-3907636b85ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.454766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-81b00c72-4b40-41ca-b38b-3907636b85ee None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:08.457723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81b00c72-4b40-41ca-b38b-3907636b85ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:08.476338 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1356/2705] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:54:08.480994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb5678-a61c-4096-a603-10a68a5811af None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:08.497400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb5678-a61c-4096-a603-10a68a5811af None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:08.505887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:08.506531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.506802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.507116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.507781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:08.508076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:08.514676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:08.537360 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1350/2706] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:08.542167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b00c72-4b40-41ca-b38b-3907636b85ee None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:08.553004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b00c72-4b40-41ca-b38b-3907636b85ee None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:08.558399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:08.559076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.559389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.559700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.560334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:08.562907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:08.623220 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1357/2707] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:54:08 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:54:08.627511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:08.638660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbda13-697b-474e-bf94-20a3d9c3b18d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:08.732071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:08.732679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.732950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.733216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.733908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:08.736904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:08.798749 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1351/2708] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:54:08 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:54:08.802781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:08.820932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9adb4b2-2149-4d49-83d9-dbe512d4eb56 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:08.829645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:08.830763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.831512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.832150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.835282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:08.837321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:08.856624 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1358/2709] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:54:08.857118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:08.869372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ec85-630d-4e1c-991b-0c62feb86156 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:08.875245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2ba45-6d93-410e-8885-6a57a44c97fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:08.875539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2ba45-6d93-410e-8885-6a57a44c97fe admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.875865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2ba45-6d93-410e-8885-6a57a44c97fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.876130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2ba45-6d93-410e-8885-6a57a44c97fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.876822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2ba45-6d93-410e-8885-6a57a44c97fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:08.878521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2ba45-6d93-410e-8885-6a57a44c97fe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:08.893435 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1352/2710] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:54:08.899253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:08.915520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6885c-3d36-4270-b192-77a0f679fac1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:08.923111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:08.925408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.925899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.926351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] PATH_INFO: `/v3/projects/52c89ce6659b4b4b9d6a53c9a8d33b41/users/d461326fa46149f5a49b278b32d9c9fb/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:08.948549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c89ce6659b4b4b9d6a53c9a8d33b41, user_id=d461326fa46149f5a49b278b32d9c9fb, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:08.952159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:08.968925 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1359/2711] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:54:08 2022] PUT /identity/v3/projects/52c89ce6659b4b4b9d6a53c9a8d33b41/users/d461326fa46149f5a49b278b32d9c9fb/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:54:08.974543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2156-a6a5-4c98-8c03-b0f7e00de013 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:08.974822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2156-a6a5-4c98-8c03-b0f7e00de013 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:08.975089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2156-a6a5-4c98-8c03-b0f7e00de013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:09.000750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-920c2156-a6a5-4c98-8c03-b0f7e00de013 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:09.003602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-920c2156-a6a5-4c98-8c03-b0f7e00de013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:09.036353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:09.081817 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1353/2712] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:09.099671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:09.103303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:09.211760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e33e3e-01a6-409a-9185-cd9e6ae757b6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:09.216226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d9508-e0ae-4b6b-8ec4-b6453a6916d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:09.217098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9508-e0ae-4b6b-8ec4-b6453a6916d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:09.217584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9508-e0ae-4b6b-8ec4-b6453a6916d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:09.217852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9508-e0ae-4b6b-8ec4-b6453a6916d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:09.230249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d9508-e0ae-4b6b-8ec4-b6453a6916d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:09.232032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d9508-e0ae-4b6b-8ec4-b6453a6916d4 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:09.245327 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1360/2713] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:54:10.092494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6871-c0d5-470a-88ae-a7a80116b695 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:10.093030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6871-c0d5-470a-88ae-a7a80116b695 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:10.093132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6871-c0d5-470a-88ae-a7a80116b695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:10.129879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2aae6871-c0d5-470a-88ae-a7a80116b695 None None] MFA Rules not processed for user `226a87a1f393420b8d19ec2390e9d063`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:10.135125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2aae6871-c0d5-470a-88ae-a7a80116b695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:10.250970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1354/2714] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:13.232526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e856-377f-42e6-8063-c99e471a6bfd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:13.233399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e856-377f-42e6-8063-c99e471a6bfd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:13.233878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e856-377f-42e6-8063-c99e471a6bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:13.256111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8234e856-377f-42e6-8063-c99e471a6bfd None None] MFA Rules not processed for user `9c2b746b7c194596808a7b0c736f3b53`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:13.257336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8234e856-377f-42e6-8063-c99e471a6bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:13.339805 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1361/2715] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:13.486252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31396e45-a3a8-41b4-a8ba-6f75ccbd13f9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:13.486842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31396e45-a3a8-41b4-a8ba-6f75ccbd13f9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:13.486842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31396e45-a3a8-41b4-a8ba-6f75ccbd13f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:13.513798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-31396e45-a3a8-41b4-a8ba-6f75ccbd13f9 None None] MFA Rules not processed for user `aaa9e5c7b7dd4a74935de34327145697`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:13.515405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-31396e45-a3a8-41b4-a8ba-6f75ccbd13f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:13.642097 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1355/2716] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:20.069216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:20.070165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:20.070498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:20.100091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] MFA Rules not processed for user `d461326fa46149f5a49b278b32d9c9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:20.101637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:20.105871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:20.190997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1362/2717] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:20.192782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:20.193253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:20.193350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:20.214355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] MFA Rules not processed for user `d461326fa46149f5a49b278b32d9c9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:20.215591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:20.223535 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1356/2718] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:20.235168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:20.236178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:20.371552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3c2d6-cbae-450e-8ed0-95492723fcd1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:20.380874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:20.381538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:20.381825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:20.382100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:20.404707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:20.406422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:20.418729 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1363/2719] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:54:30.208506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=65 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:30.404318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:30.405797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:30.413561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=59 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:30.434851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b78a40-0178-4c97-b34a-26cbf12afded None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:30.451996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:30.454160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:30.454732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:30.455525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] PATH_INFO: `/v3/users/495973792ca942bca2c6f65d545346e9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:30.457450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=495973792ca942bca2c6f65d545346e9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:30.461592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:30.582552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495973792ca942bca2c6f65d545346e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:30.583767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495973792ca942bca2c6f65d545346e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:30.618496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495973792ca942bca2c6f65d545346e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:30.619412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495973792ca942bca2c6f65d545346e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:30.655438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-724da3a5-3ff9-4d1b-a672-00b04d90b870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495973792ca942bca2c6f65d545346e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:30.676700 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1357/2720] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:54:30 2022] DELETE /identity/v3/users/495973792ca942bca2c6f65d545346e9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:31.069992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.081345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e1e1b-7cb5-43eb-9ccc-2e01af7e6a5d service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:31.208990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:31.210108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.210941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.211090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] PATH_INFO: `/v3/projects/5d29b99a6bd9481f9042d9c7a9451c49` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.212604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d29b99a6bd9481f9042d9c7a9451c49)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:31.215497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:31.284785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d29b99a6bd9481f9042d9c7a9451c49', 'request_id': 'req-d04f5f88-546a-408c-a1ee-104eb8bdbf83'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:31.300285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d29b99a6bd9481f9042d9c7a9451c49', 'request_id': 'req-d04f5f88-546a-408c-a1ee-104eb8bdbf83'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:31.308609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d29b99a6bd9481f9042d9c7a9451c49', 'request_id': 'req-d04f5f88-546a-408c-a1ee-104eb8bdbf83'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:31.322692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] The token cache is being invalidate because project 5d29b99a6bd9481f9042d9c7a9451c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:54:31.323088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:31.323864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d04f5f88-546a-408c-a1ee-104eb8bdbf83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:54:31.325825 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1364/2721] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:54:31 2022] DELETE /identity/v3/projects/5d29b99a6bd9481f9042d9c7a9451c49 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:31.331180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006299b0-1322-4c15-9dc5-abef971938cd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.331719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006299b0-1322-4c15-9dc5-abef971938cd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.331719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006299b0-1322-4c15-9dc5-abef971938cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.371149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-006299b0-1322-4c15-9dc5-abef971938cd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:31.372476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-006299b0-1322-4c15-9dc5-abef971938cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:31.441925 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1358/2722] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:31.450213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731974c-d3d4-48ce-a6a4-253216ad5a09 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.450725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731974c-d3d4-48ce-a6a4-253216ad5a09 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.450932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731974c-d3d4-48ce-a6a4-253216ad5a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.472882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9731974c-d3d4-48ce-a6a4-253216ad5a09 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:31.474726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9731974c-d3d4-48ce-a6a4-253216ad5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:31.486276 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1365/2723] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:54:31.488056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006299b0-1322-4c15-9dc5-abef971938cd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.500708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006299b0-1322-4c15-9dc5-abef971938cd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:31.511437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:31.512039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.512398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.512670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.513506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:31.513668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:31.515600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:31.529769 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1359/2724] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:31.534309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731974c-d3d4-48ce-a6a4-253216ad5a09 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.546100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731974c-d3d4-48ce-a6a4-253216ad5a09 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:31.551348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:31.551968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.552235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.552503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.553152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:31.555228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:31.589212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1366/2725] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:54:31 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:54:31.593667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.603847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3088d98-f2fc-4aea-9463-16234643f10d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:31.659995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:31.660603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.660878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.661144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.661820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:31.663773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:31.671794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:31.688417 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1360/2726] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:54:31 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:54:31.692757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.703863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e8d42-9fc3-42a1-8e00-8cc4cdafe094 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:31.708776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:31.709352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.709658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.709928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.710565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:31.712256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:31.722219 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1367/2727] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:54:31.726463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.737078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd479c6-084b-4e68-b5ab-8ad509165ebc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:31.742618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ae402-53b2-4afe-a792-8b192e803e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:31.743213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ae402-53b2-4afe-a792-8b192e803e3a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.743488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ae402-53b2-4afe-a792-8b192e803e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.743775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ae402-53b2-4afe-a792-8b192e803e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.744535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ae402-53b2-4afe-a792-8b192e803e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:31.746178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ae402-53b2-4afe-a792-8b192e803e3a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:31.757233 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1361/2728] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:54:31.761557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.771908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5317d1-b410-42f7-889e-fec37a858a08 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:31.777927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:31.778547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.778843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.779110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] PATH_INFO: `/v3/projects/822801d3963d45c1b4176259fb5c5b19/users/55e8e491f8844f7696f93d8ec5a7171b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.794169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] RBAC: Authorizing `identity:create_grant(project_id=822801d3963d45c1b4176259fb5c5b19, user_id=55e8e491f8844f7696f93d8ec5a7171b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:31.796912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:31.812438 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1368/2729] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:54:31 2022] PUT /identity/v3/projects/822801d3963d45c1b4176259fb5c5b19/users/55e8e491f8844f7696f93d8ec5a7171b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:54:31.817907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1a7a6-1ffd-44cf-9b14-cd33e0dfe24e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.818297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1a7a6-1ffd-44cf-9b14-cd33e0dfe24e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.818575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1a7a6-1ffd-44cf-9b14-cd33e0dfe24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.839588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7ae1a7a6-1ffd-44cf-9b14-cd33e0dfe24e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:31.840788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7ae1a7a6-1ffd-44cf-9b14-cd33e0dfe24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:31.913111 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1362/2730] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:54:31.924178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:31.926157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:31.997611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:32.029643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8e98-429f-449a-8d9b-6e879c59629c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:32.033412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:32.034054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:32.034336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:32.034605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:32.043902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:32.045364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:32.056138 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1369/2731] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:54:41.902638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=3,keystone:SELECT=34 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:42.051165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=14 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:43.418465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd666f-2260-4c9c-9c98-4c5996984c5c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.419758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd666f-2260-4c9c-9c98-4c5996984c5c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.420206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd666f-2260-4c9c-9c98-4c5996984c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.486068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9dcd666f-2260-4c9c-9c98-4c5996984c5c None None] MFA Rules not processed for user `55e8e491f8844f7696f93d8ec5a7171b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:43.487355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9dcd666f-2260-4c9c-9c98-4c5996984c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:43.622151 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1363/2732] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:54:43.642419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:43.665239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14817e97-9c68-4ec8-80e8-a7bca9475b90 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:43.684271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:43.685118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.686140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.686603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.688558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:43.695939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:43.748949 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1370/2733] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:54:43 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:54:43.752810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd666f-2260-4c9c-9c98-4c5996984c5c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:43.768188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd666f-2260-4c9c-9c98-4c5996984c5c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:43.925942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:43.927660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.927660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.927909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.928791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:43.931484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:43.988194 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1364/2734] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:54:43 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:54:43.989899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:44.006164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f918f35-9f54-4224-b899-615c231c962a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:44.020412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:44.021374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:44.025939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:44.026378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:44.028378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:44.035299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:44.060390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1371/2735] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:54:44.064418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:44.085669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51f67-0dc8-4a41-8342-8b86070d08d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:44.100278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:44.101397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:44.101901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:44.102332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] PATH_INFO: `/v3/projects/3da703231acd4485b6620f22891ad268/users/35e971080bf140cabf769606048f1436/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:44.138304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da703231acd4485b6620f22891ad268, user_id=35e971080bf140cabf769606048f1436, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:44.141728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:44.177850 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1365/2736] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:54:44 2022] PUT /identity/v3/projects/3da703231acd4485b6620f22891ad268/users/35e971080bf140cabf769606048f1436/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:44.182192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:44.197201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fee20d-c007-48b3-884d-4aa1dbd80335 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:44.292182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0c75f-52cb-40f1-ad16-c613b61500e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:44.293171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0c75f-52cb-40f1-ad16-c613b61500e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:44.293679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0c75f-52cb-40f1-ad16-c613b61500e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:44.294183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0c75f-52cb-40f1-ad16-c613b61500e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:44.294958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0c75f-52cb-40f1-ad16-c613b61500e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:44.298753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0c75f-52cb-40f1-ad16-c613b61500e7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:44.314959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1372/2737] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:54:44.319265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:44.338547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb57de-0af4-4598-b39d-0d0c34631a48 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:44.355642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfa389-4600-4111-bc7d-e0f583432f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:44.356349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfa389-4600-4111-bc7d-e0f583432f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:44.356630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfa389-4600-4111-bc7d-e0f583432f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:44.356909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfa389-4600-4111-bc7d-e0f583432f14 admin admin] PATH_INFO: `/v3/projects/3da703231acd4485b6620f22891ad268/users/35e971080bf140cabf769606048f1436/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:44.363310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfa389-4600-4111-bc7d-e0f583432f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da703231acd4485b6620f22891ad268, user_id=35e971080bf140cabf769606048f1436, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:44.372062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfa389-4600-4111-bc7d-e0f583432f14 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:44.406211 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1366/2738] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:54:44 2022] PUT /identity/v3/projects/3da703231acd4485b6620f22891ad268/users/35e971080bf140cabf769606048f1436/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:54.307881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:54.394230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:00.038811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:00.039357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:00.039464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:00.103599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] MFA Rules not processed for user `9c2b746b7c194596808a7b0c736f3b53`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:00.105084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:00.211350 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1373/2739] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:00.240544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:00.241250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:00.241250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:00.293991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] MFA Rules not processed for user `35e971080bf140cabf769606048f1436`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:00.295954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:00.519976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1367/2740] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:00.530582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:00.532415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:00.691746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69c60c-be6c-4d54-b354-a25674ae6de3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:00.705579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:00.706405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:00.706862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:00.707159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:00.858252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:00.861270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:00.877896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:00.894664 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1374/2741] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:10.492953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=37 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:10.714757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:10.716750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:10.831972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e48802-e6c2-4e2c-9bcc-79cf8c76d819 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:10.837128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:10.838027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:10.838475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:10.838903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] PATH_INFO: `/v3/users/42f7c0fdad9346c58949928300021b07` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:10.841138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f7c0fdad9346c58949928300021b07)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:10.843915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:10.892950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:10.908288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f7c0fdad9346c58949928300021b07'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:10.909979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f7c0fdad9346c58949928300021b07'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:10.938036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f7c0fdad9346c58949928300021b07'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:10.953119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f7c0fdad9346c58949928300021b07'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:10.992697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f7c0fdad9346c58949928300021b07'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:11.018528 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1368/2742] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:10 2022] DELETE /identity/v3/users/42f7c0fdad9346c58949928300021b07 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:12.099437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:12.126458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e780674-d56b-458f-9eea-29daf4794f10 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:12.246516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:12.247630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:12.248204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:12.248329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] PATH_INFO: `/v3/projects/1f305d163acf499c9808c5f11150a7cf` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:12.250225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f305d163acf499c9808c5f11150a7cf)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:12.253508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:12.366313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f305d163acf499c9808c5f11150a7cf', 'request_id': 'req-e87fb611-097f-4eb1-aaec-05e4bd89c52f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.392424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f305d163acf499c9808c5f11150a7cf', 'request_id': 'req-e87fb611-097f-4eb1-aaec-05e4bd89c52f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.406152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f305d163acf499c9808c5f11150a7cf', 'request_id': 'req-e87fb611-097f-4eb1-aaec-05e4bd89c52f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.437877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] The token cache is being invalidate because project 1f305d163acf499c9808c5f11150a7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:12.437877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.444229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.448372 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1375/2743] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:12 2022] DELETE /identity/v3/projects/1f305d163acf499c9808c5f11150a7cf => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:12.449637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:12.451775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:12.475940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d14ef-f8f0-457f-81d7-ecfee64edaea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:12.650746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:12.723271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:12.725299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:12.726401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:12.726739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] PATH_INFO: `/v3/users/535f4800b67b46a3a6fd2d431ca88860` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:12.728627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535f4800b67b46a3a6fd2d431ca88860)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:12.731239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:12.849338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535f4800b67b46a3a6fd2d431ca88860'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.850260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535f4800b67b46a3a6fd2d431ca88860'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.859617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535f4800b67b46a3a6fd2d431ca88860'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.862098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535f4800b67b46a3a6fd2d431ca88860'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.882455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-38d8549c-5250-499f-935e-a3e493a20f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535f4800b67b46a3a6fd2d431ca88860'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:12.925426 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1369/2744] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:12 2022] DELETE /identity/v3/users/535f4800b67b46a3a6fd2d431ca88860 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:13.588808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:13.590952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:13.614910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fb611-097f-4eb1-aaec-05e4bd89c52f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:13.730420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:13.731115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:13.731964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:13.732244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] PATH_INFO: `/v3/projects/6175b06907754fa0b579311bcf7a5d96` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:13.734994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6175b06907754fa0b579311bcf7a5d96)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:13.738292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:13.814856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:13.858757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6175b06907754fa0b579311bcf7a5d96', 'request_id': 'req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:13.868226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6175b06907754fa0b579311bcf7a5d96', 'request_id': 'req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:13.904973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6175b06907754fa0b579311bcf7a5d96', 'request_id': 'req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:13.927364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] The token cache is being invalidate because project 6175b06907754fa0b579311bcf7a5d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:13.929729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:13.930743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bc66eb0c-28a8-4cb0-93ec-98ddb239c6bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:13.938680 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1376/2745] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:13 2022] DELETE /identity/v3/projects/6175b06907754fa0b579311bcf7a5d96 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:13.939940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274790f8-af38-4ac3-b62f-81afdb9cad0f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:13.941141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274790f8-af38-4ac3-b62f-81afdb9cad0f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:13.941786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274790f8-af38-4ac3-b62f-81afdb9cad0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.042951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-274790f8-af38-4ac3-b62f-81afdb9cad0f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:14.045067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-274790f8-af38-4ac3-b62f-81afdb9cad0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:14.160176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1370/2746] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:14.162989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91b9e5-7f93-452a-b49f-febf0c32eb9d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.163450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91b9e5-7f93-452a-b49f-febf0c32eb9d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.163895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91b9e5-7f93-452a-b49f-febf0c32eb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.185374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8c91b9e5-7f93-452a-b49f-febf0c32eb9d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:14.186908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8c91b9e5-7f93-452a-b49f-febf0c32eb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:14.200753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274790f8-af38-4ac3-b62f-81afdb9cad0f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:14.206026 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1377/2747] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:14.212474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274790f8-af38-4ac3-b62f-81afdb9cad0f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:14.217623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:14.218841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.219279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.219702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.220596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:14.221096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:14.223293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:14.243080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91b9e5-7f93-452a-b49f-febf0c32eb9d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:14.253269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91b9e5-7f93-452a-b49f-febf0c32eb9d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:14.254031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1371/2748] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:14.259042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:14.259674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.259951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.260219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.260858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:14.263028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:14.317259 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1378/2749] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:14 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:14.322569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:14.336256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9cc1e-6f8e-43a5-8767-d7de2c778263 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:14.398137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:14.473342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:14.474033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.474314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.474840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.475499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:14.481862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:14.520944 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1372/2750] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:14 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:14.529906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:14.555274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d535c-057e-41a1-b4b7-6411c94c480f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:14.566182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:14.566918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.567201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.567474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.568405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:14.570902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:14.587811 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1379/2751] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:14.588703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:14.601871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f4973-4168-4a68-a379-cf35165b1456 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:14.610211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097447db-6671-4c4a-9131-433bb76a0311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:14.611402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097447db-6671-4c4a-9131-433bb76a0311 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.611843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097447db-6671-4c4a-9131-433bb76a0311 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.612330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097447db-6671-4c4a-9131-433bb76a0311 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.613151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097447db-6671-4c4a-9131-433bb76a0311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:14.616346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097447db-6671-4c4a-9131-433bb76a0311 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:14.633839 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1373/2752] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:14.638106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:14.651635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bd501-3c8d-4b11-bec5-e29cee9c6f14 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:14.657187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:14.658432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.658917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.659402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] PATH_INFO: `/v3/projects/7e713977cb174684af5ef6158b6ed24f/users/707e36ba00ae443b946a198809dfd716/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.679134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e713977cb174684af5ef6158b6ed24f, user_id=707e36ba00ae443b946a198809dfd716, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:14.683338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:14.723494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1380/2753] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:14 2022] PUT /identity/v3/projects/7e713977cb174684af5ef6158b6ed24f/users/707e36ba00ae443b946a198809dfd716/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:14.724078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ed0f-fae7-4a3f-90c9-a821ed1597e1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:14.724491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ed0f-fae7-4a3f-90c9-a821ed1597e1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:14.724869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89ed0f-fae7-4a3f-90c9-a821ed1597e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:14.789041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2f89ed0f-fae7-4a3f-90c9-a821ed1597e1 None None] MFA Rules not processed for user `707e36ba00ae443b946a198809dfd716`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:14.790434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2f89ed0f-fae7-4a3f-90c9-a821ed1597e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:14.930407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1374/2754] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:14.936705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:15.081337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7a123-29f4-4ca1-a8d4-a887e75a9c70 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:15.088801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:15.090018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:15.090498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:15.091768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:15.092286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:15.095696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:15.157763 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1381/2755] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:14 2022] POST /identity/v3/projects => generated 377 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:15.162086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ed0f-fae7-4a3f-90c9-a821ed1597e1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:15.178997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89ed0f-fae7-4a3f-90c9-a821ed1597e1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:15.297207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:15.297781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:15.298058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:15.298329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:15.299024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:15.300963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:15.351104 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1375/2756] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:15 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:15.355263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:15.381676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645fcb9-2d54-40fe-bac7-739a0647b34f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:15.391183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:15.391821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:15.392100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:15.392502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:15.393186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:15.394873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:15.420879 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1382/2757] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:15.426483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:15.445972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63cdbf-bc84-4078-bf4c-4106194803c2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:15.461800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:15.463052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:15.463334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:15.463602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] PATH_INFO: `/v3/projects/efec93346191460680ef66d4802fbf7a/users/d750b1134d6f4caabd0176a49b206094/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:15.487517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efec93346191460680ef66d4802fbf7a, user_id=d750b1134d6f4caabd0176a49b206094, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:15.493299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:15.515777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:15.535976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1376/2758] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:15 2022] PUT /identity/v3/projects/efec93346191460680ef66d4802fbf7a/users/d750b1134d6f4caabd0176a49b206094/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:15.540964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:15.559680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57a155-8130-4420-90a7-82a0bc7e60a5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:15.607653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:15.722889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f1320-d8fe-4a5a-9625-32b4d1c92c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:15.723674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f1320-d8fe-4a5a-9625-32b4d1c92c1e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:15.723674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f1320-d8fe-4a5a-9625-32b4d1c92c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:15.723881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f1320-d8fe-4a5a-9625-32b4d1c92c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:15.724790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f1320-d8fe-4a5a-9625-32b4d1c92c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:15.729808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f1320-d8fe-4a5a-9625-32b4d1c92c1e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:15.779340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:15.791193 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1383/2759] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:15.793810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aebee0-0df4-45aa-b864-bcd4b4a84ce8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:15.828720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:15.829735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:15.831702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:15.832144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] PATH_INFO: `/v3/projects/efec93346191460680ef66d4802fbf7a/users/d750b1134d6f4caabd0176a49b206094/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:15.835556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efec93346191460680ef66d4802fbf7a, user_id=d750b1134d6f4caabd0176a49b206094, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:15.840371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:15.869499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1377/2760] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:15 2022] PUT /identity/v3/projects/efec93346191460680ef66d4802fbf7a/users/d750b1134d6f4caabd0176a49b206094/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:55:15.872104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e246e-85c5-446e-abf1-cc5a3e56152a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:15.872590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e246e-85c5-446e-abf1-cc5a3e56152a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:15.873015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e246e-85c5-446e-abf1-cc5a3e56152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:16.108711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1f1e246e-85c5-446e-abf1-cc5a3e56152a None None] MFA Rules not processed for user `d750b1134d6f4caabd0176a49b206094`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:16.110450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1f1e246e-85c5-446e-abf1-cc5a3e56152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:16.220112 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1384/2761] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:16.700866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:16.734408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:16.998700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66f0c1-c855-43e9-aba3-a935e032f8f3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:17.014380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:17.015408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:17.015862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:17.016142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:17.036083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:17.043401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:17.086706 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1378/2762] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:17.965867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9ae13-108a-45f2-90be-a12bb46c3b26 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:17.966443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9ae13-108a-45f2-90be-a12bb46c3b26 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:17.966537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9ae13-108a-45f2-90be-a12bb46c3b26 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:17.968191 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1385/2763] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:55:17.973311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:55:17.993697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:18.027482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd414e-abb5-448a-ade3-d4df5b5e3f5f service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:18.041204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23efc2-a105-427b-90ca-1a4f3cdf541d tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd750b1134d6f4caabd0176a49b206094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efec93346191460680ef66d4802fbf7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:18.041903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23efc2-a105-427b-90ca-1a4f3cdf541d tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:18.044596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23efc2-a105-427b-90ca-1a4f3cdf541d tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:18.045072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23efc2-a105-427b-90ca-1a4f3cdf541d tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] PATH_INFO: `/v3/projects/7e713977cb174684af5ef6158b6ed24f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:18.049616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23efc2-a105-427b-90ca-1a4f3cdf541d tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] RBAC: Authorizing `identity:get_project(project_id=7e713977cb174684af5ef6158b6ed24f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:18.055689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23efc2-a105-427b-90ca-1a4f3cdf541d tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:18.058316 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1379/2764] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:55:17 2022] GET /identity/v3/projects/7e713977cb174684af5ef6158b6ed24f => generated 379 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:19.016091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9ae13-108a-45f2-90be-a12bb46c3b26 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:19.034713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9ae13-108a-45f2-90be-a12bb46c3b26 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:19.044116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:19.054258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:19.054913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:19.055441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:19.144066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:19.146568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:19.192820 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1386/2765] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:19.268307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16309c6-479c-4a02-8d10-585bce6f90f3 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:19.268870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16309c6-479c-4a02-8d10-585bce6f90f3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:19.268870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16309c6-479c-4a02-8d10-585bce6f90f3 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:19.270222 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1380/2766] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:55:19.281315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:55:19.282839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:19.310786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:19.330305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7305117-b46a-4cee-b9ed-8b48a169f54f service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:19.335972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d72ce1-42ca-4d2b-a2ff-528fa28a0531 tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd750b1134d6f4caabd0176a49b206094', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efec93346191460680ef66d4802fbf7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:19.336641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72ce1-42ca-4d2b-a2ff-528fa28a0531 tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:19.336911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72ce1-42ca-4d2b-a2ff-528fa28a0531 tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:19.337202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72ce1-42ca-4d2b-a2ff-528fa28a0531 tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] PATH_INFO: `/v3/projects/7e713977cb174684af5ef6158b6ed24f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:19.338463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d72ce1-42ca-4d2b-a2ff-528fa28a0531 tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] RBAC: Authorizing `identity:get_project(project_id=7e713977cb174684af5ef6158b6ed24f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:19.340536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d72ce1-42ca-4d2b-a2ff-528fa28a0531 tempest-FlavorsAccessTestJSON-969148146 tempest-FlavorsAccessTestJSON-969148146-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:19.342996 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1387/2767] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:55:19 2022] GET /identity/v3/projects/7e713977cb174684af5ef6158b6ed24f => generated 379 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:19.949535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16309c6-479c-4a02-8d10-585bce6f90f3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:20.057696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16309c6-479c-4a02-8d10-585bce6f90f3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:20.063406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:20.064059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:20.064341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:20.064612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] PATH_INFO: `/v3/users/707e36ba00ae443b946a198809dfd716` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:20.066309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=707e36ba00ae443b946a198809dfd716)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:20.069685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:20.184700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707e36ba00ae443b946a198809dfd716'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:20.185585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707e36ba00ae443b946a198809dfd716'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:20.208513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707e36ba00ae443b946a198809dfd716'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:20.209358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707e36ba00ae443b946a198809dfd716'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:20.226491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:20.235458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707e36ba00ae443b946a198809dfd716'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:20.264314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1381/2768] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:19 2022] DELETE /identity/v3/users/707e36ba00ae443b946a198809dfd716 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:55:20.269471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:20.269976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:20.270262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:20.309710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:20.311523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:20.369283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:20.403390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1388/2769] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:20.426332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:20.431532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:20.700594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621db03-768a-473b-a37e-61fc41917ecc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:20.708351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:20.709112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:20.709504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:20.709831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:20.729091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:20.730502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:20.746098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1382/2770] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:21.547945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:21.549408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:21.571843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac5eec-908d-46ae-a746-fb8202592e3d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:21.580973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:21.581992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:21.582632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:21.583077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] PATH_INFO: `/v3/projects/7e713977cb174684af5ef6158b6ed24f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:21.586104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e713977cb174684af5ef6158b6ed24f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:21.591061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:21.714410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e713977cb174684af5ef6158b6ed24f', 'request_id': 'req-25768c16-bcfc-410a-8278-91965d61bd89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:21.741980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e713977cb174684af5ef6158b6ed24f', 'request_id': 'req-25768c16-bcfc-410a-8278-91965d61bd89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:21.763836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e713977cb174684af5ef6158b6ed24f', 'request_id': 'req-25768c16-bcfc-410a-8278-91965d61bd89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:21.784263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] The token cache is being invalidate because project 7e713977cb174684af5ef6158b6ed24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:21.784531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:21.785775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:21.789632 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1389/2771] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:21 2022] DELETE /identity/v3/projects/7e713977cb174684af5ef6158b6ed24f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:21.792045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:21.804294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e405c0-6269-4151-a8bc-1ec897b00c14 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:21.910117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:21.910860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:21.911310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:21.911611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] PATH_INFO: `/v3/users/d750b1134d6f4caabd0176a49b206094` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:21.914291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d750b1134d6f4caabd0176a49b206094)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:21.916917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:22.018971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd750b1134d6f4caabd0176a49b206094'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:22.020684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd750b1134d6f4caabd0176a49b206094'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:22.044133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd750b1134d6f4caabd0176a49b206094'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:22.045639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd750b1134d6f4caabd0176a49b206094'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:22.081380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:22.083997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd750b1134d6f4caabd0176a49b206094'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:22.140173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1383/2772] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:21 2022] DELETE /identity/v3/users/d750b1134d6f4caabd0176a49b206094 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:23.240427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:23.242618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:23.264122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25768c16-bcfc-410a-8278-91965d61bd89 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:23.343368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:23.344188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:23.344464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:23.345123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] PATH_INFO: `/v3/users/aaa9e5c7b7dd4a74935de34327145697` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:23.346346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa9e5c7b7dd4a74935de34327145697)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:23.348476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:23.426984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa9e5c7b7dd4a74935de34327145697'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:23.442939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:23.449401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa9e5c7b7dd4a74935de34327145697'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:23.464019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa9e5c7b7dd4a74935de34327145697'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:23.465118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa9e5c7b7dd4a74935de34327145697'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:23.466619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d137d6e0-5c52-416e-a247-9ce00acf10d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa9e5c7b7dd4a74935de34327145697'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:23.491773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1390/2773] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:23 2022] DELETE /identity/v3/users/aaa9e5c7b7dd4a74935de34327145697 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:23.856656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:23.858197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:23.875206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09c9d8-bcdc-4c80-aed3-9f1fc7bc2f2a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:23.953675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:23.954599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:23.955296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:23.955711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] PATH_INFO: `/v3/projects/efec93346191460680ef66d4802fbf7a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:23.957071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efec93346191460680ef66d4802fbf7a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:23.959363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:24.028288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efec93346191460680ef66d4802fbf7a', 'request_id': 'req-37736008-6818-4c29-b672-acb907bfbc21'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.036259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efec93346191460680ef66d4802fbf7a', 'request_id': 'req-37736008-6818-4c29-b672-acb907bfbc21'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.042987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532b35-a4d5-4871-b318-e5538e6052d3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.043360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532b35-a4d5-4871-b318-e5538e6052d3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.043535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532b35-a4d5-4871-b318-e5538e6052d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.056742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efec93346191460680ef66d4802fbf7a', 'request_id': 'req-37736008-6818-4c29-b672-acb907bfbc21'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.078239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-00532b35-a4d5-4871-b318-e5538e6052d3 None None] MFA Rules not processed for user `55e8e491f8844f7696f93d8ec5a7171b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:24.079627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-00532b35-a4d5-4871-b318-e5538e6052d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.080282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] The token cache is being invalidate because project efec93346191460680ef66d4802fbf7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:24.080941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.082055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-37736008-6818-4c29-b672-acb907bfbc21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.084745 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1384/2774] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:23 2022] DELETE /identity/v3/projects/efec93346191460680ef66d4802fbf7a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:24.091108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998556b-120d-44c8-bcb3-ae5f78cd89d6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.091629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998556b-120d-44c8-bcb3-ae5f78cd89d6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.092109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998556b-120d-44c8-bcb3-ae5f78cd89d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.123224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f998556b-120d-44c8-bcb3-ae5f78cd89d6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:24.124727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f998556b-120d-44c8-bcb3-ae5f78cd89d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.186169 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1391/2775] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:24.196763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6f4a7-5356-44e3-89c8-4836cccea748 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.197362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6f4a7-5356-44e3-89c8-4836cccea748 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.197876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6f4a7-5356-44e3-89c8-4836cccea748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.234017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-deb6f4a7-5356-44e3-89c8-4836cccea748 None None] MFA Rules not processed for user `55e8e491f8844f7696f93d8ec5a7171b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:24.236092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-deb6f4a7-5356-44e3-89c8-4836cccea748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.238140 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1385/2776] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:24.247139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.247460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.247743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.264346 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1392/2777] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:24.282372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:24.284258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.324216 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1386/2778] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:24.329024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6f4a7-5356-44e3-89c8-4836cccea748 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.350468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6f4a7-5356-44e3-89c8-4836cccea748 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.363561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.364300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.364612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.366482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.369689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:24.370349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.372676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:24.394023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.397330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.405590 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1393/2779] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:24.406949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.416583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b381b-603a-4dab-a4db-35eb89aad4f1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.422637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21816-0665-48de-a7b2-f8095ff3d14d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.424210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.425442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.425997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.426515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] PATH_INFO: `/v3/projects/6b843f5d09c94cf3916f65e815d6d7a1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.428587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b843f5d09c94cf3916f65e815d6d7a1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.431354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:24.431602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.432228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.432512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.433015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.433697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.436381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:24.436550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:24.488935 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1394/2780] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:24.494657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.560856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b843f5d09c94cf3916f65e815d6d7a1', 'request_id': 'req-d83f7c5f-4519-45af-bc8f-73c8be44eb42'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.589247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b843f5d09c94cf3916f65e815d6d7a1', 'request_id': 'req-d83f7c5f-4519-45af-bc8f-73c8be44eb42'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.603570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b843f5d09c94cf3916f65e815d6d7a1', 'request_id': 'req-d83f7c5f-4519-45af-bc8f-73c8be44eb42'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.622732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] The token cache is being invalidate because project 6b843f5d09c94cf3916f65e815d6d7a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:24.623033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.623747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d83f7c5f-4519-45af-bc8f-73c8be44eb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:24.629048 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1387/2781] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:24 2022] DELETE /identity/v3/projects/6b843f5d09c94cf3916f65e815d6d7a1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:55:24.631675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bb45f-a12e-4b35-95c8-81e139fbb61f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.631675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bb45f-a12e-4b35-95c8-81e139fbb61f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.631942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bb45f-a12e-4b35-95c8-81e139fbb61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.638987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f3da8-07f2-41aa-a0b4-19816fb3eafe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.659661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-164bb45f-a12e-4b35-95c8-81e139fbb61f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:24.661344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-164bb45f-a12e-4b35-95c8-81e139fbb61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.763847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:24.795266 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1388/2782] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:24.797624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf93344-24d4-41a6-8654-c6ae8684d62e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.798014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf93344-24d4-41a6-8654-c6ae8684d62e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.798409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf93344-24d4-41a6-8654-c6ae8684d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.802476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.803895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.804314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.804769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.807312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.809563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:24.831204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3bf93344-24d4-41a6-8654-c6ae8684d62e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:24.832955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3bf93344-24d4-41a6-8654-c6ae8684d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.853979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1389/2783] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:24.861397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf93344-24d4-41a6-8654-c6ae8684d62e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.862699 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1395/2784] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/users => generated 286 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:24.863702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.866816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:24.886400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf93344-24d4-41a6-8654-c6ae8684d62e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.893849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.894029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589f3f1-370b-4965-a723-db4d13208230 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.894683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.895185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.895568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.896583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:24.897089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.899896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.900508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.900687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:24.900824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.901039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.901714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.918036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:24.931605 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1390/2785] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:24.937398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.956207 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1396/2786] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:24.959949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:24.975310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e03f7b-fb7e-4404-a093-868195369687 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.983575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493591d-8764-44ea-bd55-60cf67be642d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:24.990700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.991377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.991660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.993679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.994141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:24.994294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.994916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:24.995219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:24.995899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:24.996760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:24.996981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.007203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.032460 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1397/2787] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:25.037148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.043542 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1391/2788] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:24 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:55:25.048938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.056036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5868f4-9a01-49ee-9e96-70b8ec650acb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.161193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:25.184746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357b493-d8fb-45ac-b490-601964e8ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:25.185445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b493-d8fb-45ac-b490-601964e8ad04 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.185959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b493-d8fb-45ac-b490-601964e8ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.188043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b493-d8fb-45ac-b490-601964e8ad04 admin admin] PATH_INFO: `/v3/projects/a87a3b87a34a493c932f78fc5ded8eb4/users/6f2d16526463458db0ac2896095aad39/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.193006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4e85c-5d67-46dc-a602-f293605a11e3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.208074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:25.208859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.210518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.210518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.210997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:25.212960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.228156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357b493-d8fb-45ac-b490-601964e8ad04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87a3b87a34a493c932f78fc5ded8eb4, user_id=6f2d16526463458db0ac2896095aad39, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:25.231214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357b493-d8fb-45ac-b490-601964e8ad04 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.254628 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1392/2790] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:25 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:55:25.255960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.261805 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1398/2790] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:25 2022] PUT /identity/v3/projects/a87a3b87a34a493c932f78fc5ded8eb4/users/6f2d16526463458db0ac2896095aad39/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:25.266630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd8147-5a46-4128-8364-dce9152ca8a0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.267280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd8147-5a46-4128-8364-dce9152ca8a0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.267935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd8147-5a46-4128-8364-dce9152ca8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.283511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ca3-c921-4653-a823-ed526f56f349 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.349906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-82bd8147-5a46-4128-8364-dce9152ca8a0 None None] MFA Rules not processed for user `6f2d16526463458db0ac2896095aad39`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:25.351886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-82bd8147-5a46-4128-8364-dce9152ca8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:25.491350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:25.491995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.492270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.492541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.493200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:25.497325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.520367 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1399/2791] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:25.523852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd8147-5a46-4128-8364-dce9152ca8a0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.548799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd8147-5a46-4128-8364-dce9152ca8a0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.549902 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1393/2792] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:25.550974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.555043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:25.556719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.556719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.557039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.558009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:25.560292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.581011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ac529-a5fd-49c6-be4f-2c4da98a797a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.595664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:25.596398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.596730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.597065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.597810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:25.602796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.624227 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1394/2793] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:25.635468 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1400/2794] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:25 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:25.637090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.638050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.653833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:25.664103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde74dd-c3ed-4075-995c-80c4cda1d138 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.855307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04688e33-9882-487d-a9ff-de4e308826ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.865629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:25.867625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.868477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.869355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.870894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:25.873149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69060528-d6d2-4994-b168-b21b75f487eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:25.874161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69060528-d6d2-4994-b168-b21b75f487eb admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.874513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69060528-d6d2-4994-b168-b21b75f487eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.876236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.876426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69060528-d6d2-4994-b168-b21b75f487eb admin admin] PATH_INFO: `/v3/projects/2a4117ca05454573bbd93fbca50431d2/users/6e33e0b756e14af28a90cc5c76d346e6/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:25.917727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69060528-d6d2-4994-b168-b21b75f487eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4117ca05454573bbd93fbca50431d2, user_id=6e33e0b756e14af28a90cc5c76d346e6, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:25.928627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69060528-d6d2-4994-b168-b21b75f487eb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:25.930991 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1401/2795] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:25 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:25.936375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:25.961652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8533b7-ffaa-4fb0-8411-a57d544916bd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:25.963899 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1395/2796] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:25 2022] PUT /identity/v3/projects/2a4117ca05454573bbd93fbca50431d2/users/6e33e0b756e14af28a90cc5c76d346e6/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:25.968266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:25.968931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:25.969158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:26.023727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:26.026662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:26.141370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:26.142709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:26.143477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:26.143836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:26.144908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:26.145866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:26.148501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:26.175419 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1402/2797] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:26.178117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:26.181445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1396/2798] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:26.192575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:26.194496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:26.197563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d017f-8dcd-4836-bf72-58e59a79f67c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:26.207467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:26.208684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:26.209323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:26.209811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] PATH_INFO: `/v3/projects/22d4fabbe08e4aa5b84912021f9257c9/users/69e487cac9e048dc8583f2f739f04570/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:26.237659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d4fabbe08e4aa5b84912021f9257c9, user_id=69e487cac9e048dc8583f2f739f04570, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:26.241261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:26.271108 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1403/2799] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:26 2022] PUT /identity/v3/projects/22d4fabbe08e4aa5b84912021f9257c9/users/69e487cac9e048dc8583f2f739f04570/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:26.280329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:26.303205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07632a0e-faf5-48ef-8913-bb4bfc758b2b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:26.400690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee46d33-eab4-4a61-9913-1817d6c74bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:26.401446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee46d33-eab4-4a61-9913-1817d6c74bcf service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:26.401928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee46d33-eab4-4a61-9913-1817d6c74bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:26.401928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee46d33-eab4-4a61-9913-1817d6c74bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:26.521582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc9ee-12f3-40d8-8582-cffbaf4ced4e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:26.529051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:26.530145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:26.531409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:26.531853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:26.533004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:26.536171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:26.541905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee46d33-eab4-4a61-9913-1817d6c74bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:26.546904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee46d33-eab4-4a61-9913-1817d6c74bcf service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:26.556196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1404/2800] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:26.561199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:26.573340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1397/2801] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:55:26.596372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:26.613068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d14d25-ef3a-4e71-aefa-4bbe09d7d6c2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:26.626792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:26.630470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:26.631536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:26.632671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] PATH_INFO: `/v3/projects/22d4fabbe08e4aa5b84912021f9257c9/users/69e487cac9e048dc8583f2f739f04570/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:26.638758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d4fabbe08e4aa5b84912021f9257c9, user_id=69e487cac9e048dc8583f2f739f04570, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:26.649677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:26.691141 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1405/2802] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:26 2022] PUT /identity/v3/projects/22d4fabbe08e4aa5b84912021f9257c9/users/69e487cac9e048dc8583f2f739f04570/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:55:26.708500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:26.709086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:26.709675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:26.773205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] MFA Rules not processed for user `69e487cac9e048dc8583f2f739f04570`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:26.775916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:26.924308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:27.012587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1398/2803] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:28.108008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:28.112449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:28.240949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbe64b-c560-4393-a476-215cf3659b1f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:28.249862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558994ff-86d9-47f1-9480-f066e780e332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:28.250575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558994ff-86d9-47f1-9480-f066e780e332 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:28.250792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558994ff-86d9-47f1-9480-f066e780e332 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:28.251047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558994ff-86d9-47f1-9480-f066e780e332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:28.261919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558994ff-86d9-47f1-9480-f066e780e332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:28.263616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558994ff-86d9-47f1-9480-f066e780e332 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:28.282434 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1406/2804] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:30.152626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:30.154374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:30.292885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc9370-dcba-459b-b6ca-68bac38960fe None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:30.305071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:30.305845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:30.308352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:30.308763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:30.313601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:30.350441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:30.352939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:30.406290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1399/2805] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:30.906166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c605b-951c-425c-9f96-0b4393dfac1a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:30.907711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c605b-951c-425c-9f96-0b4393dfac1a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:30.908211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c605b-951c-425c-9f96-0b4393dfac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:30.947414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cd2c605b-951c-425c-9f96-0b4393dfac1a None None] MFA Rules not processed for user `d461326fa46149f5a49b278b32d9c9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:30.949263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cd2c605b-951c-425c-9f96-0b4393dfac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:31.075303 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1407/2806] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:31.593421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:31.596261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:31.709215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f7b0-9375-44a4-900b-72695b5bd8bb service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:31.714907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9e7b9-a45a-453b-9c12-b191075752b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:31.715881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e7b9-a45a-453b-9c12-b191075752b4 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:31.716375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e7b9-a45a-453b-9c12-b191075752b4 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:31.716871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e7b9-a45a-453b-9c12-b191075752b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:31.718964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7bf9e7b9-a45a-453b-9c12-b191075752b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:31.741557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9e7b9-a45a-453b-9c12-b191075752b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:31.745975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9e7b9-a45a-453b-9c12-b191075752b4 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:31.796626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1400/2807] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:32.736746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c605b-951c-425c-9f96-0b4393dfac1a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:32.857975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c605b-951c-425c-9f96-0b4393dfac1a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:32.879993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:32.880920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:32.881250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:32.881655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] PATH_INFO: `/v3/users/6f2d16526463458db0ac2896095aad39` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:32.884425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f2d16526463458db0ac2896095aad39)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:32.888035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:32.936929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:33.017674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f2d16526463458db0ac2896095aad39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:33.048196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f2d16526463458db0ac2896095aad39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:33.062628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f2d16526463458db0ac2896095aad39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:33.065030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f2d16526463458db0ac2896095aad39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:33.065938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f2d16526463458db0ac2896095aad39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:33.101750 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1408/2808] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:32 2022] DELETE /identity/v3/users/6f2d16526463458db0ac2896095aad39 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:33.102791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:33.103270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:33.103377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:33.137919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:33.149233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.285357 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1401/2809] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:33.304851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:33.307530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.367500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:33.369018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.396140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:33.474363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c7eee-1474-4f0d-8f2f-5e32ce1fa787 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:33.483115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1960b20-2204-4c0d-97c6-d468e10d6855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:33.483741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960b20-2204-4c0d-97c6-d468e10d6855 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:33.484023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960b20-2204-4c0d-97c6-d468e10d6855 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:33.484730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960b20-2204-4c0d-97c6-d468e10d6855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:33.503109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1960b20-2204-4c0d-97c6-d468e10d6855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:33.505644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1960b20-2204-4c0d-97c6-d468e10d6855 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:33.517683 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1409/2810] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:33.528489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e708645-af7c-40dc-ba9b-79edf86cc614 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:33.534636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48fcfd-195f-40c2-945c-6851dd09b8f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:33.535621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48fcfd-195f-40c2-945c-6851dd09b8f3 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:33.535775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48fcfd-195f-40c2-945c-6851dd09b8f3 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:33.535951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48fcfd-195f-40c2-945c-6851dd09b8f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:33.537880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-da48fcfd-195f-40c2-945c-6851dd09b8f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.557870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:33.558155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:33.558433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:33.616805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] MFA Rules not processed for user `6e33e0b756e14af28a90cc5c76d346e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:33.618961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.641772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48fcfd-195f-40c2-945c-6851dd09b8f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:33.644398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48fcfd-195f-40c2-945c-6851dd09b8f3 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:33.660029 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1402/2811] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:33.772329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1410/2812] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:33.773710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:33.774071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:33.774706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:33.849102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] MFA Rules not processed for user `6e33e0b756e14af28a90cc5c76d346e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:33.849962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:33.852703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.859338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.878473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1403/2813] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:33.890424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be033-1f9f-460e-95c2-a20d52e7e423 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:33.908683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:33.910086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:33.911413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:33.911990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:33.919015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:33.984400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:33.986210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:34.002821 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1411/2814] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:55:34.408799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:34.413299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:34.448921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c10174-b270-4325-b624-18b132849ad6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:34.457309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:34.458234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:34.458750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:34.459167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:34.472449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:34.474182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:34.497774 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1404/2815] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:34.796662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:34.798810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:34.853889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a229c7-84c4-4d25-afe6-34061dce0e7a service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:34.867931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:34.869088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:34.869641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:34.870190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] PATH_INFO: `/v3/projects/a87a3b87a34a493c932f78fc5ded8eb4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:34.872090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87a3b87a34a493c932f78fc5ded8eb4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:34.874630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:34.896211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:34.983656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87a3b87a34a493c932f78fc5ded8eb4', 'request_id': 'req-0e5bf57a-d535-48e9-8293-38d3bf52f487'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.008594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87a3b87a34a493c932f78fc5ded8eb4', 'request_id': 'req-0e5bf57a-d535-48e9-8293-38d3bf52f487'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.023379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87a3b87a34a493c932f78fc5ded8eb4', 'request_id': 'req-0e5bf57a-d535-48e9-8293-38d3bf52f487'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.041837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] The token cache is being invalidate because project a87a3b87a34a493c932f78fc5ded8eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:35.042688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.044501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.047216 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1412/2816] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:34 2022] DELETE /identity/v3/projects/a87a3b87a34a493c932f78fc5ded8eb4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:35.057316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:35.074364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb223c5a-a3d2-4073-b23a-ee885adf9c5b service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:35.183943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:35.184935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:35.185597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:35.186124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] PATH_INFO: `/v3/users/69e487cac9e048dc8583f2f739f04570` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:35.187576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e487cac9e048dc8583f2f739f04570)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:35.189768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:35.257741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:35.279970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e487cac9e048dc8583f2f739f04570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.281268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e487cac9e048dc8583f2f739f04570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.295377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e487cac9e048dc8583f2f739f04570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.296624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e487cac9e048dc8583f2f739f04570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.317501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15fb2f03-2c73-4484-8661-de2e9364bb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e487cac9e048dc8583f2f739f04570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:35.335369 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1405/2817] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:35 2022] DELETE /identity/v3/users/69e487cac9e048dc8583f2f739f04570 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:36.618262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:36.622382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:36.699823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bf57a-d535-48e9-8293-38d3bf52f487 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:36.854289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:36.854973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:36.855254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:36.855527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] PATH_INFO: `/v3/projects/22d4fabbe08e4aa5b84912021f9257c9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:36.861242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d4fabbe08e4aa5b84912021f9257c9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:36.863259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:37.018158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d4fabbe08e4aa5b84912021f9257c9', 'request_id': 'req-427ada35-4cf9-46ba-ab91-930f891a288b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:37.040240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d4fabbe08e4aa5b84912021f9257c9', 'request_id': 'req-427ada35-4cf9-46ba-ab91-930f891a288b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:37.048677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d4fabbe08e4aa5b84912021f9257c9', 'request_id': 'req-427ada35-4cf9-46ba-ab91-930f891a288b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:37.076642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] The token cache is being invalidate because project 22d4fabbe08e4aa5b84912021f9257c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:37.077059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:37.079324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-427ada35-4cf9-46ba-ab91-930f891a288b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:37.082153 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1413/2818] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:36 2022] DELETE /identity/v3/projects/22d4fabbe08e4aa5b84912021f9257c9 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:37.087752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506bafc-d4b7-4f4e-a3fc-0788bdac5aaf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:37.087752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506bafc-d4b7-4f4e-a3fc-0788bdac5aaf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:37.087752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506bafc-d4b7-4f4e-a3fc-0788bdac5aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:37.155488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1506bafc-d4b7-4f4e-a3fc-0788bdac5aaf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:37.157143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1506bafc-d4b7-4f4e-a3fc-0788bdac5aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:37.314982 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1406/2819] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:37.322963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ae5b-ec81-4c7d-9dae-19b47c0ca3a3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:37.323342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ae5b-ec81-4c7d-9dae-19b47c0ca3a3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:37.323524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ae5b-ec81-4c7d-9dae-19b47c0ca3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:37.374609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f263ae5b-ec81-4c7d-9dae-19b47c0ca3a3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:37.380181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f263ae5b-ec81-4c7d-9dae-19b47c0ca3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:37.407201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506bafc-d4b7-4f4e-a3fc-0788bdac5aaf None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:37.407437 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1414/2820] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:37.430616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506bafc-d4b7-4f4e-a3fc-0788bdac5aaf None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:37.451256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:37.451898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:37.452925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:37.452925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:37.453217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:37.453697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:37.461713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:37.505956 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1407/2821] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:55:37.510345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263ae5b-ec81-4c7d-9dae-19b47c0ca3a3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:37.521014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263ae5b-ec81-4c7d-9dae-19b47c0ca3a3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:37.542083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:37.542714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:37.542985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:37.543241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:37.544052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:37.546153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:37.695280 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1415/2822] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:37 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:37.699618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:37.724179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236daa-b5a1-48e3-a03e-f89f58d487e8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:37.967723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:37.970502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:37.970856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:37.971178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:37.971776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:37.977374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:38.053050 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1408/2823] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:37 2022] POST /identity/v3/users => generated 291 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:38.057826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:38.063645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:38.075520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ecb5a-a75e-4605-949e-7f343640dba8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:38.083914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:38.085079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:38.085558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:38.086605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:38.087441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:38.090473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:38.118604 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1416/2824] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:38.122908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:38.137897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e59c1-9162-4571-b43a-cee3cfe83101 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:38.154299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd71ae-3833-42b4-aa76-fcb8099d01a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:38.155819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd71ae-3833-42b4-aa76-fcb8099d01a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:38.156278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd71ae-3833-42b4-aa76-fcb8099d01a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:38.156703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd71ae-3833-42b4-aa76-fcb8099d01a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:38.157602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd71ae-3833-42b4-aa76-fcb8099d01a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:38.159414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd71ae-3833-42b4-aa76-fcb8099d01a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:38.185961 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1409/2825] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:38.188015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:38.205512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2398e-40bc-4741-a40b-0c8807e9271d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:38.218666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:38.219598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:38.220047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:38.220483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] PATH_INFO: `/v3/projects/920c712f8dd443e380d074519ae35378/users/198ccc5c01f041e0b7e255c428d83d06/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:38.255801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=920c712f8dd443e380d074519ae35378, user_id=198ccc5c01f041e0b7e255c428d83d06, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:38.258739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:38.288654 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1417/2826] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:38 2022] PUT /identity/v3/projects/920c712f8dd443e380d074519ae35378/users/198ccc5c01f041e0b7e255c428d83d06/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:38.298994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a8f46-46cd-4d70-9023-bd1f76718a1d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:38.299722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a8f46-46cd-4d70-9023-bd1f76718a1d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:38.300153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a8f46-46cd-4d70-9023-bd1f76718a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:38.370939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e64a8f46-46cd-4d70-9023-bd1f76718a1d None None] MFA Rules not processed for user `198ccc5c01f041e0b7e255c428d83d06`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:38.378305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e64a8f46-46cd-4d70-9023-bd1f76718a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:38.388071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:38.577300 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1410/2827] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:38.585762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:38.782540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a89d-2bdd-440e-9dba-b16e12cac3bb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:38.801102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:38.802023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:38.802300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:38.802571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:38.803219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:38.805191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:38.875843 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1418/2828] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:38 2022] POST /identity/v3/projects => generated 381 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:38.879964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a8f46-46cd-4d70-9023-bd1f76718a1d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:38.906879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a8f46-46cd-4d70-9023-bd1f76718a1d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:39.142156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:39.142841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:39.143277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:39.143774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:39.144604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:39.148830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:39.229100 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1411/2829] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:38 2022] POST /identity/v3/users => generated 297 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:39.231172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:39.248259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5bcf-8723-4976-b912-5a27f9bfdc5f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:39.260382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:39.261041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:39.261318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:39.261770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:39.262527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:39.264188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:39.297474 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1419/2830] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:39.301667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:39.316858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0e967-1aa6-48ea-b7d4-787f7fb7c3bb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:39.334561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:39.335552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:39.336003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:39.336441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] PATH_INFO: `/v3/projects/c5de86ae8e5e45a1ad9a025bb81bd6d7/users/bb4c939edeb84f5eb7d64001681372c5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:39.388345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5de86ae8e5e45a1ad9a025bb81bd6d7, user_id=bb4c939edeb84f5eb7d64001681372c5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:39.407081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:39.441077 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1412/2831] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:39 2022] PUT /identity/v3/projects/c5de86ae8e5e45a1ad9a025bb81bd6d7/users/bb4c939edeb84f5eb7d64001681372c5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:39.445369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:39.469774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae4de4-6e79-4e8d-aa8a-0f4b2e32d47a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:39.706668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fc0a0-e139-4dba-be7a-9ffa3e96288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:39.707340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fc0a0-e139-4dba-be7a-9ffa3e96288d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:39.707622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fc0a0-e139-4dba-be7a-9ffa3e96288d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:39.707926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fc0a0-e139-4dba-be7a-9ffa3e96288d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:39.708609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fc0a0-e139-4dba-be7a-9ffa3e96288d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:39.710276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fc0a0-e139-4dba-be7a-9ffa3e96288d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:39.749429 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1420/2832] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:39.753409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:39.773745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fab772-6a04-4384-910c-0db368a1af0d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:39.789231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:39.789963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:39.790431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:39.791078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] PATH_INFO: `/v3/projects/c5de86ae8e5e45a1ad9a025bb81bd6d7/users/bb4c939edeb84f5eb7d64001681372c5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:39.798583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5de86ae8e5e45a1ad9a025bb81bd6d7, user_id=bb4c939edeb84f5eb7d64001681372c5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:39.801911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:39.827465 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1413/2833] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:39 2022] PUT /identity/v3/projects/c5de86ae8e5e45a1ad9a025bb81bd6d7/users/bb4c939edeb84f5eb7d64001681372c5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:39.839278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:39.840339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:39.840613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:39.906643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] MFA Rules not processed for user `bb4c939edeb84f5eb7d64001681372c5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:39.908005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:39.969805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:40.061082 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1421/2834] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:40.340902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:40.343382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:40.475791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:40.515300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f83aa-46b8-46d0-b4a8-ed7fc091d29e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:40.524736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:40.525625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:40.526047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:40.526519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:40.539660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:40.542021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:40.556401 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1414/2835] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:41.745434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:41.748477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:41.906297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53856ba-9640-4486-9ff8-69ccd2d0fe98 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:41.917378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:41.918174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:41.918646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:41.919019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:41.926361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:41.964378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:41.972760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:42.008245 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1422/2836] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:42.457089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:42.532749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:42.544283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a79fd-1a22-4232-ba54-ff876a76dd6e service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:42.560386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea005a-4012-4f57-99af-aeb4cca2694c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:42.561008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea005a-4012-4f57-99af-aeb4cca2694c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:42.561452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea005a-4012-4f57-99af-aeb4cca2694c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:42.561926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea005a-4012-4f57-99af-aeb4cca2694c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:42.564356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9fea005a-4012-4f57-99af-aeb4cca2694c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:42.590458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea005a-4012-4f57-99af-aeb4cca2694c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:42.592193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea005a-4012-4f57-99af-aeb4cca2694c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:42.623480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7372a2-c33f-4415-aeeb-a562dccddb39 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:42.626493 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1423/2837] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:42.637503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:42.638165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:42.638431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:42.638697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] PATH_INFO: `/v3/users/198ccc5c01f041e0b7e255c428d83d06` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:42.639857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] RBAC: Authorizing `identity:delete_user(user_id=198ccc5c01f041e0b7e255c428d83d06)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:42.641855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:42.745152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198ccc5c01f041e0b7e255c428d83d06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:42.746059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198ccc5c01f041e0b7e255c428d83d06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:42.763857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198ccc5c01f041e0b7e255c428d83d06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:42.764633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198ccc5c01f041e0b7e255c428d83d06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:42.798079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198ccc5c01f041e0b7e255c428d83d06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:42.818435 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1415/2838] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:42 2022] DELETE /identity/v3/users/198ccc5c01f041e0b7e255c428d83d06 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:42.819851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:42.820345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:42.820445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:42.862795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:42.864140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:43.010401 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1424/2839] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:43.058545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:43.061234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:43.166747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7c722-30a8-4a86-8361-31cb6ee7f5db admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:43.174056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:43.174862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:43.175141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:43.175414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:43.191379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:43.194302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:43.209706 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1416/2840] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:45.322418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:45.330455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:45.378411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ceccac-42b9-424e-bebd-3a23cb89ba52 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:45.393066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:45.400716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:45.401185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:45.402651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] PATH_INFO: `/v3/projects/920c712f8dd443e380d074519ae35378` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:45.404701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=920c712f8dd443e380d074519ae35378)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:45.408465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:45.525341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:45.527975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '920c712f8dd443e380d074519ae35378', 'request_id': 'req-b107a16d-21b0-4f1d-a566-92af472ed950'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.550365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '920c712f8dd443e380d074519ae35378', 'request_id': 'req-b107a16d-21b0-4f1d-a566-92af472ed950'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.561920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '920c712f8dd443e380d074519ae35378', 'request_id': 'req-b107a16d-21b0-4f1d-a566-92af472ed950'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.585187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] The token cache is being invalidate because project 920c712f8dd443e380d074519ae35378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:45.586153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.587193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.593657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1425/2841] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:45 2022] DELETE /identity/v3/projects/920c712f8dd443e380d074519ae35378 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:45.598786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:45.615439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:45.624382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2b8ea-c52d-47d8-a102-c65af350c844 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:45.759435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:45.761183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:45.837677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:45.838452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:45.839984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:45.839984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] PATH_INFO: `/v3/users/bb4c939edeb84f5eb7d64001681372c5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:45.843652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4c939edeb84f5eb7d64001681372c5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:45.848465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:45.948438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4c939edeb84f5eb7d64001681372c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.949642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4c939edeb84f5eb7d64001681372c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.961633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4c939edeb84f5eb7d64001681372c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.963485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4c939edeb84f5eb7d64001681372c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:45.978089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107a16d-21b0-4f1d-a566-92af472ed950 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:45.985561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228efb41-106a-4188-896f-ada23cff7d52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:45.986239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228efb41-106a-4188-896f-ada23cff7d52 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:45.987000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228efb41-106a-4188-896f-ada23cff7d52 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:45.987280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228efb41-106a-4188-896f-ada23cff7d52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:45.990303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-228efb41-106a-4188-896f-ada23cff7d52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:46.004930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4c939edeb84f5eb7d64001681372c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:46.006581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228efb41-106a-4188-896f-ada23cff7d52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:46.008791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228efb41-106a-4188-896f-ada23cff7d52 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:46.022798 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1417/2842] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:45 2022] DELETE /identity/v3/users/bb4c939edeb84f5eb7d64001681372c5 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:46.035775 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1426/2843] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:47.605138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:47.607175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:47.792556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff546b-ebf0-4c49-8c93-087b9be513c8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:47.798739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:47.799452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:47.799940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:47.800287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] PATH_INFO: `/v3/projects/c5de86ae8e5e45a1ad9a025bb81bd6d7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:47.801528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5de86ae8e5e45a1ad9a025bb81bd6d7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:47.803870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:47.847655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:47.869727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5de86ae8e5e45a1ad9a025bb81bd6d7', 'request_id': 'req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:47.877167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5de86ae8e5e45a1ad9a025bb81bd6d7', 'request_id': 'req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:47.891291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5de86ae8e5e45a1ad9a025bb81bd6d7', 'request_id': 'req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:47.906090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] The token cache is being invalidate because project c5de86ae8e5e45a1ad9a025bb81bd6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:47.906384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:47.907025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74bd16bd-1d97-4de0-9fbe-5e9c2f6e2fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:47.908695 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1418/2844] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:47 2022] DELETE /identity/v3/projects/c5de86ae8e5e45a1ad9a025bb81bd6d7 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:47.914155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b2b4-9ac6-4af4-95ba-f45d18767df9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:47.914657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b2b4-9ac6-4af4-95ba-f45d18767df9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:47.914947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b2b4-9ac6-4af4-95ba-f45d18767df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:47.935242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3b01b2b4-9ac6-4af4-95ba-f45d18767df9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:47.936698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b01b2b4-9ac6-4af4-95ba-f45d18767df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:48.018178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1427/2845] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:48.026759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516d248-de42-49dc-8748-b53eb470eb88 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.027619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516d248-de42-49dc-8748-b53eb470eb88 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.028038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516d248-de42-49dc-8748-b53eb470eb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.049266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f516d248-de42-49dc-8748-b53eb470eb88 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:48.050649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f516d248-de42-49dc-8748-b53eb470eb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:48.075045 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1419/2846] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:48.078875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b2b4-9ac6-4af4-95ba-f45d18767df9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.092610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b2b4-9ac6-4af4-95ba-f45d18767df9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:48.108356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:48.109330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.109871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.110161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.110994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:48.111297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:48.114049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:48.153888 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1428/2847] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:48.155021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516d248-de42-49dc-8748-b53eb470eb88 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.166575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516d248-de42-49dc-8748-b53eb470eb88 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:48.171532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:48.172258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.172535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.172888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.173560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:48.175561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:48.212531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1420/2848] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:48 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:48.216293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.228758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1191c-cbab-462e-9bfb-afb7440883b3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:48.285129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:48.302168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:48.302840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.303236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.303911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.305031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:48.307306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:48.353263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1429/2849] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:48 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:48.357182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.368143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d7638-5ed4-4a94-9217-85a49c6d1cde admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:48.373187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:48.373835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.374114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.374383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.375034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:48.377103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:48.388259 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1421/2850] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:48.393764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.408734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7551aa-84f3-4d37-86f4-2d00e8fb543a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:48.414946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189b906-32f0-4d86-a96f-e878626ed6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:48.415617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189b906-32f0-4d86-a96f-e878626ed6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.416616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189b906-32f0-4d86-a96f-e878626ed6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.416966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189b906-32f0-4d86-a96f-e878626ed6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.417881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189b906-32f0-4d86-a96f-e878626ed6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:48.419932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189b906-32f0-4d86-a96f-e878626ed6d8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:48.456335 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1430/2851] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:48.460644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.478948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3895f-8bd3-48e6-803e-f9f5fc7a5806 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:48.488493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:48.489339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.489721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.490094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] PATH_INFO: `/v3/projects/eee008b613cd468f9b996e915ced3074/users/6fe33ab0e3394b7ea6b1b0b29f444e53/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.508412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee008b613cd468f9b996e915ced3074, user_id=6fe33ab0e3394b7ea6b1b0b29f444e53, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:48.511353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:48.539038 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1422/2852] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:48 2022] PUT /identity/v3/projects/eee008b613cd468f9b996e915ced3074/users/6fe33ab0e3394b7ea6b1b0b29f444e53/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:48.542239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243b3b0-44c6-40b3-9396-d1ab071304fe None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.542526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243b3b0-44c6-40b3-9396-d1ab071304fe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.544483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243b3b0-44c6-40b3-9396-d1ab071304fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.590535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c243b3b0-44c6-40b3-9396-d1ab071304fe None None] MFA Rules not processed for user `6fe33ab0e3394b7ea6b1b0b29f444e53`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:48.593524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c243b3b0-44c6-40b3-9396-d1ab071304fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:48.687964 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1431/2853] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:48.692608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.782122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a21c1-2cc4-4e47-b2ce-478a04cc6724 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:48.797673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:48.799091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:48.799530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:48.800567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:48.802752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:48.806152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:48.862845 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1423/2854] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:48 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:48.864206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243b3b0-44c6-40b3-9396-d1ab071304fe None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:48.881137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243b3b0-44c6-40b3-9396-d1ab071304fe None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:49.010811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:49.011506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:49.011711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:49.012043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:49.012688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:49.015289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:49.053959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1432/2855] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:48 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:49.058020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:49.070035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d601aa2-3889-4b9d-a2da-6fe264a68902 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:49.074928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:49.075573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:49.075852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:49.076119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:49.076784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:49.078536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:49.094614 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1424/2856] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:55:49.098794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:49.120187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409b41f-0041-4ac6-837f-dd8dea1705f8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:49.137039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:49.138110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:49.138110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:49.138110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] PATH_INFO: `/v3/projects/d937c43c346a4286ab80fda4c4601c89/users/491d2b684eb94a33a4b7a2b4d1bcaf49/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:49.169864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d937c43c346a4286ab80fda4c4601c89, user_id=491d2b684eb94a33a4b7a2b4d1bcaf49, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:49.172623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:49.197297 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1433/2857] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:49 2022] PUT /identity/v3/projects/d937c43c346a4286ab80fda4c4601c89/users/491d2b684eb94a33a4b7a2b4d1bcaf49/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:49.201276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:49.214703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350758d-0c5b-4efd-bec2-3e7a62e83656 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:49.242775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:49.316429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e7708-a493-4ffe-af83-9d4dc49cc786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:49.317386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e7708-a493-4ffe-af83-9d4dc49cc786 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:49.317759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e7708-a493-4ffe-af83-9d4dc49cc786 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:49.318227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e7708-a493-4ffe-af83-9d4dc49cc786 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:49.319115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e7708-a493-4ffe-af83-9d4dc49cc786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:49.320801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e7708-a493-4ffe-af83-9d4dc49cc786 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:49.334490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1425/2858] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:49.337784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:49.356222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b57d-95bc-4cbc-aca3-b4490c87b86c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:49.381896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f76b3-833b-4788-b673-445fef8c12de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:49.382591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f76b3-833b-4788-b673-445fef8c12de admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:49.382871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f76b3-833b-4788-b673-445fef8c12de admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:49.383144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f76b3-833b-4788-b673-445fef8c12de admin admin] PATH_INFO: `/v3/projects/d937c43c346a4286ab80fda4c4601c89/users/491d2b684eb94a33a4b7a2b4d1bcaf49/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:49.391330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f76b3-833b-4788-b673-445fef8c12de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d937c43c346a4286ab80fda4c4601c89, user_id=491d2b684eb94a33a4b7a2b4d1bcaf49, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:49.394868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f76b3-833b-4788-b673-445fef8c12de admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:49.443286 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1434/2859] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:49 2022] PUT /identity/v3/projects/d937c43c346a4286ab80fda4c4601c89/users/491d2b684eb94a33a4b7a2b4d1bcaf49/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:49.520438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:49.520796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:49.521716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:49.595084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] MFA Rules not processed for user `491d2b684eb94a33a4b7a2b4d1bcaf49`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:49.596501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:49.727318 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1426/2860] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:49.735812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7497dd-5e54-4a03-92fd-96edd0036139 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:49.736429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7497dd-5e54-4a03-92fd-96edd0036139 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:49.736903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7497dd-5e54-4a03-92fd-96edd0036139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:49.785206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3c7497dd-5e54-4a03-92fd-96edd0036139 None None] MFA Rules not processed for user `6fe33ab0e3394b7ea6b1b0b29f444e53`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:49.786631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3c7497dd-5e54-4a03-92fd-96edd0036139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:49.791191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:49.873257 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1435/2861] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:49.884957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:49.887224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:50.043227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00afd0-9a55-4d28-a921-48270e8854dc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:50.052958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:50.060097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:50.060790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:50.060790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:50.074945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:50.076573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:50.117402 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1427/2862] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:50.168459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e353789-9c38-4a20-b4a2-447a445c9396 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:50.168840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e353789-9c38-4a20-b4a2-447a445c9396 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:50.168946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e353789-9c38-4a20-b4a2-447a445c9396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:50.189663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5e353789-9c38-4a20-b4a2-447a445c9396 None None] MFA Rules not processed for user `f1134231b4a545fdb136a5204cb4f289`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:50.190898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e353789-9c38-4a20-b4a2-447a445c9396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:50.282157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1436/2863] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:50 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:53.081683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:53.082348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:53.164423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:53.246064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9108a-3356-4df0-9835-c73395926d25 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:53.260357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:53.261018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:53.261293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:53.261620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] PATH_INFO: `/v3/users/226a87a1f393420b8d19ec2390e9d063` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:53.263122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] RBAC: Authorizing `identity:delete_user(user_id=226a87a1f393420b8d19ec2390e9d063)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:53.265061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:53.333174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226a87a1f393420b8d19ec2390e9d063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.333847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226a87a1f393420b8d19ec2390e9d063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.346527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226a87a1f393420b8d19ec2390e9d063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.347303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226a87a1f393420b8d19ec2390e9d063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.364752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226a87a1f393420b8d19ec2390e9d063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.381801 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1428/2864] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:53 2022] DELETE /identity/v3/users/226a87a1f393420b8d19ec2390e9d063 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:53.386145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:53.386647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:53.386748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:53.411251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:53.414501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:53.509172 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1437/2865] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:53.552122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:53.553784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:53.561169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:53.568303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:53.590424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316782e1-32bc-4ce1-a10d-a14b94258e99 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:53.604835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:53.605500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:53.605798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:53.606034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] PATH_INFO: `/v3/users/6f7681de64c248268da4d78124f59b54` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:53.607304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7681de64c248268da4d78124f59b54)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:53.609307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:53.677653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d3b5d-f143-4871-a7e9-f260905e885c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:53.700355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:53.701547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:53.701887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7681de64c248268da4d78124f59b54'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.703683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:53.705048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:53.709326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:53.719846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7681de64c248268da4d78124f59b54'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.729504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:53.731748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:53.734633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7681de64c248268da4d78124f59b54'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.735854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7681de64c248268da4d78124f59b54'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.737063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7681de64c248268da4d78124f59b54'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:53.765034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1438/2866] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:53 2022] DELETE /identity/v3/users/6f7681de64c248268da4d78124f59b54 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:53.765626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1429/2867] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:54.355006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:54.356804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:54.516523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8e383-2aa2-490f-89bd-6c8109689e49 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:54.533612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:54.534354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:54.534910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:54.535282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] PATH_INFO: `/v3/projects/26d60144477f4a5192d581350d678979` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:54.536547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d60144477f4a5192d581350d678979)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:54.538829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:54.630968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:54.633339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:54.641715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d60144477f4a5192d581350d678979', 'request_id': 'req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:54.656157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e561ff-793f-47f4-9635-02fd605ce65b service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:54.662565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d60144477f4a5192d581350d678979', 'request_id': 'req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:54.675695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d60144477f4a5192d581350d678979', 'request_id': 'req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:54.717199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] The token cache is being invalidate because project 26d60144477f4a5192d581350d678979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:54.717442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:54.718208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5469eb5f-31c0-4e0e-9f75-93c3f6316f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:54.723856 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1439/2868] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:54 2022] DELETE /identity/v3/projects/26d60144477f4a5192d581350d678979 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:54.730296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bda43-99c4-4db8-a937-9d3f42db16bb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:54.730581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bda43-99c4-4db8-a937-9d3f42db16bb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:54.730859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bda43-99c4-4db8-a937-9d3f42db16bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:54.774691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aa1bda43-99c4-4db8-a937-9d3f42db16bb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:54.776547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa1bda43-99c4-4db8-a937-9d3f42db16bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:54.820528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:54.856193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:54.857179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:54.857619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:54.858340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] PATH_INFO: `/v3/projects/07ce9fd746374b478a743261eb014e5c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:54.862523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ce9fd746374b478a743261eb014e5c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:54.867692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:54.920844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1440/2869] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:55.005698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42413-989e-406f-8722-5f956a458c46 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.005989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42413-989e-406f-8722-5f956a458c46 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.006265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42413-989e-406f-8722-5f956a458c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.024505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:55.070531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c5f42413-989e-406f-8722-5f956a458c46 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:55.073082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c5f42413-989e-406f-8722-5f956a458c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:55.078103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ce9fd746374b478a743261eb014e5c', 'request_id': 'req-0556bfc4-5e74-47ee-9f02-79cdc54f0196'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.089155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ce9fd746374b478a743261eb014e5c', 'request_id': 'req-0556bfc4-5e74-47ee-9f02-79cdc54f0196'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.108745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ce9fd746374b478a743261eb014e5c', 'request_id': 'req-0556bfc4-5e74-47ee-9f02-79cdc54f0196'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.135227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] The token cache is being invalidate because project 07ce9fd746374b478a743261eb014e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:55.139295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.142782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.147026 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1430/2870] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:54 2022] DELETE /identity/v3/projects/07ce9fd746374b478a743261eb014e5c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:55.156630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:55.164104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:55.229193 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1441/2871] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:55.233150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42413-989e-406f-8722-5f956a458c46 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:55.255850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42413-989e-406f-8722-5f956a458c46 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:55.269727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:55.270742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.270742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.270955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.271876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:55.272314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:55.273971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:55.304781 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1442/2872] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:55:55.309635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:55.334305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556bfc4-5e74-47ee-9f02-79cdc54f0196 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:55.338332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f263-4b89-449d-bf9f-22ea36d7e012 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:55.341656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:55.342740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.343460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.343998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] PATH_INFO: `/v3/users/27c681039283443c8dd284f22277bc8f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.345916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c681039283443c8dd284f22277bc8f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:55.348839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:55.356957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:55.360316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.360548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.360807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.363669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:55.365817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:55.414676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1443/2873] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:55 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:55.419107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:55.441588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c681039283443c8dd284f22277bc8f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.442868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c681039283443c8dd284f22277bc8f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.459942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c681039283443c8dd284f22277bc8f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.460990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c681039283443c8dd284f22277bc8f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.487578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c681039283443c8dd284f22277bc8f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:55.524034 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1431/2874] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:55:55 2022] DELETE /identity/v3/users/27c681039283443c8dd284f22277bc8f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:55.595721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6251ec-c12d-4357-a5fe-e1d85a96fb24 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:55.733237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:55.735195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.735768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.736563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.739060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:55.741504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:55.777681 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1444/2875] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:55 2022] POST /identity/v3/users => generated 287 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:55:55.782336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:55.783779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:55.795358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:55.809730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ab14e-dc66-4d76-921a-5981d336e263 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:55.816378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:55.817866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.818244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.818661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.819421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:55.821144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:55.840000 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1432/2876] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:55.840315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:55.856331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:55.856331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442af4-23d5-4198-bbb9-efdfee1dca15 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:55.863882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e1e09-37a6-468a-8df5-1d9f19481775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:55.871571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1e09-37a6-468a-8df5-1d9f19481775 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.871571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1e09-37a6-468a-8df5-1d9f19481775 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.871571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1e09-37a6-468a-8df5-1d9f19481775 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.871571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e1e09-37a6-468a-8df5-1d9f19481775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:55.875994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e1e09-37a6-468a-8df5-1d9f19481775 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:55.894975 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1445/2877] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:55.900746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:55.916293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62158b-5f81-4633-8f24-5597346bfcc2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:55.926358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:55.937015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:55.939641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:55.939954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] PATH_INFO: `/v3/projects/400242d681ab472d9c90de4fc7da202b/users/bc22daf24ce44eb28bb6e000316e4df4/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:55.994933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=400242d681ab472d9c90de4fc7da202b, user_id=bc22daf24ce44eb28bb6e000316e4df4, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:55.997714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:56.025417 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1433/2878] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:55 2022] PUT /identity/v3/projects/400242d681ab472d9c90de4fc7da202b/users/bc22daf24ce44eb28bb6e000316e4df4/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:56.026653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.026939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.028071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.058572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:56.059829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:56.156229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:56.158963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:56.197047 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1446/2879] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:56.246248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:56.247681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:56.280274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df868d1-d96b-4c20-8ec8-854f2eb9d703 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:56.314449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:56.315530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.316034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.316523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] PATH_INFO: `/v3/projects/cbe06cc27b4f403faac35ee3521893ea` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.318831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe06cc27b4f403faac35ee3521893ea)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:56.322314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:56.401067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc718d-29ae-4c1f-882c-cf84403d2733 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:56.437834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe06cc27b4f403faac35ee3521893ea', 'request_id': 'req-8705388f-ad00-4d0f-8561-189c1f51f898'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:56.468648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe06cc27b4f403faac35ee3521893ea', 'request_id': 'req-8705388f-ad00-4d0f-8561-189c1f51f898'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:56.496202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe06cc27b4f403faac35ee3521893ea', 'request_id': 'req-8705388f-ad00-4d0f-8561-189c1f51f898'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:56.519586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16141138-04e6-4041-895a-304d3610c4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:56.520226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141138-04e6-4041-895a-304d3610c4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.520511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141138-04e6-4041-895a-304d3610c4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.520788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141138-04e6-4041-895a-304d3610c4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.528425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] The token cache is being invalidate because project cbe06cc27b4f403faac35ee3521893ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:55:56.528810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:56.529722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8705388f-ad00-4d0f-8561-189c1f51f898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:55:56.531695 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1434/2880] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:55:56 2022] DELETE /identity/v3/projects/cbe06cc27b4f403faac35ee3521893ea => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:55:56.536962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4eb51-e30c-42b4-a97f-53921461bc96 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.537348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4eb51-e30c-42b4-a97f-53921461bc96 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.537847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4eb51-e30c-42b4-a97f-53921461bc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.579448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-20d4eb51-e30c-42b4-a97f-53921461bc96 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:56.580843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-20d4eb51-e30c-42b4-a97f-53921461bc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:56.646027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16141138-04e6-4041-895a-304d3610c4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:56.648007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16141138-04e6-4041-895a-304d3610c4f5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:56.653841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:56.659963 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1447/2881] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:55:56.687778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:56.718031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1435/2882] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:56.720721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997135-1eac-4666-9cba-22af3befcba3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.721093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997135-1eac-4666-9cba-22af3befcba3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.722112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997135-1eac-4666-9cba-22af3befcba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.748080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-dc997135-1eac-4666-9cba-22af3befcba3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:56.749497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dc997135-1eac-4666-9cba-22af3befcba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:56.760184 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1448/2883] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:56.764793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4eb51-e30c-42b4-a97f-53921461bc96 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:56.776333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4eb51-e30c-42b4-a97f-53921461bc96 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:56.781702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:56.782424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.782992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.783669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.784567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:56.784966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:56.787174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:56.802741 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1436/2884] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:56.807216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997135-1eac-4666-9cba-22af3befcba3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:56.819054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997135-1eac-4666-9cba-22af3befcba3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:56.826176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:56.826868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.827145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.827474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.828115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:56.830508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:56.868466 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1449/2885] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:56 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:56.873042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:56.885781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d72d9-ed64-476d-8419-596182fd7389 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:56.964498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:56.965273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:56.965828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:56.966219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:56.966955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:56.968965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.025710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:57.032499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1437/2886] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:56 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:57.045851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4725bd0-b768-4ed1-8783-0793baa6290a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:57.061577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:57.062396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.073282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.077249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.078157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:57.082820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.113524 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1450/2887] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:55:57.125321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:57.140683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794728b-a4ed-4b0b-a193-6c3c7e3c3e9f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:57.158534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e167f06-61d7-4162-a283-df7a57275d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:57.161806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e167f06-61d7-4162-a283-df7a57275d51 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.162965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e167f06-61d7-4162-a283-df7a57275d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.163343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e167f06-61d7-4162-a283-df7a57275d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.165361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e167f06-61d7-4162-a283-df7a57275d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:57.171400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e167f06-61d7-4162-a283-df7a57275d51 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.209936 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1438/2888] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:57.214951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:57.230641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54154509-b6a1-44a5-b3c3-af62b2964c41 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:57.237584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:57.238752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.239224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.239610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] PATH_INFO: `/v3/projects/f71e8dabd2374f829279fdcadd7b904e/users/632da36c30b3422e996e3f2033869c8d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.260887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71e8dabd2374f829279fdcadd7b904e, user_id=632da36c30b3422e996e3f2033869c8d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:57.264104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.299584 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1451/2889] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:57 2022] PUT /identity/v3/projects/f71e8dabd2374f829279fdcadd7b904e/users/632da36c30b3422e996e3f2033869c8d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:57.302789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cae67-a5e6-4eb4-9fee-b48e1ca0cf33 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.303248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cae67-a5e6-4eb4-9fee-b48e1ca0cf33 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.303704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cae67-a5e6-4eb4-9fee-b48e1ca0cf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.335741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-392cae67-a5e6-4eb4-9fee-b48e1ca0cf33 None None] MFA Rules not processed for user `632da36c30b3422e996e3f2033869c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:57.337941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-392cae67-a5e6-4eb4-9fee-b48e1ca0cf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:57.456709 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1439/2890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:55:57.461391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:57.571238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1c071-8074-4f64-9a08-d683b205323e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:57.579168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:57.580783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.581299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.583460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.584399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:57.587381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.642603 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1452/2891] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:55:57 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:57.649214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cae67-a5e6-4eb4-9fee-b48e1ca0cf33 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:57.676221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cae67-a5e6-4eb4-9fee-b48e1ca0cf33 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:57.765499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:57.783320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:57.785585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.786112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.786770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.787626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:57.790588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.833155 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1440/2892] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:57 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:57.835792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:57.855173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9d247-7b5d-44c3-accf-2d335ca98ee2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:57.861181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:57.861875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.862209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.862413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.863288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:57.865129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.882771 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1453/2893] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:57.886774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:57.900962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81fd85-7732-4361-a6a0-dd6cd22f5e11 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:57.906786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:57.907469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:57.907841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:57.908123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] PATH_INFO: `/v3/projects/cdcb250378084ff2a62c36a8c836ea60/users/c04b2fdb31bd4fcea2fa8edac17600f2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:57.956919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdcb250378084ff2a62c36a8c836ea60, user_id=c04b2fdb31bd4fcea2fa8edac17600f2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:57.961045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:57.993147 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1441/2894] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:57 2022] PUT /identity/v3/projects/cdcb250378084ff2a62c36a8c836ea60/users/c04b2fdb31bd4fcea2fa8edac17600f2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:57.998204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:58.043721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0954dd-8a55-4321-b1df-92a2479fcd32 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:58.192348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228352b-e170-4571-89a1-0ba2abd5eafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:58.203967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228352b-e170-4571-89a1-0ba2abd5eafb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:58.204313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228352b-e170-4571-89a1-0ba2abd5eafb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:58.204515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228352b-e170-4571-89a1-0ba2abd5eafb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:58.205205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228352b-e170-4571-89a1-0ba2abd5eafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:58.210410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228352b-e170-4571-89a1-0ba2abd5eafb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:58.243257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:58.244317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1454/2895] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:58.258885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc50bbb-9c45-474d-b12e-a824a41d0462 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:58.270749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:58.272044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:58.272950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:58.273536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] PATH_INFO: `/v3/projects/cdcb250378084ff2a62c36a8c836ea60/users/c04b2fdb31bd4fcea2fa8edac17600f2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:58.278111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdcb250378084ff2a62c36a8c836ea60, user_id=c04b2fdb31bd4fcea2fa8edac17600f2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:58.283259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:58.313936 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1442/2896] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:55:58 2022] PUT /identity/v3/projects/cdcb250378084ff2a62c36a8c836ea60/users/c04b2fdb31bd4fcea2fa8edac17600f2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:55:58.325331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4d226-9da9-49a4-8e6a-119d6bcd28ca None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:58.325688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4d226-9da9-49a4-8e6a-119d6bcd28ca None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:58.326040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4d226-9da9-49a4-8e6a-119d6bcd28ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:58.352240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:58.393305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-33b4d226-9da9-49a4-8e6a-119d6bcd28ca None None] MFA Rules not processed for user `c04b2fdb31bd4fcea2fa8edac17600f2`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:58.394722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-33b4d226-9da9-49a4-8e6a-119d6bcd28ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:58.599303 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1455/2897] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:55:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:55:59.508241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:59.510003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:59.618104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2073f3-8297-4530-9f41-3bc88ccf70ed admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:59.631376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eb2da-65df-4e26-9b59-3251f17d62d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:59.641588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eb2da-65df-4e26-9b59-3251f17d62d5 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:59.642243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eb2da-65df-4e26-9b59-3251f17d62d5 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:59.643034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eb2da-65df-4e26-9b59-3251f17d62d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:59.664105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eb2da-65df-4e26-9b59-3251f17d62d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:59.666246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eb2da-65df-4e26-9b59-3251f17d62d5 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:59.682447 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1443/2898] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:55:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:01.668762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4d226-9da9-49a4-8e6a-119d6bcd28ca None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:01.752136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4d226-9da9-49a4-8e6a-119d6bcd28ca None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:01.757873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:01.758530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:01.758813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:01.759087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] PATH_INFO: `/v3/users/632da36c30b3422e996e3f2033869c8d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:01.760286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=632da36c30b3422e996e3f2033869c8d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:01.763577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:01.855221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632da36c30b3422e996e3f2033869c8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:01.869693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632da36c30b3422e996e3f2033869c8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:01.890039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632da36c30b3422e996e3f2033869c8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:01.891239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632da36c30b3422e996e3f2033869c8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:01.891949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632da36c30b3422e996e3f2033869c8d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:01.914329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1456/2899] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:56:01 2022] DELETE /identity/v3/users/632da36c30b3422e996e3f2033869c8d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:56:01.918701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:01.920259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:01.920724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:01.945820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:01.947070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:02.068820 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1444/2900] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:02.087318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:02.089064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:02.148853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:02.286001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e0b38-d71a-4a98-b5bb-faf08987e0de admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:02.290505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9b4bd-ee9f-4efd-b03e-d94369ef7fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:02.291172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9b4bd-ee9f-4efd-b03e-d94369ef7fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:02.291448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9b4bd-ee9f-4efd-b03e-d94369ef7fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:02.291897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9b4bd-ee9f-4efd-b03e-d94369ef7fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:02.314258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9b4bd-ee9f-4efd-b03e-d94369ef7fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:02.317965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9b4bd-ee9f-4efd-b03e-d94369ef7fd3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:02.331307 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1457/2901] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:04.377880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:04.383444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:04.421234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df4e46-a29c-4324-b6df-48a66f20602e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:04.448109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:04.452357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:04.454398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:04.455751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] PATH_INFO: `/v3/projects/f71e8dabd2374f829279fdcadd7b904e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:04.466783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71e8dabd2374f829279fdcadd7b904e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:04.478288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:04.592342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:04.593287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:04.593619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:04.631593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:04.696677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:04.697975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71e8dabd2374f829279fdcadd7b904e', 'request_id': 'req-d1d080de-0f9d-4f7b-9045-74cde9ce1970'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:04.699415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:04.712624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71e8dabd2374f829279fdcadd7b904e', 'request_id': 'req-d1d080de-0f9d-4f7b-9045-74cde9ce1970'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:04.731742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71e8dabd2374f829279fdcadd7b904e', 'request_id': 'req-d1d080de-0f9d-4f7b-9045-74cde9ce1970'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:04.758214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] The token cache is being invalidate because project f71e8dabd2374f829279fdcadd7b904e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:56:04.758481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:04.761315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:04.770141 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1445/2902] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:56:04 2022] DELETE /identity/v3/projects/f71e8dabd2374f829279fdcadd7b904e => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:56:04.771488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:04.774469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:04.836161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1458/2903] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:56:04.918739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d080de-0f9d-4f7b-9045-74cde9ce1970 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:04.938810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:04.939500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:04.941756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:04.942058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] PATH_INFO: `/v3/users/c04b2fdb31bd4fcea2fa8edac17600f2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:04.949495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c04b2fdb31bd4fcea2fa8edac17600f2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:04.951900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:05.070216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c04b2fdb31bd4fcea2fa8edac17600f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:05.072207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c04b2fdb31bd4fcea2fa8edac17600f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:05.091320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c04b2fdb31bd4fcea2fa8edac17600f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:05.092229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c04b2fdb31bd4fcea2fa8edac17600f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:05.099734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:05.101381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:05.118136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c04b2fdb31bd4fcea2fa8edac17600f2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:05.154849 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1446/2904] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:56:04 2022] DELETE /identity/v3/users/c04b2fdb31bd4fcea2fa8edac17600f2 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:56:05.297986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9a495-700f-45d4-9428-8cc99af66959 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:05.397633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:05.426928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf0ae8-baa5-459a-9005-30cd2eaede03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:05.429502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0ae8-baa5-459a-9005-30cd2eaede03 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:05.429802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0ae8-baa5-459a-9005-30cd2eaede03 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:05.430079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0ae8-baa5-459a-9005-30cd2eaede03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:05.442903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e9bf0ae8-baa5-459a-9005-30cd2eaede03 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:05.566107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf0ae8-baa5-459a-9005-30cd2eaede03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:05.568272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf0ae8-baa5-459a-9005-30cd2eaede03 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:05.592912 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1459/2905] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:56:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:56:05.911753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:05.914121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:06.036224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:06.109860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecd969-bd02-4d58-b6c3-c6c9567b696f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:06.125623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:06.126326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:06.126614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:06.126887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] PATH_INFO: `/v3/projects/cdcb250378084ff2a62c36a8c836ea60` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:06.128268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdcb250378084ff2a62c36a8c836ea60)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:06.130296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:06.252435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdcb250378084ff2a62c36a8c836ea60', 'request_id': 'req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:06.269971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdcb250378084ff2a62c36a8c836ea60', 'request_id': 'req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:06.290140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdcb250378084ff2a62c36a8c836ea60', 'request_id': 'req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:06.315283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] The token cache is being invalidate because project cdcb250378084ff2a62c36a8c836ea60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:56:06.316962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:06.318705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d0b1043-af3d-4eb9-bde7-191b1e91e6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:06.321351 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1447/2906] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:56:05 2022] DELETE /identity/v3/projects/cdcb250378084ff2a62c36a8c836ea60 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:56:06.326254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10077525-9e48-434e-92ac-0db8277fbc5c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:06.326783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10077525-9e48-434e-92ac-0db8277fbc5c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:06.327217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10077525-9e48-434e-92ac-0db8277fbc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:06.370994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-10077525-9e48-434e-92ac-0db8277fbc5c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:06.372454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10077525-9e48-434e-92ac-0db8277fbc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:06.592329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1460/2907] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:06.594049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1584b5-6158-4c55-9795-734861d838ee None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:06.594512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1584b5-6158-4c55-9795-734861d838ee None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:06.594965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1584b5-6158-4c55-9795-734861d838ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:06.668467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-aa1584b5-6158-4c55-9795-734861d838ee None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:06.669753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aa1584b5-6158-4c55-9795-734861d838ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:06.716573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1448/2908] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:56:06.721397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10077525-9e48-434e-92ac-0db8277fbc5c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:06.781231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10077525-9e48-434e-92ac-0db8277fbc5c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:06.817805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:06.819797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:06.820088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:06.822670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:06.825539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:56:06.825927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:06.833883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:06.888215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1584b5-6158-4c55-9795-734861d838ee None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:06.890997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1461/2909] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:56:06.908419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1584b5-6158-4c55-9795-734861d838ee None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:06.918482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:06.919201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:06.919383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:06.919828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:06.923014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:06.925933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:07.003509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1449/2910] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:56:06 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:56:07.007424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:07.033250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad09c20-95dd-44c5-a24a-ef1265bec6e6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:07.199053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:07.199702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:07.199968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:07.200241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:07.200897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:07.212470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:07.287798 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1462/2911] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:56:07 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:56:07.291360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:07.306346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e5358-be2d-41a8-8278-9b5f2c71dbf6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:07.323741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:07.324774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:07.325291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:07.325749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:07.326738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:07.328646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:07.351165 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1450/2912] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:56:07.355348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:07.372891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ba2f9-d08e-452f-bd43-34031b7d6dfe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:07.387507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf7026-2c0b-472c-9198-c80a81ce3656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:07.388745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf7026-2c0b-472c-9198-c80a81ce3656 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:07.389029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf7026-2c0b-472c-9198-c80a81ce3656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:07.389301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf7026-2c0b-472c-9198-c80a81ce3656 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:07.390000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf7026-2c0b-472c-9198-c80a81ce3656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:07.391646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf7026-2c0b-472c-9198-c80a81ce3656 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:07.406209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:07.423937 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1463/2913] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:56:07.428184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:07.445768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e10b5b-c660-475b-bdef-cfac2778d6e3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:07.455722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:07.456430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:07.456745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:07.457039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] PATH_INFO: `/v3/projects/84caeb4c4b2e4dbda9bf7def13420af1/users/4d9c7f4729e04bf6b915246ca7b5a3cf/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:07.493902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84caeb4c4b2e4dbda9bf7def13420af1, user_id=4d9c7f4729e04bf6b915246ca7b5a3cf, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:07.501915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:07.562852 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1451/2914] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:56:07 2022] PUT /identity/v3/projects/84caeb4c4b2e4dbda9bf7def13420af1/users/4d9c7f4729e04bf6b915246ca7b5a3cf/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:56:07.566774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3879eb-9e27-4d64-8e14-9cda2fd51a41 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:07.568451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3879eb-9e27-4d64-8e14-9cda2fd51a41 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:07.568451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3879eb-9e27-4d64-8e14-9cda2fd51a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:07.660101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bc3879eb-9e27-4d64-8e14-9cda2fd51a41 None None] MFA Rules not processed for user `4d9c7f4729e04bf6b915246ca7b5a3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:07.664309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bc3879eb-9e27-4d64-8e14-9cda2fd51a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:07.726153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:07.727351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:07.853427 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1464/2915] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:56:07.857676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3879eb-9e27-4d64-8e14-9cda2fd51a41 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:07.899907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:07.963540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ead97-7ca5-4c11-9ba4-15a3d1e52669 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:07.983862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:07.985776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:07.986059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:07.986324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:07.994394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:08.022631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.024261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.049430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3879eb-9e27-4d64-8e14-9cda2fd51a41 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:08.061937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:08.063148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.064215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.064708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.072194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.083676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1452/2916] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:56:08.087294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.171953 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1465/2917] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:56:07 2022] POST /identity/v3/projects => generated 389 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:56:08.176020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:08.189146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ddae7-a7c6-4ef8-a324-f597ba915dfb service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:08.317073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:08.318092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.318602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.319475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.320329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.323020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.349154 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1453/2918] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:56:08 2022] POST /identity/v3/users => generated 301 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:56:08.352867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:08.367649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8f8d9-2dd6-42a4-8fbf-00cd8078e1f7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:08.380697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:08.381412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.381926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.382535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.383723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.385646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.410030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1466/2919] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:56:08.412709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:08.425195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3a601-ea90-4ba7-9fb6-a1e13c177fce admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:08.445362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:08.446778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.447138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.447516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] PATH_INFO: `/v3/projects/eddfa4151eed49beafb4a87d152dc8c4/users/1f386dc88f214f489cb4105c2f6c50b0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.497118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddfa4151eed49beafb4a87d152dc8c4, user_id=1f386dc88f214f489cb4105c2f6c50b0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.500932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.532779 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1454/2920] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:56:08 2022] PUT /identity/v3/projects/eddfa4151eed49beafb4a87d152dc8c4/users/1f386dc88f214f489cb4105c2f6c50b0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:56:08.536679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:08.551123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634017-a4ea-463c-bbc8-01cb435f7b45 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:08.608492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:08.610989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:08.691064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:08.691975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.692143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.695631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.695631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.700659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.724376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:08.743015 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1467/2921] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:56:08.745370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:08.775943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345ac05-9312-4aeb-901b-64f9fed4e9bc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:08.796155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed73e60-da5f-4717-b8fe-76c8ebe2dc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:08.796859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73e60-da5f-4717-b8fe-76c8ebe2dc9c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.797128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73e60-da5f-4717-b8fe-76c8ebe2dc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.797398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed73e60-da5f-4717-b8fe-76c8ebe2dc9c admin admin] PATH_INFO: `/v3/projects/eddfa4151eed49beafb4a87d152dc8c4/users/1f386dc88f214f489cb4105c2f6c50b0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.800623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aab6c6-86bc-49cd-9d0a-33aa9e7ed67c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:08.806542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed73e60-da5f-4717-b8fe-76c8ebe2dc9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddfa4151eed49beafb4a87d152dc8c4, user_id=1f386dc88f214f489cb4105c2f6c50b0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.809256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed73e60-da5f-4717-b8fe-76c8ebe2dc9c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.816972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d16f46-b9f0-44cd-b007-8c0d1e0dc0ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:08.817636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16f46-b9f0-44cd-b007-8c0d1e0dc0ea service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.817932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16f46-b9f0-44cd-b007-8c0d1e0dc0ea service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.819998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16f46-b9f0-44cd-b007-8c0d1e0dc0ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.825396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d0d16f46-b9f0-44cd-b007-8c0d1e0dc0ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:08.848447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1468/2922] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:56:08 2022] PUT /identity/v3/projects/eddfa4151eed49beafb4a87d152dc8c4/users/1f386dc88f214f489cb4105c2f6c50b0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:56:08.853253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d16f46-b9f0-44cd-b007-8c0d1e0dc0ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:08.855135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d16f46-b9f0-44cd-b007-8c0d1e0dc0ea service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:08.858416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0779b-e1d8-4feb-b4d3-272d91ea67e3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:08.858908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0779b-e1d8-4feb-b4d3-272d91ea67e3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:08.859381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0779b-e1d8-4feb-b4d3-272d91ea67e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:08.885236 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1455/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:56:08.932047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-86e0779b-e1d8-4feb-b4d3-272d91ea67e3 None None] MFA Rules not processed for user `1f386dc88f214f489cb4105c2f6c50b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:08.933697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-86e0779b-e1d8-4feb-b4d3-272d91ea67e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:09.101540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1469/2924] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:56:09.105583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:09.106270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:09.106270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:09.138698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] MFA Rules not processed for user `4d9c7f4729e04bf6b915246ca7b5a3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:09.141542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:09.218586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:09.263535 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1456/2925] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:10.903106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:10.903628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:10.904407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:10.952112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] MFA Rules not processed for user `bc22daf24ce44eb28bb6e000316e4df4`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:10.954164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:11.062379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:11.064515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:11.078802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1470/2926] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:11.090526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:11.095974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:11.254317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563a0d-5612-4a9b-bbe2-bef5c5570275 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:11.262993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:11.263794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:11.265041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:11.265413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:11.280480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:11.285951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:11.316398 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1471/2927] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:11.394307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729335a-f709-41f4-9b5c-ef582a10f7f7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:11.435309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b2e5d-1bcf-471b-9afa-6f0620f8c705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:11.437716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b2e5d-1bcf-471b-9afa-6f0620f8c705 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:11.438214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b2e5d-1bcf-471b-9afa-6f0620f8c705 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:11.441157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b2e5d-1bcf-471b-9afa-6f0620f8c705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:11.465611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b2e5d-1bcf-471b-9afa-6f0620f8c705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:11.468305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b2e5d-1bcf-471b-9afa-6f0620f8c705 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:11.494455 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1457/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:14.743178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:14.755055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:14.770876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412dc5-0057-4b65-b5cc-148b1b73c669 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:14.802142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699baeb-db30-44e4-ac5b-a2c5d3226c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:14.805059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699baeb-db30-44e4-ac5b-a2c5d3226c7e service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:14.807075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699baeb-db30-44e4-ac5b-a2c5d3226c7e service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:14.809448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699baeb-db30-44e4-ac5b-a2c5d3226c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:14.818005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8699baeb-db30-44e4-ac5b-a2c5d3226c7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:15.028645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699baeb-db30-44e4-ac5b-a2c5d3226c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:15.030518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699baeb-db30-44e4-ac5b-a2c5d3226c7e service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:15.053401 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1472/2929] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:16.129874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d3b6-e172-4998-be63-33f3fb4c5de3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:16.130684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d3b6-e172-4998-be63-33f3fb4c5de3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:16.131204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216d3b6-e172-4998-be63-33f3fb4c5de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:16.212994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aacc-a2a3-4c01-8d40-ac65a8c347cf None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:16.213656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aacc-a2a3-4c01-8d40-ac65a8c347cf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:16.213656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aacc-a2a3-4c01-8d40-ac65a8c347cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:16.236492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2216d3b6-e172-4998-be63-33f3fb4c5de3 None None] MFA Rules not processed for user `bc22daf24ce44eb28bb6e000316e4df4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:16.243080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2216d3b6-e172-4998-be63-33f3fb4c5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:16.288546 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1458/2930] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:16.289816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5bd1aacc-a2a3-4c01-8d40-ac65a8c347cf None None] MFA Rules not processed for user `6fe33ab0e3394b7ea6b1b0b29f444e53`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:16.291284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5bd1aacc-a2a3-4c01-8d40-ac65a8c347cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:16.532992 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1473/2931] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:56:17.841108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216d3b6-e172-4998-be63-33f3fb4c5de3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:17.887916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216d3b6-e172-4998-be63-33f3fb4c5de3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:17.914121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:17.914761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:17.915036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:17.915305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:17.928961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:17.986322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:17.987943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:18.051683 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1459/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:26.511204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=53 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:28.005974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=50 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:33.259097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694a039-721a-4954-b5e0-a6de61c57dce None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:33.259675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694a039-721a-4954-b5e0-a6de61c57dce None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:33.259675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694a039-721a-4954-b5e0-a6de61c57dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:33.279727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0694a039-721a-4954-b5e0-a6de61c57dce None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:33.280922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0694a039-721a-4954-b5e0-a6de61c57dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:33.337428 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1474/2933] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 20:56:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:33.350191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:33.351656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:33.423797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595aed0-edf8-4f6c-8e20-db0c087db90f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:33.429513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:33.430138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:33.430414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:33.430685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:33.441244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:33.442922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:33.453944 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1460/2934] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:56:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:56:34.247611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939e61-5a4e-481d-beb1-3a3d91e4639b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:34.248351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939e61-5a4e-481d-beb1-3a3d91e4639b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:34.248858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939e61-5a4e-481d-beb1-3a3d91e4639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:34.278028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2c939e61-5a4e-481d-beb1-3a3d91e4639b None None] MFA Rules not processed for user `6e33e0b756e14af28a90cc5c76d346e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:34.282569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2c939e61-5a4e-481d-beb1-3a3d91e4639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:34.374034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1475/2935] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:38.618023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:38.632973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f3a6f-688d-4ae5-aff8-78935fad5639 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:38.640085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614cbff-c1d7-4421-90ee-a315d305dad1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:38.640689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614cbff-c1d7-4421-90ee-a315d305dad1 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:38.640960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614cbff-c1d7-4421-90ee-a315d305dad1 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:38.641222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614cbff-c1d7-4421-90ee-a315d305dad1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:38.651428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614cbff-c1d7-4421-90ee-a315d305dad1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:38.653274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614cbff-c1d7-4421-90ee-a315d305dad1 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:38.665114 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1461/2936] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:38.718200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae852d3-ef21-4631-b322-f0b7e494d45e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:38.719313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae852d3-ef21-4631-b322-f0b7e494d45e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:38.722559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae852d3-ef21-4631-b322-f0b7e494d45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:38.787150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cae852d3-ef21-4631-b322-f0b7e494d45e None None] MFA Rules not processed for user `4d9c7f4729e04bf6b915246ca7b5a3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:38.790539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cae852d3-ef21-4631-b322-f0b7e494d45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:38.815090 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1476/2937] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:43.583537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06265c2-c91b-49b3-b562-37034c5e6077 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:43.584012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06265c2-c91b-49b3-b562-37034c5e6077 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:43.584012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06265c2-c91b-49b3-b562-37034c5e6077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:43.622085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c06265c2-c91b-49b3-b562-37034c5e6077 None None] MFA Rules not processed for user `9c2b746b7c194596808a7b0c736f3b53`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:43.623316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c06265c2-c91b-49b3-b562-37034c5e6077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:43.714738 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1462/2938] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:48.755837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=53 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:51.222285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:51.223330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:51.223817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:51.257307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] MFA Rules not processed for user `d461326fa46149f5a49b278b32d9c9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:51.258979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:51.348744 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1477/2939] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:51.372689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06265c2-c91b-49b3-b562-37034c5e6077 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:51.389897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06265c2-c91b-49b3-b562-37034c5e6077 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:51.395009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e8332-d7bb-413d-bc7a-8aede844205b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:51.396106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8332-d7bb-413d-bc7a-8aede844205b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:51.396410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8332-d7bb-413d-bc7a-8aede844205b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:51.396947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e8332-d7bb-413d-bc7a-8aede844205b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:51.399557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a4e8332-d7bb-413d-bc7a-8aede844205b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:51.497390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e8332-d7bb-413d-bc7a-8aede844205b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:51.499039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e8332-d7bb-413d-bc7a-8aede844205b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:51.521821 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1463/2940] 10.222.0.26 () {60 vars in 1396 bytes} [Sat Sep 17 20:56:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3274 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:59.650278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:59.651946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:59.729346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe9256-1475-4925-afc7-6188b723d000 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:59.733859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:59.734497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:59.734771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:59.735036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] PATH_INFO: `/v3/users/6fe33ab0e3394b7ea6b1b0b29f444e53` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:59.736166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe33ab0e3394b7ea6b1b0b29f444e53)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:59.738152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:59.786267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe33ab0e3394b7ea6b1b0b29f444e53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:59.797622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe33ab0e3394b7ea6b1b0b29f444e53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:59.806468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe33ab0e3394b7ea6b1b0b29f444e53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:59.807060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe33ab0e3394b7ea6b1b0b29f444e53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:59.807665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe33ab0e3394b7ea6b1b0b29f444e53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:56:59.821319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1478/2941] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:56:59 2022] DELETE /identity/v3/users/6fe33ab0e3394b7ea6b1b0b29f444e53 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:56:59.826306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:59.826833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:59.826833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:59.834099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:59.847407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:59.848596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:59.926205 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1464/2942] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:56:59.937107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:59.938681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:59.968563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:00.002891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06521b08-d138-4b04-a753-40382c9b5426 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:00.007427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:00.008048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:00.008369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:00.008891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:00.021026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:00.022551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:00.033217 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1479/2943] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:00.213011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:00.214526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:00.227728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111266ae-2fa9-433c-9410-d31054a3b931 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:00.240509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:00.241175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:00.241452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:00.241760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] PATH_INFO: `/v3/projects/eee008b613cd468f9b996e915ced3074` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:00.242997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee008b613cd468f9b996e915ced3074)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:00.245175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:00.298784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee008b613cd468f9b996e915ced3074', 'request_id': 'req-1c71467d-a97d-4664-878d-d3ad19c0c055'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.305264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee008b613cd468f9b996e915ced3074', 'request_id': 'req-1c71467d-a97d-4664-878d-d3ad19c0c055'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.313695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee008b613cd468f9b996e915ced3074', 'request_id': 'req-1c71467d-a97d-4664-878d-d3ad19c0c055'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.326667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] The token cache is being invalidate because project eee008b613cd468f9b996e915ced3074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:00.326966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.327701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.329499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1465/2944] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:00 2022] DELETE /identity/v3/projects/eee008b613cd468f9b996e915ced3074 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:00.334093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:00.345522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387a30a-e536-45ce-a967-7c6d4a4110bd service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:00.452935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:00.454293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:00.456841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:00.457826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] PATH_INFO: `/v3/users/491d2b684eb94a33a4b7a2b4d1bcaf49` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:00.463117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=491d2b684eb94a33a4b7a2b4d1bcaf49)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:00.466427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:00.538845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491d2b684eb94a33a4b7a2b4d1bcaf49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.557487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491d2b684eb94a33a4b7a2b4d1bcaf49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.570063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491d2b684eb94a33a4b7a2b4d1bcaf49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.571000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491d2b684eb94a33a4b7a2b4d1bcaf49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.571999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edad50-a705-4158-8d21-76115ae36f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491d2b684eb94a33a4b7a2b4d1bcaf49'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:00.607576 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1480/2945] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:57:00 2022] DELETE /identity/v3/users/491d2b684eb94a33a4b7a2b4d1bcaf49 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:00.894147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:00.896121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:00.932701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71467d-a97d-4664-878d-d3ad19c0c055 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:01.007457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:01.008105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.008584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.009133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] PATH_INFO: `/v3/projects/d937c43c346a4286ab80fda4c4601c89` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.010933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d937c43c346a4286ab80fda4c4601c89)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:01.013142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:01.059077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:01.084900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd937c43c346a4286ab80fda4c4601c89', 'request_id': 'req-e4fe8e19-3428-4e3c-b940-546033f8084d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:01.092422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd937c43c346a4286ab80fda4c4601c89', 'request_id': 'req-e4fe8e19-3428-4e3c-b940-546033f8084d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:01.105484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd937c43c346a4286ab80fda4c4601c89', 'request_id': 'req-e4fe8e19-3428-4e3c-b940-546033f8084d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:01.131204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] The token cache is being invalidate because project d937c43c346a4286ab80fda4c4601c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:01.131204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:01.131873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4fe8e19-3428-4e3c-b940-546033f8084d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:01.156852 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1466/2946] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:00 2022] DELETE /identity/v3/projects/d937c43c346a4286ab80fda4c4601c89 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:01.162687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f904b-6cd2-434f-9ca3-2ad3f4c739a3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.163261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f904b-6cd2-434f-9ca3-2ad3f4c739a3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.163261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f904b-6cd2-434f-9ca3-2ad3f4c739a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.194507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3e0f904b-6cd2-434f-9ca3-2ad3f4c739a3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:01.196325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3e0f904b-6cd2-434f-9ca3-2ad3f4c739a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:01.249787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:01.286109 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1481/2947] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:01.287870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e12e25-a3ac-4704-a65f-fd597ac6e58e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.288161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e12e25-a3ac-4704-a65f-fd597ac6e58e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.288433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e12e25-a3ac-4704-a65f-fd597ac6e58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.321231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-92e12e25-a3ac-4704-a65f-fd597ac6e58e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:01.323985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-92e12e25-a3ac-4704-a65f-fd597ac6e58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:01.341137 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1467/2948] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:01.342303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f904b-6cd2-434f-9ca3-2ad3f4c739a3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:01.357511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f904b-6cd2-434f-9ca3-2ad3f4c739a3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:01.368484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:01.369927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.370420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.370942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.373062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:57:01.373904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:01.378225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:01.404871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e12e25-a3ac-4704-a65f-fd597ac6e58e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:01.437103 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1482/2949] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:57:01.439733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e12e25-a3ac-4704-a65f-fd597ac6e58e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:01.457772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:01.459911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.460265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.460483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.461804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:01.465528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:01.511128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:01.523015 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1468/2950] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:57:01 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:01.525920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac61b2-e400-4f66-b82d-919f689f7637 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:01.594965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:01.595533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.595814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.596088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.597342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:01.599643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:01.626671 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1483/2951] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:57:01 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:01.630463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:01.642458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d59189-b648-47ad-b88b-79b2ec674e7d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:01.648854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:01.649499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.649791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.650061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.650714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:01.652723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:01.672311 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1469/2952] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:01.673796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:01.695396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0145f2-8b2f-46c2-8543-a3ee82e01c0b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:01.702139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19666e-6207-48e6-ab56-d67dbacbe10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:01.702887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19666e-6207-48e6-ab56-d67dbacbe10d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.703397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19666e-6207-48e6-ab56-d67dbacbe10d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.703827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19666e-6207-48e6-ab56-d67dbacbe10d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.704650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19666e-6207-48e6-ab56-d67dbacbe10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:01.706489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19666e-6207-48e6-ab56-d67dbacbe10d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:01.719307 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1484/2953] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:01.726288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:01.739582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef3444-9f16-4423-b5dc-3ab8a9aac2d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:01.748500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:01.749330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.749648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.749924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] PATH_INFO: `/v3/projects/20bd9597d34a43d08a9609dc4dba4958/users/52132aa6175c41a3b2bc16c427da72b0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.772617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bd9597d34a43d08a9609dc4dba4958, user_id=52132aa6175c41a3b2bc16c427da72b0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:01.775598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:01.809572 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1470/2954] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:57:01 2022] PUT /identity/v3/projects/20bd9597d34a43d08a9609dc4dba4958/users/52132aa6175c41a3b2bc16c427da72b0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:01.810441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365007fc-3a56-457a-8e6f-d665989d597c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:01.810916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365007fc-3a56-457a-8e6f-d665989d597c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:01.811192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365007fc-3a56-457a-8e6f-d665989d597c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:01.852657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-365007fc-3a56-457a-8e6f-d665989d597c None None] MFA Rules not processed for user `52132aa6175c41a3b2bc16c427da72b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:01.854245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-365007fc-3a56-457a-8e6f-d665989d597c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:01.929414 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1485/2955] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:02.069023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:02.070796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:02.155770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a6180-615f-4c25-8e70-8d046aaa74df admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:02.161572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:02.162276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:02.162622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:02.163351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:02.174878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:02.176507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:02.186941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1471/2956] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:02.390203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc2c45-3569-492b-989c-ab7c731df94d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:02.390716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc2c45-3569-492b-989c-ab7c731df94d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:02.390841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc2c45-3569-492b-989c-ab7c731df94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:02.420959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9fdc2c45-3569-492b-989c-ab7c731df94d None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:02.422245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9fdc2c45-3569-492b-989c-ab7c731df94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:02.461705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:02.479735 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1486/2957] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:02.491903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:02.493312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:02.535375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:02.565740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6207e-6b6f-47f2-9d26-0c8a2541485a service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:02.571194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576f9b5-d7e1-40e2-825d-49683bb9b847 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:02.571808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576f9b5-d7e1-40e2-825d-49683bb9b847 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:02.572085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576f9b5-d7e1-40e2-825d-49683bb9b847 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:02.572385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576f9b5-d7e1-40e2-825d-49683bb9b847 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:02.580819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576f9b5-d7e1-40e2-825d-49683bb9b847 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:02.582422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576f9b5-d7e1-40e2-825d-49683bb9b847 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:02.591135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1472/2958] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:57:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:57:02.719520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78778504-7f8e-41a4-845b-c51be5fda2e3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:02.719813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78778504-7f8e-41a4-845b-c51be5fda2e3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:02.720095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78778504-7f8e-41a4-845b-c51be5fda2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:02.739475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-78778504-7f8e-41a4-845b-c51be5fda2e3 None None] MFA Rules not processed for user `52132aa6175c41a3b2bc16c427da72b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:02.740803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78778504-7f8e-41a4-845b-c51be5fda2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:02.747932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1487/2959] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:03.385372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0ba1-789e-42fd-bca0-5d0893f75789 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:03.385908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0ba1-789e-42fd-bca0-5d0893f75789 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:03.386009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0ba1-789e-42fd-bca0-5d0893f75789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:03.407158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-290c0ba1-789e-42fd-bca0-5d0893f75789 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:03.408566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-290c0ba1-789e-42fd-bca0-5d0893f75789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:03.416934 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1473/2960] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:03.431547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78778504-7f8e-41a4-845b-c51be5fda2e3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:03.446696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78778504-7f8e-41a4-845b-c51be5fda2e3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:03.451736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a155535-0da6-4955-b9bb-2a04f66ddcef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:03.452336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a155535-0da6-4955-b9bb-2a04f66ddcef service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:03.452836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a155535-0da6-4955-b9bb-2a04f66ddcef service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:03.453102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a155535-0da6-4955-b9bb-2a04f66ddcef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:03.462205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a155535-0da6-4955-b9bb-2a04f66ddcef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:03.463838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a155535-0da6-4955-b9bb-2a04f66ddcef service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:03.472939 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1488/2961] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:57:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:57:03.587710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c0ba1-789e-42fd-bca0-5d0893f75789 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:03.676400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c0ba1-789e-42fd-bca0-5d0893f75789 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:03.685863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:03.686542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:03.686893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:03.687220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] PATH_INFO: `/v3/users/52132aa6175c41a3b2bc16c427da72b0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:03.688478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52132aa6175c41a3b2bc16c427da72b0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:03.690683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:03.746799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52132aa6175c41a3b2bc16c427da72b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:03.747544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52132aa6175c41a3b2bc16c427da72b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:03.756713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52132aa6175c41a3b2bc16c427da72b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:03.757344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52132aa6175c41a3b2bc16c427da72b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:03.781761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52132aa6175c41a3b2bc16c427da72b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:03.801175 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1474/2962] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:57:03 2022] DELETE /identity/v3/users/52132aa6175c41a3b2bc16c427da72b0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:03.802868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:03.803156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:03.803427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:03.829444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:03.831556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:03.965814 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1489/2963] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:03.978366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:03.981024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:04.060008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:04.083224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46e75a-5a62-4d1b-945b-39b94fb3e5e5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:04.096930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78c4c9-7fb2-4a84-ae92-ccd61c952f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:04.100675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c4c9-7fb2-4a84-ae92-ccd61c952f5f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:04.101234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c4c9-7fb2-4a84-ae92-ccd61c952f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:04.101799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c4c9-7fb2-4a84-ae92-ccd61c952f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:04.130815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78c4c9-7fb2-4a84-ae92-ccd61c952f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:04.132661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78c4c9-7fb2-4a84-ae92-ccd61c952f5f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:04.150058 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1475/2964] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:04.916457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:04.918095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:04.948490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b53f0-a120-46ed-9915-04aaab2e2818 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:04.954000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:04.954585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:04.954865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:04.955135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] PATH_INFO: `/v3/projects/20bd9597d34a43d08a9609dc4dba4958` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:04.956279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bd9597d34a43d08a9609dc4dba4958)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:04.959153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:05.025606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bd9597d34a43d08a9609dc4dba4958', 'request_id': 'req-db1604f0-24be-41e6-a2b1-aa771b19d829'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:05.035458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bd9597d34a43d08a9609dc4dba4958', 'request_id': 'req-db1604f0-24be-41e6-a2b1-aa771b19d829'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:05.043156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bd9597d34a43d08a9609dc4dba4958', 'request_id': 'req-db1604f0-24be-41e6-a2b1-aa771b19d829'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:05.069004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] The token cache is being invalidate because project 20bd9597d34a43d08a9609dc4dba4958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:05.069390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:05.070203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db1604f0-24be-41e6-a2b1-aa771b19d829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:05.075532 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1490/2965] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:04 2022] DELETE /identity/v3/projects/20bd9597d34a43d08a9609dc4dba4958 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:05.077646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5643c9-ea0a-48e9-a703-910faf40e286 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.078139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5643c9-ea0a-48e9-a703-910faf40e286 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.078247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5643c9-ea0a-48e9-a703-910faf40e286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.105244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5d5643c9-ea0a-48e9-a703-910faf40e286 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:05.106505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5d5643c9-ea0a-48e9-a703-910faf40e286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:05.185810 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1476/2966] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:05.193726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309645-e815-48fd-8b0a-b08537181674 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.194232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309645-e815-48fd-8b0a-b08537181674 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.194655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a309645-e815-48fd-8b0a-b08537181674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.220287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0a309645-e815-48fd-8b0a-b08537181674 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:05.222180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0a309645-e815-48fd-8b0a-b08537181674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:05.239471 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1491/2967] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:05.239696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5643c9-ea0a-48e9-a703-910faf40e286 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:05.253851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5643c9-ea0a-48e9-a703-910faf40e286 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:05.259934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:05.260817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.261322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.261860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.262755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:57:05.263293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:05.266055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:05.280795 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1477/2968] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:57:05.285487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a309645-e815-48fd-8b0a-b08537181674 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:05.296123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a309645-e815-48fd-8b0a-b08537181674 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:05.303720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:05.304261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.304529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.304804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.305545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:05.307433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:05.354296 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1492/2969] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:57:05 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:05.360342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:05.396844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af158d8d-3dbe-4edb-991c-1abda20e0d3a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:05.499044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:05.499843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.500191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.500521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.501284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:05.503597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:05.531145 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1478/2970] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:57:05 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:05.535753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:05.546956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cd0e7-32db-4067-9f02-b426359b4dbb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:05.552060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:05.552676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.552948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.553219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.553975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:05.555553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:05.567318 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1493/2971] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:57:05.571606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:05.585319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b9b2-c992-4c6b-b215-689b645cdcf1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:05.592099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d31f8-b2ee-4843-860c-a37ed1b720ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:05.592720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d31f8-b2ee-4843-860c-a37ed1b720ed admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.592996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d31f8-b2ee-4843-860c-a37ed1b720ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.593268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d31f8-b2ee-4843-860c-a37ed1b720ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.594056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d31f8-b2ee-4843-860c-a37ed1b720ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:05.595827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d31f8-b2ee-4843-860c-a37ed1b720ed admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:05.606570 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1479/2972] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:57:05.611266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:05.629082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f0c0c-d9e7-41ab-96b2-3dd144e7f004 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:05.634072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:05.634754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.635005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.635275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] PATH_INFO: `/v3/projects/89b46ae7347d4f2daf02f627e41a4847/users/3009c5feb1ef43a08b07996d8f63bf11/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.646094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:05.656893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b46ae7347d4f2daf02f627e41a4847, user_id=3009c5feb1ef43a08b07996d8f63bf11, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:05.660853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:05.681583 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1494/2973] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:57:05 2022] PUT /identity/v3/projects/89b46ae7347d4f2daf02f627e41a4847/users/3009c5feb1ef43a08b07996d8f63bf11/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:57:05.687215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99ae9-a16b-4342-9c29-5c24322b0080 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.687215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99ae9-a16b-4342-9c29-5c24322b0080 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.687440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c99ae9-a16b-4342-9c29-5c24322b0080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.725127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-60c99ae9-a16b-4342-9c29-5c24322b0080 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:05.728330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-60c99ae9-a16b-4342-9c29-5c24322b0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:05.802288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:05.829613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1480/2974] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:05.840951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:05.843632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:05.973093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80003-98d9-4655-b949-eae72d1a0541 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:05.979744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4982a-83fc-4b99-87a0-29597a7a1704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:05.980380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4982a-83fc-4b99-87a0-29597a7a1704 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:05.980708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4982a-83fc-4b99-87a0-29597a7a1704 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:05.981025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4982a-83fc-4b99-87a0-29597a7a1704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:05.998799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4982a-83fc-4b99-87a0-29597a7a1704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:06.000316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4982a-83fc-4b99-87a0-29597a7a1704 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:06.012113 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1495/2975] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:15.416620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:15.417104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:15.417213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:15.460649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:15.462088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:15.551463 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1481/2976] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:15.562061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:15.563060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:15.563381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:15.587980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:15.590087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:15.599954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1496/2977] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:15.686281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:15.688001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:15.856419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a936c-5ae2-4fea-86fd-a9a2cbd48749 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:15.874450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:15.875102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:15.875378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:15.875647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:15.886346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:15.889626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:15.910002 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1482/2978] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:57:16.620910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:16.622377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:16.654054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b459b-0bfb-426e-b511-2771d1d26ffc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:16.660670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:16.662201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:16.662970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:16.663466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:16.666527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:16.693568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:16.696184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:16.712278 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1497/2979] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:18.126618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:18.130355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:18.377400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfe795-b80f-43ce-99f6-2ec4245520b2 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:18.386618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d129a-549f-4f63-9782-0c39f8f4a979 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:18.388004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d129a-549f-4f63-9782-0c39f8f4a979 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:18.388503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d129a-549f-4f63-9782-0c39f8f4a979 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:18.388982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d129a-549f-4f63-9782-0c39f8f4a979 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:18.394424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7d2d129a-549f-4f63-9782-0c39f8f4a979 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:18.410305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:18.419307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d129a-549f-4f63-9782-0c39f8f4a979 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:18.421191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d129a-549f-4f63-9782-0c39f8f4a979 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:18.455124 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1483/2980] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:20.868448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:20.870265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:20.930375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabb478-79af-47dd-bbbc-e2233f4fa9d3 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:20.944241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:20.944898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:20.945339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:20.945447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] PATH_INFO: `/v3/users/4d9c7f4729e04bf6b915246ca7b5a3cf` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:20.947689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9c7f4729e04bf6b915246ca7b5a3cf)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:20.949912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:21.041308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9c7f4729e04bf6b915246ca7b5a3cf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:21.059102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9c7f4729e04bf6b915246ca7b5a3cf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:21.074243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9c7f4729e04bf6b915246ca7b5a3cf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:21.075554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9c7f4729e04bf6b915246ca7b5a3cf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:21.076162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9c7f4729e04bf6b915246ca7b5a3cf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:21.139540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1498/2981] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:57:20 2022] DELETE /identity/v3/users/4d9c7f4729e04bf6b915246ca7b5a3cf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:21.146885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:21.148827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:21.148959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:21.187657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:21.189643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:21.312782 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1484/2982] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:21.349644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:21.351392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:21.496042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:21.515452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533df59-d8c2-4d12-bb72-5deda92da934 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:21.520578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:21.521010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:21.521354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:21.521720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:21.533552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:21.535209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:21.547559 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1499/2983] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:22.071584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:22.073257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:22.096509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0b67d-bcc0-4251-a01d-a5256b264321 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:22.109364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:22.112187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:22.114835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:22.115445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] PATH_INFO: `/v3/projects/84caeb4c4b2e4dbda9bf7def13420af1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:22.124326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84caeb4c4b2e4dbda9bf7def13420af1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:22.132377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:22.300878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84caeb4c4b2e4dbda9bf7def13420af1', 'request_id': 'req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.315403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84caeb4c4b2e4dbda9bf7def13420af1', 'request_id': 'req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.331292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84caeb4c4b2e4dbda9bf7def13420af1', 'request_id': 'req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.358674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] The token cache is being invalidate because project 84caeb4c4b2e4dbda9bf7def13420af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:22.359918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.361085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.363933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1485/2984] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:22 2022] DELETE /identity/v3/projects/84caeb4c4b2e4dbda9bf7def13420af1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:22.368988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:22.387100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd4863-89bc-4779-ad7b-ea2d86e7285c service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:22.552979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:22.553957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:22.556796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:22.557427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] PATH_INFO: `/v3/users/1f386dc88f214f489cb4105c2f6c50b0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:22.562145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f386dc88f214f489cb4105c2f6c50b0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:22.570444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:22.712049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f386dc88f214f489cb4105c2f6c50b0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.734293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f386dc88f214f489cb4105c2f6c50b0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.759312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f386dc88f214f489cb4105c2f6c50b0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.760646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f386dc88f214f489cb4105c2f6c50b0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.762073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28d88f7a-a7af-4a1e-9148-fca57b573042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f386dc88f214f489cb4105c2f6c50b0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:22.790352 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1500/2985] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:57:22 2022] DELETE /identity/v3/users/1f386dc88f214f489cb4105c2f6c50b0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:24.315057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:24.317411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:24.336349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1e8e9-d2f0-414d-ba4e-b5edb60b05d4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:24.461405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:24.462334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:24.462749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:24.463153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] PATH_INFO: `/v3/projects/eddfa4151eed49beafb4a87d152dc8c4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:24.467730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eddfa4151eed49beafb4a87d152dc8c4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:24.470122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:24.566938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:24.644070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eddfa4151eed49beafb4a87d152dc8c4', 'request_id': 'req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:24.652660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eddfa4151eed49beafb4a87d152dc8c4', 'request_id': 'req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:24.666117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eddfa4151eed49beafb4a87d152dc8c4', 'request_id': 'req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:24.692848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] The token cache is being invalidate because project eddfa4151eed49beafb4a87d152dc8c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:24.692848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:24.693762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-95899b2c-00d9-4d8d-a3bb-619f29cbda2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:24.697522 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1486/2986] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:24 2022] DELETE /identity/v3/projects/eddfa4151eed49beafb4a87d152dc8c4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:57:24.703822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d2cab-b904-4be9-a5e9-f375770238ee None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:24.704721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d2cab-b904-4be9-a5e9-f375770238ee None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:24.705358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d2cab-b904-4be9-a5e9-f375770238ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:24.730671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-175d2cab-b904-4be9-a5e9-f375770238ee None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:24.732167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-175d2cab-b904-4be9-a5e9-f375770238ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:24.823580 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1501/2987] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:24.832811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa03ae8-59b7-44dc-aba4-0f9dcd131f6b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:24.833316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa03ae8-59b7-44dc-aba4-0f9dcd131f6b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:24.833914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa03ae8-59b7-44dc-aba4-0f9dcd131f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:24.883512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-efa03ae8-59b7-44dc-aba4-0f9dcd131f6b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:24.885690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-efa03ae8-59b7-44dc-aba4-0f9dcd131f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:24.902416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d2cab-b904-4be9-a5e9-f375770238ee None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:24.910897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:24.913400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1487/2988] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:24.917421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d2cab-b904-4be9-a5e9-f375770238ee None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:24.926863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:24.927537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:24.927994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:24.928270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:24.928937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:57:24.929232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:24.931349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:24.950698 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1502/2989] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:57:24.956268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa03ae8-59b7-44dc-aba4-0f9dcd131f6b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:24.966615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa03ae8-59b7-44dc-aba4-0f9dcd131f6b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:24.974632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:24.975562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:24.976253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:24.976540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:24.977192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:24.979782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:25.024652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:25.035275 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1488/2990] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:57:24 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:25.037653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d7678-ead2-4f70-bb0c-c0a5de9f1446 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:25.205070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:25.205736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:25.206280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:25.206280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:25.207555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:25.211523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:25.277902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:25.289844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1503/2991] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:57:25 2022] POST /identity/v3/users => generated 291 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:25.292554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f5aa7-ad7b-4d7e-8b99-889c280a0f11 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:25.311697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:25.314492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:25.316075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:25.325124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:25.326585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:25.332611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:25.376243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:25.377088 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1489/2992] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:25.407853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aaaf18-2fba-411a-ad13-6bbcebcfa509 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:25.437781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0c813-c64d-4340-b7c9-d70376d0cbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:25.438931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c813-c64d-4340-b7c9-d70376d0cbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:25.439208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c813-c64d-4340-b7c9-d70376d0cbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:25.439479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c813-c64d-4340-b7c9-d70376d0cbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:25.440138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0c813-c64d-4340-b7c9-d70376d0cbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:25.441809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0c813-c64d-4340-b7c9-d70376d0cbc9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:25.480502 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1504/2993] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:25.485313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:25.504887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b4d5-96b8-4e45-b07b-e158f30efbb7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:25.523073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:25.524099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:25.524543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:25.525021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] PATH_INFO: `/v3/projects/3515123097e744ef855ef1129fb59cf1/users/0d48260054d84784b89a52d9b766d7c1/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:25.587373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3515123097e744ef855ef1129fb59cf1, user_id=0d48260054d84784b89a52d9b766d7c1, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:25.590238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:25.641781 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1490/2994] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:57:25 2022] PUT /identity/v3/projects/3515123097e744ef855ef1129fb59cf1/users/0d48260054d84784b89a52d9b766d7c1/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:25.653576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f078-0459-4d01-b097-cf50e3038989 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:25.654265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f078-0459-4d01-b097-cf50e3038989 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:25.654265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f078-0459-4d01-b097-cf50e3038989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:25.709090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6cc6f078-0459-4d01-b097-cf50e3038989 None None] MFA Rules not processed for user `0d48260054d84784b89a52d9b766d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:25.710298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6cc6f078-0459-4d01-b097-cf50e3038989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:25.861020 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1505/2995] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:25.865065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:26.066858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5f936-d5f0-4d17-ae4e-8cb8fd70f58e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:26.076566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:26.077519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.077956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.078446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.079370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:26.084462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:26.170999 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1491/2996] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:57:25 2022] POST /identity/v3/projects => generated 383 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:26.172231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f078-0459-4d01-b097-cf50e3038989 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:26.191981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f078-0459-4d01-b097-cf50e3038989 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:26.327425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:26.329921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.330446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.330865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.331702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:26.333939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:26.346249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:26.373335 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1506/2997] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:57:26 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:26.377895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:26.396836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5e4cf-5bcc-4bf8-962d-e058e8a20fdc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:26.410050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:26.410050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.410050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.410050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.410542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:26.412370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:26.472906 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1492/2998] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:26.474962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:26.490793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919501aa-8d10-4272-aa38-2c8db47cc0f1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:26.503496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:26.504817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.505347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.506156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] PATH_INFO: `/v3/projects/e0fca99ea853470d9fa1df0bd87fcfcc/users/d3738b013fa24822a134fee91bffa9c1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.557969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fca99ea853470d9fa1df0bd87fcfcc, user_id=d3738b013fa24822a134fee91bffa9c1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:26.565148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:26.622048 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1507/2999] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:57:26 2022] PUT /identity/v3/projects/e0fca99ea853470d9fa1df0bd87fcfcc/users/d3738b013fa24822a134fee91bffa9c1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:26.626054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:26.640402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f79f-bac3-43cb-95da-0e8481f49c7d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:26.700257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:26.777072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bc157-1662-4ee7-a3c2-bbe230e7d6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:26.778265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bc157-1662-4ee7-a3c2-bbe230e7d6dd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.778842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bc157-1662-4ee7-a3c2-bbe230e7d6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.779350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bc157-1662-4ee7-a3c2-bbe230e7d6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.780278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bc157-1662-4ee7-a3c2-bbe230e7d6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:26.782567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bc157-1662-4ee7-a3c2-bbe230e7d6dd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:26.809223 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1493/3000] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:26.814132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:26.828964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf173cf3-8fff-498b-830f-a92faf9adf0d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:26.839866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc5425-d232-456c-9cf4-774d0fe45827 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.840224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc5425-d232-456c-9cf4-774d0fe45827 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.840574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc5425-d232-456c-9cf4-774d0fe45827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.862635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b936b8-8742-48e3-902b-7d01c4f6cdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:26.863339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b936b8-8742-48e3-902b-7d01c4f6cdf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.863673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b936b8-8742-48e3-902b-7d01c4f6cdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.864140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b936b8-8742-48e3-902b-7d01c4f6cdf0 admin admin] PATH_INFO: `/v3/projects/e0fca99ea853470d9fa1df0bd87fcfcc/users/d3738b013fa24822a134fee91bffa9c1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.873946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b936b8-8742-48e3-902b-7d01c4f6cdf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fca99ea853470d9fa1df0bd87fcfcc, user_id=d3738b013fa24822a134fee91bffa9c1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:26.878824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b936b8-8742-48e3-902b-7d01c4f6cdf0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:26.900268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-43dc5425-d232-456c-9cf4-774d0fe45827 None None] MFA Rules not processed for user `bc22daf24ce44eb28bb6e000316e4df4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:26.902764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43dc5425-d232-456c-9cf4-774d0fe45827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:26.920998 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1508/3001] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:57:26 2022] PUT /identity/v3/projects/e0fca99ea853470d9fa1df0bd87fcfcc/users/d3738b013fa24822a134fee91bffa9c1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:26.935010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.936876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.937166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:27.071557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] MFA Rules not processed for user `d3738b013fa24822a134fee91bffa9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:27.072851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:27.177317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1494/3002] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:57:27.316226 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1509/3003] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:27.330848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:27.331439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:27.331996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:27.361620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] MFA Rules not processed for user `0d48260054d84784b89a52d9b766d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:27.363583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:27.446730 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1495/3004] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:27.642754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:27.643860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:27.654626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:27.655590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:27.853249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e5dd2-bde2-4299-9088-5a8a9fc79640 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:27.863591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:27.864927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:27.865895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:27.866169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:27.870376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:27.879808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:27.894850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d706d7b-8343-4ba5-ba2a-63e937e1ee56 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:27.907969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769874ab-80a9-4cb8-83bd-8e6f5138ccbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:27.910298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769874ab-80a9-4cb8-83bd-8e6f5138ccbf service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:27.910579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769874ab-80a9-4cb8-83bd-8e6f5138ccbf service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:27.910843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769874ab-80a9-4cb8-83bd-8e6f5138ccbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:27.932346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769874ab-80a9-4cb8-83bd-8e6f5138ccbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:27.934037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769874ab-80a9-4cb8-83bd-8e6f5138ccbf service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:27.976885 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1496/3005] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:28.006840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:28.044421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:28.045936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:28.077721 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1510/3006] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:30.745567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:30.746437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:30.746437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:30.786412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:30.789579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:30.942095 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1497/3007] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:30.969302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:30.970929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:31.110033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:31.115855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:31.143006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d39505-e97e-40ed-aa75-988832e402de None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:31.143403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a298c2-0b9b-4735-a143-4c278809f64d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:31.158225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:31.158962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:31.159268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:31.159542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:31.168049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:31.168805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:31.169079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:31.169349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] PATH_INFO: `/v3/users/d461326fa46149f5a49b278b32d9c9fb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:31.173025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d461326fa46149f5a49b278b32d9c9fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:31.178024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:31.180502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:31.181746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:31.198794 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1511/3008] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:57:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:57:31.297441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd461326fa46149f5a49b278b32d9c9fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:31.298272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd461326fa46149f5a49b278b32d9c9fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:31.313889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd461326fa46149f5a49b278b32d9c9fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:31.316237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd461326fa46149f5a49b278b32d9c9fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:31.344873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1836f0c9-1691-4ec3-a7be-424064b96157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd461326fa46149f5a49b278b32d9c9fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:31.376310 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1498/3009] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:57:31 2022] DELETE /identity/v3/users/d461326fa46149f5a49b278b32d9c9fb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:57:31.767532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:31.792366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2287f08-8099-47ca-85fc-036831b025a1 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:31.971799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:31.973440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:31.974003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:31.975584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] PATH_INFO: `/v3/projects/52c89ce6659b4b4b9d6a53c9a8d33b41` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:31.981578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c89ce6659b4b4b9d6a53c9a8d33b41)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:31.987120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:32.129045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c89ce6659b4b4b9d6a53c9a8d33b41', 'request_id': 'req-db86c2e0-22ff-45b4-9cb7-07174a8d769a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:32.150593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c89ce6659b4b4b9d6a53c9a8d33b41', 'request_id': 'req-db86c2e0-22ff-45b4-9cb7-07174a8d769a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:32.165919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c89ce6659b4b4b9d6a53c9a8d33b41', 'request_id': 'req-db86c2e0-22ff-45b4-9cb7-07174a8d769a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:32.212413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] The token cache is being invalidate because project 52c89ce6659b4b4b9d6a53c9a8d33b41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:32.212715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:32.227278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-db86c2e0-22ff-45b4-9cb7-07174a8d769a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:32.235503 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1512/3010] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:31 2022] DELETE /identity/v3/projects/52c89ce6659b4b4b9d6a53c9a8d33b41 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:32.238296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9c536-759f-4cb0-8eab-68eae4e21500 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:32.239162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9c536-759f-4cb0-8eab-68eae4e21500 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:32.239607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9c536-759f-4cb0-8eab-68eae4e21500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:32.307059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-52f9c536-759f-4cb0-8eab-68eae4e21500 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:32.317731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-52f9c536-759f-4cb0-8eab-68eae4e21500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:32.495497 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1499/3011] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:32.505605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcdfa1-572d-4d51-bd83-51ff591c75e1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:32.505865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcdfa1-572d-4d51-bd83-51ff591c75e1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:32.506142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcdfa1-572d-4d51-bd83-51ff591c75e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:32.548712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:32.558194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-97bcdfa1-572d-4d51-bd83-51ff591c75e1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:32.560017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-97bcdfa1-572d-4d51-bd83-51ff591c75e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:32.588685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1513/3012] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:32.589496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9c536-759f-4cb0-8eab-68eae4e21500 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:32.604026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9c536-759f-4cb0-8eab-68eae4e21500 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:32.625226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:32.626536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:32.629170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:32.631936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:32.636484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:57:32.637730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:32.641368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:32.658843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:32.693284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcdfa1-572d-4d51-bd83-51ff591c75e1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:32.703910 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1500/3013] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:57:32.710121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcdfa1-572d-4d51-bd83-51ff591c75e1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:32.727681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:32.728509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:32.728789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:32.729054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:32.729865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:32.732091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:32.800342 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1514/3014] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:57:32 2022] POST /identity/v3/projects => generated 399 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:32.801252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:32.824276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98b92-0d3e-47c7-9784-45da20a1161e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:32.990989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:32.992925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:32.992925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:32.992925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:32.992925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:32.996124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:33.051232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:33.053070 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1501/3015] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:57:32 2022] POST /identity/v3/users => generated 300 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:57:33.093177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9a937-2784-4f80-82e8-e4e25b368d4e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:33.110780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:33.115924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:33.117812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:33.118838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:33.122831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:33.128444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:33.194364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:33.202916 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1515/3016] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:33.211945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1a037-b94a-4a44-bc04-d2c06e51cb32 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:33.225336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baf23b-d555-48fc-bc87-0c1f4f4321eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:33.227482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baf23b-d555-48fc-bc87-0c1f4f4321eb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:33.228729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baf23b-d555-48fc-bc87-0c1f4f4321eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:33.230260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baf23b-d555-48fc-bc87-0c1f4f4321eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:33.231495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baf23b-d555-48fc-bc87-0c1f4f4321eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:33.236620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baf23b-d555-48fc-bc87-0c1f4f4321eb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:33.278445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:33.282175 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1502/3017] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:33.310426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5d881-71cc-431c-9d59-361df665a1d6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:33.322058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:33.323446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:33.324459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:33.326134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] PATH_INFO: `/v3/projects/044cdbed43ed4de6b4cb7e1cd4802b65/users/8db879fde0bd4bea81374a7f7bb627e6/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:33.363147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044cdbed43ed4de6b4cb7e1cd4802b65, user_id=8db879fde0bd4bea81374a7f7bb627e6, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:33.367862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:33.409238 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1516/3018] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:57:33 2022] PUT /identity/v3/projects/044cdbed43ed4de6b4cb7e1cd4802b65/users/8db879fde0bd4bea81374a7f7bb627e6/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:33.410128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b870-d52f-410c-b690-46f3f4ab3f51 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:33.410694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b870-d52f-410c-b690-46f3f4ab3f51 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:33.411475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b870-d52f-410c-b690-46f3f4ab3f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:33.458708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-66b5b870-d52f-410c-b690-46f3f4ab3f51 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:33.470164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-66b5b870-d52f-410c-b690-46f3f4ab3f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:33.584270 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1503/3019] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:33.640954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:33.647442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:33.769022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8cbbb-202b-4845-9df1-50429c2e6712 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:33.773247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:33.774006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:33.774378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:33.774739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:33.786047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:33.788522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:33.800563 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1517/3020] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:43.570566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:43.795079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:50.264896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf3443-3375-4285-a78e-b3076301b7f2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:50.265966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf3443-3375-4285-a78e-b3076301b7f2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:50.265966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf3443-3375-4285-a78e-b3076301b7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:50.405434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-17bf3443-3375-4285-a78e-b3076301b7f2 None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:50.418554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17bf3443-3375-4285-a78e-b3076301b7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:50.601510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1504/3021] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:52.899850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:52.902261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:53.116938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f44e5-97ac-44cc-ac68-95a1767ef330 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:53.126153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:53.128770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:53.128770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:53.130753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:53.137082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:53.365989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:53.367408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:53.381196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1518/3022] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:55.451602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b8b59-0f18-4c3b-948d-f55fc1ba9dc2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:55.452266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b8b59-0f18-4c3b-948d-f55fc1ba9dc2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:55.452507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b8b59-0f18-4c3b-948d-f55fc1ba9dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:55.514894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-357b8b59-0f18-4c3b-948d-f55fc1ba9dc2 None None] MFA Rules not processed for user `0d48260054d84784b89a52d9b766d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:55.516387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-357b8b59-0f18-4c3b-948d-f55fc1ba9dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:55.586325 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1505/3023] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:55.819574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:55.823395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:55.865003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68b7b2-45ee-48c3-a629-8a2f46aadad0 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:55.875434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:55.876785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:55.877307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:55.877839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] PATH_INFO: `/v3/users/f1134231b4a545fdb136a5204cb4f289` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:55.879243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1134231b4a545fdb136a5204cb4f289)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:55.881906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:56.012225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1134231b4a545fdb136a5204cb4f289'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:56.048933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1134231b4a545fdb136a5204cb4f289'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:56.071289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1134231b4a545fdb136a5204cb4f289'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:56.071978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1134231b4a545fdb136a5204cb4f289'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:56.073730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4335d720-4226-47ef-9c3d-dc8be0b61130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1134231b4a545fdb136a5204cb4f289'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:56.102124 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1519/3024] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:57:55 2022] DELETE /identity/v3/users/f1134231b4a545fdb136a5204cb4f289 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:56.121160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffd321-0e12-4a2f-b6a1-ddbaf9d577a1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:56.121747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffd321-0e12-4a2f-b6a1-ddbaf9d577a1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:56.122824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffd321-0e12-4a2f-b6a1-ddbaf9d577a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:56.178787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-76ffd321-0e12-4a2f-b6a1-ddbaf9d577a1 None None] MFA Rules not processed for user `8db879fde0bd4bea81374a7f7bb627e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:56.180053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-76ffd321-0e12-4a2f-b6a1-ddbaf9d577a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:56.289015 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1506/3025] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:56.300687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:56.301003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:56.301400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:56.313123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:56.328307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] MFA Rules not processed for user `8db879fde0bd4bea81374a7f7bb627e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:56.332107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:56.347676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1520/3026] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:56.424649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:56.425670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:56.426169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:56.468486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:56.473189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:56.539136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:56.574627 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1507/3027] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:56.602028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:56.611095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:56.792296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4291c-1226-4dd0-9dd3-78193fb607a4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:56.799721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:56.800592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:56.800952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:56.801405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:56.817354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:56.819280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:56.856274 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1521/3028] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:57:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 20:57:56.887871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:56.889598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:57.051048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09d98-a935-49fe-9a73-7f02dc8f9878 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:57.064570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:57.065555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:57.067570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:57.067924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] PATH_INFO: `/v3/projects/7dc8bc81c2d74bae9612f7c9807350a7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:57.071034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc8bc81c2d74bae9612f7c9807350a7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:57.076686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:57.208745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc8bc81c2d74bae9612f7c9807350a7', 'request_id': 'req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.218020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc8bc81c2d74bae9612f7c9807350a7', 'request_id': 'req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.237794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc8bc81c2d74bae9612f7c9807350a7', 'request_id': 'req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.261138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] The token cache is being invalidate because project 7dc8bc81c2d74bae9612f7c9807350a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:57.262314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.265225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.271697 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1508/3029] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:56 2022] DELETE /identity/v3/projects/7dc8bc81c2d74bae9612f7c9807350a7 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:57.276242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:57.306186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fd2b5-625a-47c1-bc92-96eb53219b1b service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:57.364904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:57.368798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:57.561088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:57.564044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:57.564622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:57.568895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] PATH_INFO: `/v3/users/2bdc03ea28d940518fb8d67567c9f2bc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:57.584233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bdc03ea28d940518fb8d67567c9f2bc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:57.595310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:57.621898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9d82c-bdc2-4f71-aa4b-33101e97d6ec admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:57.631114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0add9-5aec-42c4-83ab-89616b3df4e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:57.633250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0add9-5aec-42c4-83ab-89616b3df4e8 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:57.634498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0add9-5aec-42c4-83ab-89616b3df4e8 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:57.634952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0add9-5aec-42c4-83ab-89616b3df4e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:57.639792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ebc0add9-5aec-42c4-83ab-89616b3df4e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:57.713040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:57.764566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bdc03ea28d940518fb8d67567c9f2bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.801825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bdc03ea28d940518fb8d67567c9f2bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.817190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bdc03ea28d940518fb8d67567c9f2bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.818238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bdc03ea28d940518fb8d67567c9f2bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.819504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bdc03ea28d940518fb8d67567c9f2bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:57.851249 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1522/3030] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:57:57 2022] DELETE /identity/v3/users/2bdc03ea28d940518fb8d67567c9f2bc => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:57.958321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0add9-5aec-42c4-83ab-89616b3df4e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:57.969250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0add9-5aec-42c4-83ab-89616b3df4e8 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:58.017270 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1509/3031] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:57 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:57:58.435378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:58.438331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:58.458545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:58.620867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36693a6-67df-4e36-96d4-14ee42066a11 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:58.636216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:58.637089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:58.640104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:58.640614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] PATH_INFO: `/v3/projects/9264cb99cc4246b2a5fac162a65cc9c8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:58.643562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9264cb99cc4246b2a5fac162a65cc9c8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:58.646984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:58.808035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9264cb99cc4246b2a5fac162a65cc9c8', 'request_id': 'req-d3de57e7-0eae-49b9-af60-9b98a6c18f93'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:58.833638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9264cb99cc4246b2a5fac162a65cc9c8', 'request_id': 'req-d3de57e7-0eae-49b9-af60-9b98a6c18f93'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:58.851767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9264cb99cc4246b2a5fac162a65cc9c8', 'request_id': 'req-d3de57e7-0eae-49b9-af60-9b98a6c18f93'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:58.883520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] The token cache is being invalidate because project 9264cb99cc4246b2a5fac162a65cc9c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:57:58.884064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:58.885595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d3de57e7-0eae-49b9-af60-9b98a6c18f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:57:58.888684 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1523/3032] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:57:58 2022] DELETE /identity/v3/projects/9264cb99cc4246b2a5fac162a65cc9c8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:57:58.897305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8ecb-bd59-43d6-8124-c1a4461fa1fc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:58.899163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8ecb-bd59-43d6-8124-c1a4461fa1fc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:58.899672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8ecb-bd59-43d6-8124-c1a4461fa1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:59.012220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c52d8ecb-bd59-43d6-8124-c1a4461fa1fc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:59.015572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c52d8ecb-bd59-43d6-8124-c1a4461fa1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:59.260961 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1510/3033] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:59.273913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce3e6f-4527-4ff8-8700-fdb9e60c2ca5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:59.275188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce3e6f-4527-4ff8-8700-fdb9e60c2ca5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:59.277921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce3e6f-4527-4ff8-8700-fdb9e60c2ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:59.346008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-72ce3e6f-4527-4ff8-8700-fdb9e60c2ca5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:59.349384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-72ce3e6f-4527-4ff8-8700-fdb9e60c2ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:59.393159 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1524/3034] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:59.397592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d8ecb-bd59-43d6-8124-c1a4461fa1fc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:59.413184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d8ecb-bd59-43d6-8124-c1a4461fa1fc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:59.443606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:59.446792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:59.447981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:59.449363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:59.454943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:57:59.456790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:59.466271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:59.486668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16acfcb-9e0a-4405-ae7d-219d64025d91 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:59.487647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16acfcb-9e0a-4405-ae7d-219d64025d91 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:59.488206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16acfcb-9e0a-4405-ae7d-219d64025d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:59.510593 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1511/3035] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:57:59.511596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:59.536406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f6267-b493-4147-968e-6768efe1d26c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:59.539970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d16acfcb-9e0a-4405-ae7d-219d64025d91 None None] MFA Rules not processed for user `6e33e0b756e14af28a90cc5c76d346e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:59.542168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d16acfcb-9e0a-4405-ae7d-219d64025d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:59.551991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:59.553263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:59.553572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:59.553848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:59.554766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:59.569505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:59.633447 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1512/3036] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:57:59 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:59.639235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:59.686837 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1525/3037] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:59.799159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172a38b-188f-4400-a088-6367b206cdf8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:59.814443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:59.815180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:59.818336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:59.819551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:59.819551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:59.824280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:59.871867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1513/3038] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:57:59 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:57:59.876737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16acfcb-9e0a-4405-ae7d-219d64025d91 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:59.895456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16acfcb-9e0a-4405-ae7d-219d64025d91 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:59.909682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:59.910659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:59.911039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:59.911546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:59.913694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:59.923883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:59.951874 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1526/3039] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:57:59.954297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:59.990934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b56605-25f3-4ae1-a41e-cac2640597fb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:00.006711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c06b0-9021-42e2-a160-9a854ea94356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:00.007668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c06b0-9021-42e2-a160-9a854ea94356 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:00.008116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c06b0-9021-42e2-a160-9a854ea94356 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:00.008639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c06b0-9021-42e2-a160-9a854ea94356 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:00.009593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c06b0-9021-42e2-a160-9a854ea94356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:00.012336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c06b0-9021-42e2-a160-9a854ea94356 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:00.015923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:00.042746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1514/3040] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:58:00.046962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:00.074492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5add77-f602-40bb-acb8-35533ffe2eb1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:00.088258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:00.089348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:00.090244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:00.090588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] PATH_INFO: `/v3/projects/377bd8a00ad34071a222402d67a435d2/users/4c104fc78767424d858b40c2f438d99f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:00.119860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=377bd8a00ad34071a222402d67a435d2, user_id=4c104fc78767424d858b40c2f438d99f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:00.123574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:00.146553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:00.153910 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1527/3041] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:00 2022] PUT /identity/v3/projects/377bd8a00ad34071a222402d67a435d2/users/4c104fc78767424d858b40c2f438d99f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:00.166524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09316d-859e-4119-b6e9-441cd689f510 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:00.167362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09316d-859e-4119-b6e9-441cd689f510 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:00.167945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09316d-859e-4119-b6e9-441cd689f510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:00.246758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cd09316d-859e-4119-b6e9-441cd689f510 None None] MFA Rules not processed for user `4c104fc78767424d858b40c2f438d99f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:00.248135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cd09316d-859e-4119-b6e9-441cd689f510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:00.471225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1515/3042] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:00.474698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:00.669288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723fee3-ec13-468b-9016-7b2aa23abc35 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:00.684738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:00.686032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:00.686564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:00.687084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:00.689138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:00.691437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:00.755168 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1528/3043] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:00 2022] POST /identity/v3/projects => generated 375 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:00.759287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09316d-859e-4119-b6e9-441cd689f510 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:00.778156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09316d-859e-4119-b6e9-441cd689f510 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:00.900263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:00.902846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:00.903535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:00.903740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:00.904498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:00.930269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.010482 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1516/3044] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:00 2022] POST /identity/v3/users => generated 294 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:58:01.013798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.032282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efcb61-08cd-4456-b702-be0f6d7281f7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:01.051581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:01.052320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.052741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.053128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.054028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:01.055891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.084132 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1529/3045] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:58:01.091767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.118819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9f773-c1bd-4343-8c1e-c68ea3001137 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:01.141193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:01.142592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.143108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.143557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] PATH_INFO: `/v3/projects/fb983558bfa146c8b93b913a383fd348/users/bb54a704392b414ca3f5457b92c2eb20/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.184947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb983558bfa146c8b93b913a383fd348, user_id=bb54a704392b414ca3f5457b92c2eb20, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:01.189063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.239955 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1517/3046] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:01 2022] PUT /identity/v3/projects/fb983558bfa146c8b93b913a383fd348/users/bb54a704392b414ca3f5457b92c2eb20/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:01.240509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.255490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a443c8-68a5-44d4-8dca-a3fd484b9118 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:01.357642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:01.358634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.359100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.359522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.361676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:01.363751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.379372 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1530/3047] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:01.383830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.401255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.407339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:01.410295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a2185-341b-441d-932a-34291a46968c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:01.425644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77777c6-3c86-4766-92a5-23c66ebe2c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:01.428270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77777c6-3c86-4766-92a5-23c66ebe2c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.428565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77777c6-3c86-4766-92a5-23c66ebe2c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.428859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77777c6-3c86-4766-92a5-23c66ebe2c0a admin admin] PATH_INFO: `/v3/projects/fb983558bfa146c8b93b913a383fd348/users/bb54a704392b414ca3f5457b92c2eb20/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.447215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77777c6-3c86-4766-92a5-23c66ebe2c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb983558bfa146c8b93b913a383fd348, user_id=bb54a704392b414ca3f5457b92c2eb20, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:01.450803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77777c6-3c86-4766-92a5-23c66ebe2c0a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.472433 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1518/3048] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:01 2022] PUT /identity/v3/projects/fb983558bfa146c8b93b913a383fd348/users/bb54a704392b414ca3f5457b92c2eb20/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:58:01.487157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b77f3-5cc4-4449-b442-1c0f69c1d7c6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.487919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b77f3-5cc4-4449-b442-1c0f69c1d7c6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.488365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b77f3-5cc4-4449-b442-1c0f69c1d7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.527177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60c734-3cec-430a-a48b-b5553a127dfc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:01.574725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2a1b77f3-5cc4-4449-b442-1c0f69c1d7c6 None None] MFA Rules not processed for user `bb54a704392b414ca3f5457b92c2eb20`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:01.578975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a1b77f3-5cc4-4449-b442-1c0f69c1d7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:01.604184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:01.658060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:01.713775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:01.717742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.720089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.720416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.724918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:01.752564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:01.756948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.785147 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1519/3049] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:01.795435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b77f3-5cc4-4449-b442-1c0f69c1d7c6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.811313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b77f3-5cc4-4449-b442-1c0f69c1d7c6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:01.823045 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1531/3050] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:01.830340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:01.830996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.831280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.831560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.832242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:01.832542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:01.834296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.853256 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1520/3051] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:58:01.858774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.895454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059fa91-2b80-4f67-8380-6994fc21cb4c service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:01.907062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:01.907791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:01.908118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:01.908389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:01.909032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:01.916697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:01.957377 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1532/3052] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:01 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:01.961780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:01.976021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681849-c549-4b9f-ac4e-5dbdb8f2b6bb tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:02.139045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a896a-3c99-4283-bc55-ac322d979b1e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:02.141975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a896a-3c99-4283-bc55-ac322d979b1e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:02.143710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a896a-3c99-4283-bc55-ac322d979b1e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:02.143710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a896a-3c99-4283-bc55-ac322d979b1e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:02.143710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a896a-3c99-4283-bc55-ac322d979b1e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:02.144196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a896a-3c99-4283-bc55-ac322d979b1e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:02.147287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a896a-3c99-4283-bc55-ac322d979b1e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:02.183033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1521/3053] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:02.570899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:02.572264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:02.715678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84106876-6481-44e7-9aa0-030657e7d595 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:02.724887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:02.726022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:02.726458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:02.727189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:02.738611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:02.740452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:02.754621 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1533/3054] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:02.778198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751416-48b0-4c65-b09c-3a933618f5d8 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:02.778574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751416-48b0-4c65-b09c-3a933618f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:02.779143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60751416-48b0-4c65-b09c-3a933618f5d8 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:02.780915 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1522/3055] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:58:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:02.787366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:58:02.802887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:02.814949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee47c14-15b3-47a7-baca-7b96c0740d3a service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:02.820817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:02.821587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:02.821862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:02.822166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:02.823785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:get_project(project_id=0da0997d32f248d0ba6fe26ccb0e60e4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:02.826452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:02.834131 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1534/3056] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:58:02 2022] GET /identity/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4 => generated 371 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:58:03.115740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700da4-3bc5-4867-962d-2be55bcce0ea None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.116490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700da4-3bc5-4867-962d-2be55bcce0ea None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.117044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700da4-3bc5-4867-962d-2be55bcce0ea None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.119216 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1523/3057] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:58:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:03.124865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:58:03.146501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.156634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74431d32-05d8-4697-ac91-d539c79fd892 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:03.161494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:03.162158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.162663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.162796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.164101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:get_project(project_id=0da0997d32f248d0ba6fe26ccb0e60e4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:03.166620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:03.168776 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1535/3058] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:58:03 2022] GET /identity/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4 => generated 371 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:03.222735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700da4-3bc5-4867-962d-2be55bcce0ea None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.239555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700da4-3bc5-4867-962d-2be55bcce0ea None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:03.245168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:03.245858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.246271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.246720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.247310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:03.247649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:03.249743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:03.262972 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1524/3059] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:03.267779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.280062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035be2f-2ad8-430e-9db4-34c06eaeab08 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:03.284880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f72a3-740d-404d-a16b-9fd3808ea39e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:03.285528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f72a3-740d-404d-a16b-9fd3808ea39e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.285811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f72a3-740d-404d-a16b-9fd3808ea39e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.286084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f72a3-740d-404d-a16b-9fd3808ea39e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.286770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f72a3-740d-404d-a16b-9fd3808ea39e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:03.288835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f72a3-740d-404d-a16b-9fd3808ea39e tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:03.313774 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1536/3060] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:03 2022] POST /identity/v3/users => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:58:03.318137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.330077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71ad4-d2ee-45ad-8b9b-94b5ec594186 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:03.335591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:03.336426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.336777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.336965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.337976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:03.338288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:03.340080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:03.354838 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1525/3061] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:03.375288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8dc4-bc0d-4e92-a11a-694127ad8eca None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.375636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8dc4-bc0d-4e92-a11a-694127ad8eca None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.375931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af8dc4-bc0d-4e92-a11a-694127ad8eca None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.377345 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1537/3062] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:58:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:03.382677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:58:03.397998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.408247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ac732-2f83-48cb-bd08-c3aed7cc99ec tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:03.414035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:03.414950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.415262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.415532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.416741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:get_project(project_id=0da0997d32f248d0ba6fe26ccb0e60e4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:03.419092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:03.421382 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1526/3063] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:58:03 2022] GET /identity/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4 => generated 371 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:03.668335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03c055-2f16-4a01-960b-c2839db36f9c None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.669022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03c055-2f16-4a01-960b-c2839db36f9c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.669547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03c055-2f16-4a01-960b-c2839db36f9c None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.671573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1538/3064] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:58:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:03.682345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 20:58:03.704844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.717592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7fb8c-02e1-4109-95cc-d63ab4dbf402 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:03.722856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:03.723829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.724339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.724837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.726368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:get_project(project_id=0da0997d32f248d0ba6fe26ccb0e60e4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:03.728937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:03.731137 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1527/3065] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 20:58:03 2022] GET /identity/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4 => generated 371 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:58:03.778059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03c055-2f16-4a01-960b-c2839db36f9c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.799388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03c055-2f16-4a01-960b-c2839db36f9c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:03.809888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:03.810750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.811474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.811920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/users/9c420ada25594d48b3886759f0ed791f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.834440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:delete_user(user_id=9c420ada25594d48b3886759f0ed791f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:03.836870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:03.892459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:03.902090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c420ada25594d48b3886759f0ed791f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:03.921221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c420ada25594d48b3886759f0ed791f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:03.941742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c420ada25594d48b3886759f0ed791f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:03.942865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c420ada25594d48b3886759f0ed791f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:03.943828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c420ada25594d48b3886759f0ed791f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:03.968440 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1539/3066] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:03 2022] DELETE /identity/v3/users/9c420ada25594d48b3886759f0ed791f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:03.973824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:03.981123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:04.031986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bdb57-393e-4eeb-bae0-1cb71c4117c2 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:04.159046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb54a704392b414ca3f5457b92c2eb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb983558bfa146c8b93b913a383fd348', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:04.160315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:04.161391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:04.161884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] PATH_INFO: `/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:04.163369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0da0997d32f248d0ba6fe26ccb0e60e4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:04.167077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:04.170118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:04.263460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da0997d32f248d0ba6fe26ccb0e60e4', 'request_id': 'req-52edcfae-0691-43e3-a7fc-995d5914b9f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.281571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da0997d32f248d0ba6fe26ccb0e60e4', 'request_id': 'req-52edcfae-0691-43e3-a7fc-995d5914b9f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.302992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da0997d32f248d0ba6fe26ccb0e60e4', 'request_id': 'req-52edcfae-0691-43e3-a7fc-995d5914b9f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.329099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] The token cache is being invalidate because project 0da0997d32f248d0ba6fe26ccb0e60e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:04.335943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.338507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-52edcfae-0691-43e3-a7fc-995d5914b9f7 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.343177 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1528/3067] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:03 2022] DELETE /identity/v3/projects/0da0997d32f248d0ba6fe26ccb0e60e4 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:04.349398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:04.351126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:04.477718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0d31c-7ca8-4ea3-8c77-e397cf800880 tempest-QuotasAdminTestV236-1723358146 tempest-QuotasAdminTestV236-1723358146-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:04.590460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:04.590992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:04.591274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:04.591535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] PATH_INFO: `/v3/users/4c104fc78767424d858b40c2f438d99f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:04.596022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c104fc78767424d858b40c2f438d99f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:04.597593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:04.698159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c104fc78767424d858b40c2f438d99f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.739436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c104fc78767424d858b40c2f438d99f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.752736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:04.771175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c104fc78767424d858b40c2f438d99f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.772576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c104fc78767424d858b40c2f438d99f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.773979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c104fc78767424d858b40c2f438d99f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:04.809595 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1540/3068] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:04 2022] DELETE /identity/v3/users/4c104fc78767424d858b40c2f438d99f => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:04.814180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:04.814486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:04.814749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:04.858835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:04.860393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:05.016227 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1529/3069] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:05.092137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:05.093702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:05.203825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59e9b-68c1-48dc-8bfd-6a7e2fee6ee1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:05.207840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:05.208863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:05.209129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:05.209401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:05.224178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:05.225809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:05.236203 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1541/3070] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:06.830406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:06.832121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:06.852393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d080-60d5-4b02-b54b-dadcfc5e09e4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:06.860150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:06.861105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:06.861590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:06.862331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] PATH_INFO: `/v3/projects/377bd8a00ad34071a222402d67a435d2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:06.863721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=377bd8a00ad34071a222402d67a435d2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:06.865677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:07.080772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377bd8a00ad34071a222402d67a435d2', 'request_id': 'req-5e65620e-d79d-4100-b7ea-e3c293676938'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.101251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377bd8a00ad34071a222402d67a435d2', 'request_id': 'req-5e65620e-d79d-4100-b7ea-e3c293676938'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.119201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377bd8a00ad34071a222402d67a435d2', 'request_id': 'req-5e65620e-d79d-4100-b7ea-e3c293676938'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.148090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] The token cache is being invalidate because project 377bd8a00ad34071a222402d67a435d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:07.148379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.150426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.153713 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1530/3071] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:06 2022] DELETE /identity/v3/projects/377bd8a00ad34071a222402d67a435d2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:07.159125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:07.178910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813187ad-8d45-4b89-88c0-b51a8010e948 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:07.359207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:07.361915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:07.361915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:07.361915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] PATH_INFO: `/v3/users/bb54a704392b414ca3f5457b92c2eb20` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:07.366102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb54a704392b414ca3f5457b92c2eb20)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:07.368639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:07.521782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb54a704392b414ca3f5457b92c2eb20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.554230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb54a704392b414ca3f5457b92c2eb20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.575775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb54a704392b414ca3f5457b92c2eb20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.576415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb54a704392b414ca3f5457b92c2eb20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.578566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4dda7e8-f2da-4f44-bd19-99c0448bbefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb54a704392b414ca3f5457b92c2eb20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:07.606508 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1542/3072] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:07 2022] DELETE /identity/v3/users/bb54a704392b414ca3f5457b92c2eb20 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:09.420512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:09.423765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:09.464150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65620e-d79d-4100-b7ea-e3c293676938 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:09.488702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:09.647992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:09.649977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:09.650622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:09.651121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] PATH_INFO: `/v3/projects/fb983558bfa146c8b93b913a383fd348` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:09.654360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb983558bfa146c8b93b913a383fd348)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:09.659004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:09.788195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb983558bfa146c8b93b913a383fd348', 'request_id': 'req-a08da86a-cb9f-4961-8984-20ca4295b878'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:09.802284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb983558bfa146c8b93b913a383fd348', 'request_id': 'req-a08da86a-cb9f-4961-8984-20ca4295b878'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:09.816374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb983558bfa146c8b93b913a383fd348', 'request_id': 'req-a08da86a-cb9f-4961-8984-20ca4295b878'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:09.832630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] The token cache is being invalidate because project fb983558bfa146c8b93b913a383fd348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:09.832865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:09.834309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a08da86a-cb9f-4961-8984-20ca4295b878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:09.844798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ac857-a22a-484c-b639-66567ab9658b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:09.845676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ac857-a22a-484c-b639-66567ab9658b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:09.846119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ac857-a22a-484c-b639-66567ab9658b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:09.849686 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1531/3073] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:09 2022] DELETE /identity/v3/projects/fb983558bfa146c8b93b913a383fd348 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:09.869388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:09.889436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-454ac857-a22a-484c-b639-66567ab9658b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:09.897035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-454ac857-a22a-484c-b639-66567ab9658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:09.991359 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1543/3074] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:10.000476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008e70c-a0bc-452b-8a21-4d9f4b2f0d3d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.000759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008e70c-a0bc-452b-8a21-4d9f4b2f0d3d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.001425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008e70c-a0bc-452b-8a21-4d9f4b2f0d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.024461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3008e70c-a0bc-452b-8a21-4d9f4b2f0d3d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:10.026103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3008e70c-a0bc-452b-8a21-4d9f4b2f0d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:10.039150 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1532/3075] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:10.039850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ac857-a22a-484c-b639-66567ab9658b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:10.051121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ac857-a22a-484c-b639-66567ab9658b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:10.057057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:10.057961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.058420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.058851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.060788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:10.061277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:10.063438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:10.107578 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1544/3076] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:58:10.112270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008e70c-a0bc-452b-8a21-4d9f4b2f0d3d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:10.129008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008e70c-a0bc-452b-8a21-4d9f4b2f0d3d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:10.135650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:10.137191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.137546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.138264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.153782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:10.157197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:10.213053 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1533/3077] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:10 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:10.218302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:10.262849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9df1b1-69fb-4b92-b13e-a5356579a9ee admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:10.392749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:10.393396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.393689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.394076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.399455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:10.406189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:10.460988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:10.465817 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1545/3078] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:10 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:10.473177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97cc60-597e-422b-b59a-3f3132590993 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:10.481624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:10.483065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.483412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.483944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.486443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:10.488298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:10.509254 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1534/3079] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:10.521377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:10.538767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deef91-15a0-463d-8b18-7676dcd469a5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:10.548899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4741e-1012-4f64-bf8f-bad13f06176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:10.550194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4741e-1012-4f64-bf8f-bad13f06176c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.550631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4741e-1012-4f64-bf8f-bad13f06176c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.551116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4741e-1012-4f64-bf8f-bad13f06176c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.552139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4741e-1012-4f64-bf8f-bad13f06176c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:10.554622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4741e-1012-4f64-bf8f-bad13f06176c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:10.574760 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1546/3080] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:10.579418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:10.602582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaf382-bebe-4415-b42d-f9b4c9fc0f2b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:10.614986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:10.615692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.615975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.616578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] PATH_INFO: `/v3/projects/31d6f02f91d84fd59de856b10c42ac5d/users/a6645db569614a10b1388a69b7e5a64f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.656734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d6f02f91d84fd59de856b10c42ac5d, user_id=a6645db569614a10b1388a69b7e5a64f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:10.659520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:10.691103 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1535/3081] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:10 2022] PUT /identity/v3/projects/31d6f02f91d84fd59de856b10c42ac5d/users/a6645db569614a10b1388a69b7e5a64f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:10.707017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5c7a2-f363-4e60-9553-ea2ee9a0f090 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.707601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5c7a2-f363-4e60-9553-ea2ee9a0f090 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.708317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5c7a2-f363-4e60-9553-ea2ee9a0f090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.790725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-13d5c7a2-f363-4e60-9553-ea2ee9a0f090 None None] MFA Rules not processed for user `a6645db569614a10b1388a69b7e5a64f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:10.798645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-13d5c7a2-f363-4e60-9553-ea2ee9a0f090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:10.985695 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1547/3082] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:10.990405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.057409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:11.086780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44f731-e7c3-4cea-8c20-9ebfee7db628 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:11.092528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:11.093236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.093546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.093822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.094469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:11.096442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:11.132491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1536/3083] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:10 2022] POST /identity/v3/projects => generated 393 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:58:11.134494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5c7a2-f363-4e60-9553-ea2ee9a0f090 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.158401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:11.177692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5c7a2-f363-4e60-9553-ea2ee9a0f090 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:11.232744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.236432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:11.241445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:11.242363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.242363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.242630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.243271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:11.245412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:11.288747 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1548/3084] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:11 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:11.294732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.311103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab238-5ab3-4e2b-9d16-9f67ead797dc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:11.320832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:11.322009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.322295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.322851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.323559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:11.325285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:11.344260 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1549/3085] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:58:11.346935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.367308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3e47f-a5b6-421e-9db9-a3656fa0bb5a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:11.374388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:11.375335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.375619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.376063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] PATH_INFO: `/v3/projects/1b873d1590c54955a06bec83655f84de/users/e08f8a13e8bf4c119329b24e0cf90301/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.399196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3b769-f759-475d-ac1e-444e4f8e3dea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:11.401401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b873d1590c54955a06bec83655f84de, user_id=e08f8a13e8bf4c119329b24e0cf90301, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:11.405106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:11.406792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:11.407838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.408501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.408989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.412493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:11.438650 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1550/3086] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:11 2022] PUT /identity/v3/projects/1b873d1590c54955a06bec83655f84de/users/e08f8a13e8bf4c119329b24e0cf90301/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:11.439897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.442170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:11.443975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:11.498761 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1537/3087] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:58:11.641932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c54f7-aca1-4ce8-a1e3-af3e2dd51b63 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:11.651013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab2c04-31f2-40bb-945a-65f0ba5271a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:11.652016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab2c04-31f2-40bb-945a-65f0ba5271a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.652552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab2c04-31f2-40bb-945a-65f0ba5271a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.653039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab2c04-31f2-40bb-945a-65f0ba5271a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.653962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab2c04-31f2-40bb-945a-65f0ba5271a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:11.655882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab2c04-31f2-40bb-945a-65f0ba5271a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:11.669519 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1551/3088] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:11.673412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.690654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e8513-8265-49de-be07-d92f2232d098 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:11.712668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cfb74-3c00-4ad8-9940-f1a2c5564606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:11.714079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cfb74-3c00-4ad8-9940-f1a2c5564606 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.714079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cfb74-3c00-4ad8-9940-f1a2c5564606 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.714310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cfb74-3c00-4ad8-9940-f1a2c5564606 admin admin] PATH_INFO: `/v3/projects/1b873d1590c54955a06bec83655f84de/users/e08f8a13e8bf4c119329b24e0cf90301/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.717786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cfb74-3c00-4ad8-9940-f1a2c5564606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b873d1590c54955a06bec83655f84de, user_id=e08f8a13e8bf4c119329b24e0cf90301, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:11.720929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cfb74-3c00-4ad8-9940-f1a2c5564606 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:11.734215 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1538/3089] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:11 2022] PUT /identity/v3/projects/1b873d1590c54955a06bec83655f84de/users/e08f8a13e8bf4c119329b24e0cf90301/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:11.748815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92efcf1-731c-4eea-89fd-055096ec27eb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.749569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92efcf1-731c-4eea-89fd-055096ec27eb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.750060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92efcf1-731c-4eea-89fd-055096ec27eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.799120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d92efcf1-731c-4eea-89fd-055096ec27eb None None] MFA Rules not processed for user `e08f8a13e8bf4c119329b24e0cf90301`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:11.800920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d92efcf1-731c-4eea-89fd-055096ec27eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:11.860401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:11.911803 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1552/3090] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:11.924071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.924359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.924839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.952107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] MFA Rules not processed for user `a6645db569614a10b1388a69b7e5a64f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:11.953600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:12.059235 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1539/3091] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:12.584678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92efcf1-731c-4eea-89fd-055096ec27eb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:12.757334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92efcf1-731c-4eea-89fd-055096ec27eb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:12.772847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:12.774855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:12.775171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:12.775846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:12.814994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:12.816590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:12.854860 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1553/3092] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:13.986331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:13.989058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:14.111154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:14.204403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3006c-9bf4-4754-8ac0-0a3adbd4ab56 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:14.210420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:14.211045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:14.211323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:14.211594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:14.213571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:14.229757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:14.231430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:14.244452 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1540/3093] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:15.750791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:15.768785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d33d-7207-490d-a111-475f00515fae service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:15.782447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:15.783069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:15.783677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:15.784148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:15.789324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:15.819548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:15.821450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:15.835024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1554/3094] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:16.472732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:16.474437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:16.507353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e72053-1bda-44d8-a1f5-6f73d3b96e9e service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:16.517030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9816b-e059-46a2-aa90-b99555e6f4a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:16.518215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9816b-e059-46a2-aa90-b99555e6f4a7 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:16.518692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9816b-e059-46a2-aa90-b99555e6f4a7 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:16.519554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9816b-e059-46a2-aa90-b99555e6f4a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:16.522597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5fe9816b-e059-46a2-aa90-b99555e6f4a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:16.552274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9816b-e059-46a2-aa90-b99555e6f4a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:16.554224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9816b-e059-46a2-aa90-b99555e6f4a7 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:16.566672 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1541/3095] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:18.267578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:18.269057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:18.332406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7dd03-6851-4e16-9e5a-f9f277092e41 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:18.336535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a978ef-32d9-4195-8d95-1af089b8df09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:18.337159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a978ef-32d9-4195-8d95-1af089b8df09 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:18.337502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a978ef-32d9-4195-8d95-1af089b8df09 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:18.338018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a978ef-32d9-4195-8d95-1af089b8df09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:18.340293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-14a978ef-32d9-4195-8d95-1af089b8df09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:18.524789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a978ef-32d9-4195-8d95-1af089b8df09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:18.528573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a978ef-32d9-4195-8d95-1af089b8df09 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:18.545225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:18.572124 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1555/3096] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:23.184630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057c3c-513f-4bad-8a00-945c9c16fb51 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:23.185293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057c3c-513f-4bad-8a00-945c9c16fb51 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:23.185428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057c3c-513f-4bad-8a00-945c9c16fb51 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:23.189206 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1542/3097] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 20:58:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:23.198895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29f7f-cb2c-48ce-affc-9f6e11ec3ef1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:23.199453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29f7f-cb2c-48ce-affc-9f6e11ec3ef1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:23.199562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29f7f-cb2c-48ce-affc-9f6e11ec3ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:23.233967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9dc29f7f-cb2c-48ce-affc-9f6e11ec3ef1 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:23.236282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9dc29f7f-cb2c-48ce-affc-9f6e11ec3ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:23.256622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1556/3098] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:23.348604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057c3c-513f-4bad-8a00-945c9c16fb51 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:23.368641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057c3c-513f-4bad-8a00-945c9c16fb51 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:23.384253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:23.385261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:23.386249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:23.387032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:23.399581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:23.402387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:23.417098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1543/3099] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:25.178596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:25.179156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:25.179262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:25.201454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:25.202795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:25.299014 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1557/3100] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:27.386021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:27.402419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:27.403863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:27.422159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ed3ac-6179-4640-bdda-c9cad9beae89 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:27.432125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8123-5495-458e-8461-be5a66cb5ce0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:27.433207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8123-5495-458e-8461-be5a66cb5ce0 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:27.433790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8123-5495-458e-8461-be5a66cb5ce0 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:27.434452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8123-5495-458e-8461-be5a66cb5ce0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:27.437294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2ddd8123-5495-458e-8461-be5a66cb5ce0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:27.449186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22c216-fc97-4e80-9157-a9f26244638c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:27.453559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd8123-5495-458e-8461-be5a66cb5ce0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:27.455718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd8123-5495-458e-8461-be5a66cb5ce0 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:27.470337 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1544/3101] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:27.491958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:27.495957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:27.496563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:27.496846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] PATH_INFO: `/v3/users/bc22daf24ce44eb28bb6e000316e4df4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:27.500316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc22daf24ce44eb28bb6e000316e4df4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:27.506093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:27.632651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc22daf24ce44eb28bb6e000316e4df4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:27.646640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc22daf24ce44eb28bb6e000316e4df4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:27.659664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc22daf24ce44eb28bb6e000316e4df4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:27.660731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc22daf24ce44eb28bb6e000316e4df4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:27.662213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc22daf24ce44eb28bb6e000316e4df4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:27.706798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203346d7-19c1-48cc-9f9b-c9f229f1c891 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:27.707746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203346d7-19c1-48cc-9f9b-c9f229f1c891 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:27.712757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203346d7-19c1-48cc-9f9b-c9f229f1c891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:27.727718 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1558/3102] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:27 2022] DELETE /identity/v3/users/bc22daf24ce44eb28bb6e000316e4df4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:27.750150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-203346d7-19c1-48cc-9f9b-c9f229f1c891 None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:27.755613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-203346d7-19c1-48cc-9f9b-c9f229f1c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:27.833195 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1545/3103] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:28.354880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:28.356610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:28.510526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36526b5b-0ed0-4613-8a1f-d0903e3798b1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:28.524548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:28.525416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:28.526046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:28.526494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] PATH_INFO: `/v3/projects/400242d681ab472d9c90de4fc7da202b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:28.528000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=400242d681ab472d9c90de4fc7da202b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:28.530395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:28.546847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:28.634665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '400242d681ab472d9c90de4fc7da202b', 'request_id': 'req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:28.645205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '400242d681ab472d9c90de4fc7da202b', 'request_id': 'req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:28.654731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '400242d681ab472d9c90de4fc7da202b', 'request_id': 'req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:28.673027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] The token cache is being invalidate because project 400242d681ab472d9c90de4fc7da202b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:28.673379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:28.674160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee20160a-7cbf-4b12-b02c-acd3a7f2e8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:28.676324 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1559/3104] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:28 2022] DELETE /identity/v3/projects/400242d681ab472d9c90de4fc7da202b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:28.680720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f5c43-2d1f-4138-bfa9-5df1c132b016 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:28.681580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f5c43-2d1f-4138-bfa9-5df1c132b016 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:28.683366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f5c43-2d1f-4138-bfa9-5df1c132b016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:28.735940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0c5f5c43-2d1f-4138-bfa9-5df1c132b016 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:28.738131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c5f5c43-2d1f-4138-bfa9-5df1c132b016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:28.945982 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1546/3105] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:28.961790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae2f4-122e-4fa4-a9a6-7038303ca528 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:28.962469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae2f4-122e-4fa4-a9a6-7038303ca528 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:28.963124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae2f4-122e-4fa4-a9a6-7038303ca528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.032313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2ebae2f4-122e-4fa4-a9a6-7038303ca528 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:29.035503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2ebae2f4-122e-4fa4-a9a6-7038303ca528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:29.073523 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1560/3106] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:29.077048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f5c43-2d1f-4138-bfa9-5df1c132b016 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.093133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f5c43-2d1f-4138-bfa9-5df1c132b016 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:29.102924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:29.103553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.103825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.104095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.104748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:29.105059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:29.110238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:29.142159 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1547/3107] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:29.150145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebae2f4-122e-4fa4-a9a6-7038303ca528 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.172709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebae2f4-122e-4fa4-a9a6-7038303ca528 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:29.187433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf0967-1bfa-4f5d-8bba-d7c1430e8345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:29.188773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0967-1bfa-4f5d-8bba-d7c1430e8345 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.189351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0967-1bfa-4f5d-8bba-d7c1430e8345 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.189996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0967-1bfa-4f5d-8bba-d7c1430e8345 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.192015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf0967-1bfa-4f5d-8bba-d7c1430e8345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:29.194663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf0967-1bfa-4f5d-8bba-d7c1430e8345 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:29.294256 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1561/3108] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:29 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:29.295152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.302966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:29.310153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838ff11-243e-4005-b42a-9088876b9c21 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:29.407588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.407894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.408173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.446549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:29.447805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:29.455510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:29.456101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.456709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.456982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.457711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:29.459969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:29.503767 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1548/3109] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:29 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:29.505505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.526751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69701e22-4fb6-4a36-a2f7-1b3afa818a9e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:29.534047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:29.535170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.535547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.536444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.538252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:29.540861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:29.562434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1549/3110] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:29.563820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.578151 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1562/3111] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:29.593644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.593870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560b283-a2c6-43ae-a98f-da55931b9405 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:29.595901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:29.601428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:29.603046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.603517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.603789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.604598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:29.609677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:29.631314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1550/3112] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:29.632533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.648576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669742d-31ac-45d8-86d6-d7410afdacd2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:29.662043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee2f0a-8f4b-42f2-a2eb-2f9160b1a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:29.664268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2f0a-8f4b-42f2-a2eb-2f9160b1a996 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.665090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2f0a-8f4b-42f2-a2eb-2f9160b1a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.665090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2f0a-8f4b-42f2-a2eb-2f9160b1a996 admin admin] PATH_INFO: `/v3/projects/d957f65ae29b4829b478e118ed2f89d5/users/87ee132f0e2f42f69b149570329b4855/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.699619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee2f0a-8f4b-42f2-a2eb-2f9160b1a996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d957f65ae29b4829b478e118ed2f89d5, user_id=87ee132f0e2f42f69b149570329b4855, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:29.702382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee2f0a-8f4b-42f2-a2eb-2f9160b1a996 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:29.710189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba9bca-eb0e-4156-a830-638587879d00 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:29.717633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:29.718674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.719032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.719624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.729085 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1551/3113] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:29 2022] PUT /identity/v3/projects/d957f65ae29b4829b478e118ed2f89d5/users/87ee132f0e2f42f69b149570329b4855/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:29.738613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:29.740315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:29.740720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f09b6-16ce-4051-8718-0d45edf04cd2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.741165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f09b6-16ce-4051-8718-0d45edf04cd2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.741624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f09b6-16ce-4051-8718-0d45edf04cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.766622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1563/3114] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:58:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 20:58:29.786642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-be7f09b6-16ce-4051-8718-0d45edf04cd2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:29.787864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-be7f09b6-16ce-4051-8718-0d45edf04cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:29.948784 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1552/3115] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:30.047952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:30.055927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:30.070798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:30.216961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432d63-40a1-4bf0-b93a-3c0d8b65ad02 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:30.385600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:30.386229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:30.386503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:30.387349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:30.404945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:30.407041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:30.441969 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1564/3116] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:32.139225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e101d78-0f12-48ff-b966-48c8cce12433 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:32.139879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e101d78-0f12-48ff-b966-48c8cce12433 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:32.140003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e101d78-0f12-48ff-b966-48c8cce12433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:32.192827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3e101d78-0f12-48ff-b966-48c8cce12433 None None] MFA Rules not processed for user `0d48260054d84784b89a52d9b766d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:32.198223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3e101d78-0f12-48ff-b966-48c8cce12433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:32.259770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:32.279729 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1553/3117] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:37.425130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:37.426683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:37.444731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be54071-03f7-40e0-ba0b-533585799568 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:37.450721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:37.451925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:37.452390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:37.452833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] PATH_INFO: `/v3/users/9c2b746b7c194596808a7b0c736f3b53` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:37.455107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c2b746b7c194596808a7b0c736f3b53)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:37.457892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:37.628033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c2b746b7c194596808a7b0c736f3b53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:37.656841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c2b746b7c194596808a7b0c736f3b53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:37.675888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c2b746b7c194596808a7b0c736f3b53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:37.677496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c2b746b7c194596808a7b0c736f3b53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:37.678189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c2b746b7c194596808a7b0c736f3b53'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:37.718705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1565/3118] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:37 2022] DELETE /identity/v3/users/9c2b746b7c194596808a7b0c736f3b53 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:39.429108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e101d78-0f12-48ff-b966-48c8cce12433 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:39.447556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e101d78-0f12-48ff-b966-48c8cce12433 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:39.578567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:39.579191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:39.579479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:39.579754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] PATH_INFO: `/v3/projects/f6b44bc44e7c4a998bfd99e7d11e99fc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:39.581667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b44bc44e7c4a998bfd99e7d11e99fc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:39.587413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:39.703799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b44bc44e7c4a998bfd99e7d11e99fc', 'request_id': 'req-88a79004-9afa-47c7-a9d9-49e0df73b815'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:39.722757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b44bc44e7c4a998bfd99e7d11e99fc', 'request_id': 'req-88a79004-9afa-47c7-a9d9-49e0df73b815'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:39.740551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b44bc44e7c4a998bfd99e7d11e99fc', 'request_id': 'req-88a79004-9afa-47c7-a9d9-49e0df73b815'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:39.755581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] The token cache is being invalidate because project f6b44bc44e7c4a998bfd99e7d11e99fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:39.755883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:39.756548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:39.761269 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1554/3119] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:39 2022] DELETE /identity/v3/projects/f6b44bc44e7c4a998bfd99e7d11e99fc => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:39.764113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:39.766879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:39.813883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41daba63-0e9a-45af-b26e-4a0cce376059 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:39.863133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:39.900785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:39.901718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:39.902965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:39.903415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] PATH_INFO: `/v3/users/eda2d41b016543d98d09539949cc1e72` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:39.905777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda2d41b016543d98d09539949cc1e72)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:39.908232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:40.000311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda2d41b016543d98d09539949cc1e72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:40.017526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda2d41b016543d98d09539949cc1e72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:40.045119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda2d41b016543d98d09539949cc1e72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:40.045851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda2d41b016543d98d09539949cc1e72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:40.046552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda2d41b016543d98d09539949cc1e72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:40.092350 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1566/3120] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:39 2022] DELETE /identity/v3/users/eda2d41b016543d98d09539949cc1e72 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:41.299796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:41.301790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:41.328557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a79004-9afa-47c7-a9d9-49e0df73b815 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:41.470319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:41.471433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:41.471966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:41.472572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] PATH_INFO: `/v3/users/a6645db569614a10b1388a69b7e5a64f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:41.474161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6645db569614a10b1388a69b7e5a64f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:41.477982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:41.524474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:41.559421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6645db569614a10b1388a69b7e5a64f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:41.560626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6645db569614a10b1388a69b7e5a64f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:41.614245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6645db569614a10b1388a69b7e5a64f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:41.616579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6645db569614a10b1388a69b7e5a64f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:41.646411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:41.647986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:41.650622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6661c73e-a35b-4d56-9d7d-7966bd302296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6645db569614a10b1388a69b7e5a64f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:41.672193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1555/3121] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:41 2022] DELETE /identity/v3/users/a6645db569614a10b1388a69b7e5a64f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:41.676763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:41.677047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:41.677314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:41.690023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367707b-4d57-4ded-86ce-e6a07f3abb3b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:41.731885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:41.735799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:41.850223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:41.851006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:41.851468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:41.851751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] PATH_INFO: `/v3/projects/c3928c4d218a4173b46e6ca792c337d1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:41.858334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3928c4d218a4173b46e6ca792c337d1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:41.863074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:41.900739 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1556/3122] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:41.998051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3928c4d218a4173b46e6ca792c337d1', 'request_id': 'req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:42.017289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:42.019165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3928c4d218a4173b46e6ca792c337d1', 'request_id': 'req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:42.021360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:42.022279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:42.043962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3928c4d218a4173b46e6ca792c337d1', 'request_id': 'req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:42.070077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] The token cache is being invalidate because project c3928c4d218a4173b46e6ca792c337d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:42.070399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:42.074194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c4f42582-d3ea-41fa-80ec-a5f19c65e63c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:42.080328 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1567/3123] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:41 2022] DELETE /identity/v3/projects/c3928c4d218a4173b46e6ca792c337d1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:42.082977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4ff1-a3c9-4e6f-aa8b-9b668018ae10 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.083265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4ff1-a3c9-4e6f-aa8b-9b668018ae10 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.083537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4ff1-a3c9-4e6f-aa8b-9b668018ae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.127945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2bbd4ff1-a3c9-4e6f-aa8b-9b668018ae10 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:42.129208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2bbd4ff1-a3c9-4e6f-aa8b-9b668018ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:42.187235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec6971-d5f9-455a-83f7-7a81e21e13ec None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.268575 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1568/3124] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:42.277325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3ae24-4fde-46fa-b2d5-9694d709686f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.277640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3ae24-4fde-46fa-b2d5-9694d709686f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.277916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3ae24-4fde-46fa-b2d5-9694d709686f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.298135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.299368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.299647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.299926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.306541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:42.316438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a7f3ae24-4fde-46fa-b2d5-9694d709686f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:42.316438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7f3ae24-4fde-46fa-b2d5-9694d709686f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:42.332726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.334234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.336355 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1569/3125] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:42.340686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3ae24-4fde-46fa-b2d5-9694d709686f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:42.359811 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1557/3126] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:42.360360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3ae24-4fde-46fa-b2d5-9694d709686f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.375441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.375441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.375441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.375821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.376233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:42.376531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.380544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.414206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1570/3127] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:42.415328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:42.421212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:42.436830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae040e-9877-4c7b-ad3a-57497781a67d service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.447789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.449864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.450147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.450421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.451413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.453393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.514895 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1558/3128] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:42 2022] POST /identity/v3/projects => generated 361 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:42.516441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:42.538876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9263b8-df12-4236-bd78-c28e973aba1f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.680792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.681816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.682134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.682414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.683072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.685076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.741537 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1571/3129] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:42 2022] POST /identity/v3/users => generated 281 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:42.742036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:42.760046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4c69d-a49a-4485-a309-b6c90e990d04 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.770398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.770931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.771209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.771482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.772186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.773799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.798163 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1559/3130] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:58:42.799460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:42.814416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09011dd0-949c-4ff6-b117-3e85b669982f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.835072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ee954-b688-4190-b3fa-52046c6d1eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.835983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ee954-b688-4190-b3fa-52046c6d1eac admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.835983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ee954-b688-4190-b3fa-52046c6d1eac admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.836290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ee954-b688-4190-b3fa-52046c6d1eac admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.836883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ee954-b688-4190-b3fa-52046c6d1eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.838550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ee954-b688-4190-b3fa-52046c6d1eac admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.892815 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1572/3131] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:42.894962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:42.907699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f080766-69d4-41b7-8ce3-7f52adc3b4e2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.920622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.921304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.921618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.921968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] PATH_INFO: `/v3/projects/b7e4562cc5ba4042aeaf004834de8312/users/edc3526e73d047c395a1c355e259d27a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.956405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7e4562cc5ba4042aeaf004834de8312, user_id=edc3526e73d047c395a1c355e259d27a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.959233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.981587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1560/3132] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:42 2022] PUT /identity/v3/projects/b7e4562cc5ba4042aeaf004834de8312/users/edc3526e73d047c395a1c355e259d27a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:42.992162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605255a5-8234-4d0d-9be8-5101f2acc2b6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.992458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605255a5-8234-4d0d-9be8-5101f2acc2b6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.992740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605255a5-8234-4d0d-9be8-5101f2acc2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:43.051786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-605255a5-8234-4d0d-9be8-5101f2acc2b6 None None] MFA Rules not processed for user `edc3526e73d047c395a1c355e259d27a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:43.053032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-605255a5-8234-4d0d-9be8-5101f2acc2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:43.126885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:43.202436 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1573/3133] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:43.206790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:43.367481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15093-60d3-4c97-b893-361ab3fa82e2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:43.380355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:43.380998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:43.381498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:43.381912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:43.382709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:43.389296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:43.452115 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1561/3134] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:43 2022] POST /identity/v3/projects => generated 359 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:43.456459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605255a5-8234-4d0d-9be8-5101f2acc2b6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:43.474411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605255a5-8234-4d0d-9be8-5101f2acc2b6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:43.623778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:43.624417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:43.624696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:43.624969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:43.625887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:43.627904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:43.682852 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1574/3135] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:43 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:43.687163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:43.703647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b58493-3a3e-40de-808f-b7be8ad862eb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:43.719348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:43.719983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:43.720626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:43.720754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:43.721329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:43.722981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:43.751481 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1562/3136] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:43.753490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:43.768273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156d53a-a71a-4f59-a971-e51110e8c096 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:43.781437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:43.782146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:43.782422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:43.782690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] PATH_INFO: `/v3/projects/5ed0bfddfd6241a2b5a8a58aed84b662/users/d5b91eddf20f44ef8102038757d6d296/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:43.830402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed0bfddfd6241a2b5a8a58aed84b662, user_id=d5b91eddf20f44ef8102038757d6d296, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:43.833109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:43.873043 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1575/3137] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:43 2022] PUT /identity/v3/projects/5ed0bfddfd6241a2b5a8a58aed84b662/users/d5b91eddf20f44ef8102038757d6d296/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:43.874948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:43.876230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:43.880040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:43.899049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0fc52-c889-4076-8e70-a1957dafa602 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:44.130291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:44.130940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.131217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.131489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.132152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:44.133808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:44.148003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02efc-fe95-4380-80ff-dae4f8ed4d49 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:44.174186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:44.177698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.178238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.178728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] PATH_INFO: `/v3/users/8db879fde0bd4bea81374a7f7bb627e6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.181475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db879fde0bd4bea81374a7f7bb627e6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:44.182998 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1563/3138] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:44.184467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:44.185966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:44.249130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f87f7b-aebb-4d61-ba4e-2dd82e7704ba admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:44.265900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:44.266905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.268199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:44.268710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.269377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] PATH_INFO: `/v3/projects/5ed0bfddfd6241a2b5a8a58aed84b662/users/d5b91eddf20f44ef8102038757d6d296/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.274886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db879fde0bd4bea81374a7f7bb627e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.275233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed0bfddfd6241a2b5a8a58aed84b662, user_id=d5b91eddf20f44ef8102038757d6d296, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:44.289355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:44.318219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db879fde0bd4bea81374a7f7bb627e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.324170 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1564/3139] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:44 2022] PUT /identity/v3/projects/5ed0bfddfd6241a2b5a8a58aed84b662/users/d5b91eddf20f44ef8102038757d6d296/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:44.326397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:44.328868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:44.341339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:44.346488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db879fde0bd4bea81374a7f7bb627e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.347949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db879fde0bd4bea81374a7f7bb627e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.351822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e63a962-ecd1-4473-9e76-87c545a43bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db879fde0bd4bea81374a7f7bb627e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.388940 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1576/3140] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:43 2022] DELETE /identity/v3/users/8db879fde0bd4bea81374a7f7bb627e6 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:44.390868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.394532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.395056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.466433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] MFA Rules not processed for user `d5b91eddf20f44ef8102038757d6d296`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:44.469520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:44.477304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188b9e1-8329-4695-a711-dce2bcbb61f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:44.573386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:44.574625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.574893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.575160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] PATH_INFO: `/v3/projects/31d6f02f91d84fd59de856b10c42ac5d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.577816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d6f02f91d84fd59de856b10c42ac5d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:44.580114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:44.612048 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1577/3141] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:44.620974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:44.623309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:44.678390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d6f02f91d84fd59de856b10c42ac5d', 'request_id': 'req-40a0da07-7dee-452a-9cf4-5b73c69e3335'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.686471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d6f02f91d84fd59de856b10c42ac5d', 'request_id': 'req-40a0da07-7dee-452a-9cf4-5b73c69e3335'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.706776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d6f02f91d84fd59de856b10c42ac5d', 'request_id': 'req-40a0da07-7dee-452a-9cf4-5b73c69e3335'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.745661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] The token cache is being invalidate because project 31d6f02f91d84fd59de856b10c42ac5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:44.745976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.746911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:44.748667 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1565/3142] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:44 2022] DELETE /identity/v3/projects/31d6f02f91d84fd59de856b10c42ac5d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:44.752790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:44.755464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:44.773287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7e053-cb74-40ba-acdd-b10be7463ee1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:44.885777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:44.892239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:44.945010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:44.945682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.945965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.946368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.948479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:45.026900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0da07-7dee-452a-9cf4-5b73c69e3335 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:45.061944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:45.062624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:45.062907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:45.063182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] PATH_INFO: `/v3/users/e08f8a13e8bf4c119329b24e0cf90301` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:45.067176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08f8a13e8bf4c119329b24e0cf90301)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:45.073560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:45.234259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:45.235927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:45.250657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1578/3143] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:58:45.252833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:45.254614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:45.264157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08f8a13e8bf4c119329b24e0cf90301'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.269976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08f8a13e8bf4c119329b24e0cf90301'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.272327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c418-3e80-40ca-9563-525ab8cc0dcb service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:45.277289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:45.278238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:45.278658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:45.279100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] PATH_INFO: `/v3/projects/044cdbed43ed4de6b4cb7e1cd4802b65` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:45.280442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=044cdbed43ed4de6b4cb7e1cd4802b65)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:45.282599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:45.308053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08f8a13e8bf4c119329b24e0cf90301'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.308670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08f8a13e8bf4c119329b24e0cf90301'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.332722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aab63494-bd06-48c3-ae5f-c9af50561b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08f8a13e8bf4c119329b24e0cf90301'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.364295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044cdbed43ed4de6b4cb7e1cd4802b65', 'request_id': 'req-87dee07c-b93c-42e0-b2c1-aff255f8f490'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.366628 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1566/3144] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:44 2022] DELETE /identity/v3/users/e08f8a13e8bf4c119329b24e0cf90301 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:45.381107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044cdbed43ed4de6b4cb7e1cd4802b65', 'request_id': 'req-87dee07c-b93c-42e0-b2c1-aff255f8f490'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.392817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044cdbed43ed4de6b4cb7e1cd4802b65', 'request_id': 'req-87dee07c-b93c-42e0-b2c1-aff255f8f490'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.424608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] The token cache is being invalidate because project 044cdbed43ed4de6b4cb7e1cd4802b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:45.425882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.426758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87dee07c-b93c-42e0-b2c1-aff255f8f490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:45.429363 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1579/3145] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:45 2022] DELETE /identity/v3/projects/044cdbed43ed4de6b4cb7e1cd4802b65 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:45.434273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc455-6c87-42b0-8a77-65751e05f1b5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:45.434731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc455-6c87-42b0-8a77-65751e05f1b5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:45.435237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc455-6c87-42b0-8a77-65751e05f1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:45.503820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-09bcc455-6c87-42b0-8a77-65751e05f1b5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:45.512111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-09bcc455-6c87-42b0-8a77-65751e05f1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:45.795397 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1567/3146] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:45.797023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234682d-dc4b-4213-ad2d-b163cc4c45bd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:45.797577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234682d-dc4b-4213-ad2d-b163cc4c45bd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:45.797799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234682d-dc4b-4213-ad2d-b163cc4c45bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:45.838712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2234682d-dc4b-4213-ad2d-b163cc4c45bd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:45.839952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2234682d-dc4b-4213-ad2d-b163cc4c45bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:45.855291 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1580/3147] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:45.859722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcc455-6c87-42b0-8a77-65751e05f1b5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:45.877956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcc455-6c87-42b0-8a77-65751e05f1b5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:45.893814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:45.895702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:45.896201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:45.897017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:45.900204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:45.900756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:45.904450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:45.940965 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1568/3148] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:45.943974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234682d-dc4b-4213-ad2d-b163cc4c45bd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:45.954803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234682d-dc4b-4213-ad2d-b163cc4c45bd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:45.969914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:45.972070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:45.972390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:45.972603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:45.973747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:45.975694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:46.039375 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1581/3149] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:45 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:46.043467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:46.064337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c344-6b6e-46e2-aa55-1cbe38195f2b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:46.168868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:46.205661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:46.206958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:46.207379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:46.208082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:46.208745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:46.212292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:46.261068 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1569/3150] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:46 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:46.261700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:46.278013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed355aa-0f5d-4c48-a8b1-e329e33da521 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:46.292888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:46.293582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:46.293915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:46.294133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:46.294813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:46.296447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:46.319806 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1582/3151] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:46.325734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:46.351016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:46.351880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f836e-feea-41cc-9397-585a7a48a6db admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:46.353089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:46.358466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:46.359642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:46.359642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:46.360131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:46.360879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:46.362714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:46.374140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49834aaa-0c97-4a06-a4b5-36e3f79d604a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:46.380588 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1570/3152] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:46.381798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:46.384904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:46.385709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:46.385982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:46.387438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] PATH_INFO: `/v3/users/edc3526e73d047c395a1c355e259d27a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:46.390965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc3526e73d047c395a1c355e259d27a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:46.396401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b524e-2a5a-46b1-8420-a10557dfd6bf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:46.396598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:46.402471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945ead0-d00e-429d-a54d-4ae9e96734d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:46.403345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945ead0-d00e-429d-a54d-4ae9e96734d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:46.403759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945ead0-d00e-429d-a54d-4ae9e96734d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:46.404338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945ead0-d00e-429d-a54d-4ae9e96734d9 admin admin] PATH_INFO: `/v3/projects/9b45a150f2db4c2bb4871cd1d1cab898/users/da76bbe04c22456f957be0675892359d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:46.429877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:46.442819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945ead0-d00e-429d-a54d-4ae9e96734d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b45a150f2db4c2bb4871cd1d1cab898, user_id=da76bbe04c22456f957be0675892359d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:46.447919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945ead0-d00e-429d-a54d-4ae9e96734d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:46.499674 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1571/3153] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:46 2022] PUT /identity/v3/projects/9b45a150f2db4c2bb4871cd1d1cab898/users/da76bbe04c22456f957be0675892359d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:46.504329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc3526e73d047c395a1c355e259d27a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:46.510340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:46.510980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:46.511270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:46.529441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc3526e73d047c395a1c355e259d27a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:46.561760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:46.563100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:46.569064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc3526e73d047c395a1c355e259d27a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:46.570322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc3526e73d047c395a1c355e259d27a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:46.571656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0735f936-7548-41f8-97c7-772134b46ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc3526e73d047c395a1c355e259d27a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:46.610798 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1583/3154] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:46 2022] DELETE /identity/v3/users/edc3526e73d047c395a1c355e259d27a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:58:46.618974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:46.620094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:46.620828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:46.673176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:46.673176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:46.749517 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1572/3155] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:46.781226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:46.783413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:46.880147 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1584/3156] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:46.899450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:46.901488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:46.953600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e2825-157d-4541-8446-fc8372a757b6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:47.097855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a768dd-5b5a-4b06-92cc-584cce3ed4cc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:47.106379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:47.108381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.108659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.109017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.130695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead9f79-0a21-46ab-b874-2469029d0c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:47.131725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9f79-0a21-46ab-b874-2469029d0c64 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.131999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9f79-0a21-46ab-b874-2469029d0c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.134747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9f79-0a21-46ab-b874-2469029d0c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.137537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:47.139867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:47.159069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead9f79-0a21-46ab-b874-2469029d0c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:47.163520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead9f79-0a21-46ab-b874-2469029d0c64 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:47.172601 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1585/3157] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:47.176558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:47.184146 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1573/3158] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:47.289923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:47.291414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:47.328276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fcdd9-3807-409a-afe5-efa5b0fc755d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:47.337723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:47.338700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.339114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.339520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] PATH_INFO: `/v3/projects/1b873d1590c54955a06bec83655f84de` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.342972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b873d1590c54955a06bec83655f84de)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:47.346626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:47.438984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:47.461870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b873d1590c54955a06bec83655f84de', 'request_id': 'req-792f27d5-f259-4c2a-9f41-fb4e3610f745'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:47.482939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b873d1590c54955a06bec83655f84de', 'request_id': 'req-792f27d5-f259-4c2a-9f41-fb4e3610f745'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:47.501859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b873d1590c54955a06bec83655f84de', 'request_id': 'req-792f27d5-f259-4c2a-9f41-fb4e3610f745'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:47.524334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] The token cache is being invalidate because project 1b873d1590c54955a06bec83655f84de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:47.524859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:47.525831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792f27d5-f259-4c2a-9f41-fb4e3610f745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:47.529119 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1586/3159] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:47 2022] DELETE /identity/v3/projects/1b873d1590c54955a06bec83655f84de => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:47.534237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2389ca-4db0-4c52-9082-08cd691fdaf9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.534698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2389ca-4db0-4c52-9082-08cd691fdaf9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.534775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2389ca-4db0-4c52-9082-08cd691fdaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.573962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ef2389ca-4db0-4c52-9082-08cd691fdaf9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:47.575409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ef2389ca-4db0-4c52-9082-08cd691fdaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:47.727396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1574/3160] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:47.737353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994dd03-87df-4d16-9ff7-cc805411fc7b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.737783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994dd03-87df-4d16-9ff7-cc805411fc7b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.738063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994dd03-87df-4d16-9ff7-cc805411fc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.767299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3994dd03-87df-4d16-9ff7-cc805411fc7b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:47.768210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3994dd03-87df-4d16-9ff7-cc805411fc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:47.781744 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1587/3161] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:47.786557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2389ca-4db0-4c52-9082-08cd691fdaf9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:47.801328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2389ca-4db0-4c52-9082-08cd691fdaf9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:47.807047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:47.808270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.809215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.809693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.810535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:47.810882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:47.812833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:47.843080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994dd03-87df-4d16-9ff7-cc805411fc7b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:47.853577 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1575/3162] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:47.854227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994dd03-87df-4d16-9ff7-cc805411fc7b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:47.859510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:47.860381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.860381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.860712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.861835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:47.864588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:47.916614 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1588/3163] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:47 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:47.921223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:47.935252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeba975-9533-4099-b252-84ba5afed2bf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:48.027113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:48.027781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.028225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.028540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.030369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:48.033130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:48.089213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:48.106924 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1576/3164] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:47 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:48.109674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000015-83a3-43ef-b6a0-23ef2af28e1a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:48.116279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:48.116906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.117186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.117521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.118241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:48.120029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:48.139097 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1589/3165] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:48.143694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:48.157181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa5212-b9f6-413c-8d68-ed021c958daf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:48.163422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf60e65-b7b2-4122-bbfb-2ad7fde05f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:48.164102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf60e65-b7b2-4122-bbfb-2ad7fde05f2b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.164381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf60e65-b7b2-4122-bbfb-2ad7fde05f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.164695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf60e65-b7b2-4122-bbfb-2ad7fde05f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.165522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf60e65-b7b2-4122-bbfb-2ad7fde05f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:48.168081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf60e65-b7b2-4122-bbfb-2ad7fde05f2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:48.189857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:48.202791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1577/3166] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:58:48.204997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06960950-4f90-4013-bcb2-7477ca2093a6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:48.211499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:48.212172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.212547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.212821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] PATH_INFO: `/v3/projects/b90e3cb8cd5e487f88be77242e41eb44/users/930853ac3d5c4d669b0379f1b0987aac/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.237498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90e3cb8cd5e487f88be77242e41eb44, user_id=930853ac3d5c4d669b0379f1b0987aac, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:48.241209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:48.320424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.320424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.320677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.321228 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1590/3167] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:48 2022] PUT /identity/v3/projects/b90e3cb8cd5e487f88be77242e41eb44/users/930853ac3d5c4d669b0379f1b0987aac/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:48.377309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] MFA Rules not processed for user `930853ac3d5c4d669b0379f1b0987aac`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:48.378883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:48.552823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:48.592479 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1578/3168] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:48.597801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:48.620361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:48.621851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:48.800358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b8657-fe68-4fd4-b991-62a82a9317c4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:48.804056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa559ee-ae59-4681-a468-cdcd40f8906a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:48.810228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:48.810880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.811156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.811589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.812070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:48.819634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:48.824010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:48.824667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.824946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.825219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] PATH_INFO: `/v3/projects/b7e4562cc5ba4042aeaf004834de8312` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.826642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7e4562cc5ba4042aeaf004834de8312)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:48.828787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:48.942166 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1591/3169] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:48 2022] POST /identity/v3/projects => generated 379 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:48.969816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:48.975592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7e4562cc5ba4042aeaf004834de8312', 'request_id': 'req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.004461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7e4562cc5ba4042aeaf004834de8312', 'request_id': 'req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.018673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7e4562cc5ba4042aeaf004834de8312', 'request_id': 'req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.045047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] The token cache is being invalidate because project b7e4562cc5ba4042aeaf004834de8312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:49.045380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.051475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.056835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1579/3170] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:48 2022] DELETE /identity/v3/projects/b7e4562cc5ba4042aeaf004834de8312 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:58:49.058372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:49.064294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:49.148360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:49.224681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7cedc-a84d-483a-9702-acf5d343316b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:49.250697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:49.251563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:49.251977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:49.254251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:49.255435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:49.260780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:49.269374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014b090-f4b8-4e3e-b5f3-dbfb5e1e5e21 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:49.282907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:49.283672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:49.284036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:49.284511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] PATH_INFO: `/v3/users/d5b91eddf20f44ef8102038757d6d296` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:49.286008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b91eddf20f44ef8102038757d6d296)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:49.288465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:49.323110 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1592/3171] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:48 2022] POST /identity/v3/users => generated 296 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:49.329441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:49.331581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:49.349007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:49.352721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc0dc0-ee06-4ce0-a25e-cb2b453e64d2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:49.362975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:49.365168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:49.365722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:49.366225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:49.369950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:49.374060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:49.387382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b91eddf20f44ef8102038757d6d296'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.388146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b91eddf20f44ef8102038757d6d296'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.395598 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1593/3172] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:49.397865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:49.408574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b91eddf20f44ef8102038757d6d296'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.409337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b91eddf20f44ef8102038757d6d296'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.414689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7b82f-f8dc-47c6-b4f8-a0907a0f9d1e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:49.422604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:49.426683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:49.427205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:49.428098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] PATH_INFO: `/v3/projects/8d84be7f68134cfcaa15ab86d82d6722/users/59d41b7300ca42159d1fa04d030bb37c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:49.432493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b91eddf20f44ef8102038757d6d296'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:49.456873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d84be7f68134cfcaa15ab86d82d6722, user_id=59d41b7300ca42159d1fa04d030bb37c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:49.460297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:49.462337 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1580/3173] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:49 2022] DELETE /identity/v3/users/d5b91eddf20f44ef8102038757d6d296 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:49.482160 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1594/3174] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:49 2022] PUT /identity/v3/projects/8d84be7f68134cfcaa15ab86d82d6722/users/59d41b7300ca42159d1fa04d030bb37c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:49.486372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:49.487981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:49.600906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18215d-c9c4-4884-822b-2f1da67ac106 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:49.745703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab30ee-e889-4230-9ffa-38a5db2891d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:49.746349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab30ee-e889-4230-9ffa-38a5db2891d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:49.746625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab30ee-e889-4230-9ffa-38a5db2891d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:49.746893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab30ee-e889-4230-9ffa-38a5db2891d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:49.747550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab30ee-e889-4230-9ffa-38a5db2891d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:49.749694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab30ee-e889-4230-9ffa-38a5db2891d3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:49.772739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:49.774230 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1581/3175] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:49.802280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abf7c-33dd-4d41-a621-5ae32a696b6d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:49.811867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:49.812622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:49.812999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:49.813334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] PATH_INFO: `/v3/projects/8d84be7f68134cfcaa15ab86d82d6722/users/59d41b7300ca42159d1fa04d030bb37c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:49.816147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d84be7f68134cfcaa15ab86d82d6722, user_id=59d41b7300ca42159d1fa04d030bb37c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:49.823972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:49.845574 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1595/3176] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:49 2022] PUT /identity/v3/projects/8d84be7f68134cfcaa15ab86d82d6722/users/59d41b7300ca42159d1fa04d030bb37c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:49.855534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3606c0-3d53-47c9-9877-f14b083692c9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:49.855913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3606c0-3d53-47c9-9877-f14b083692c9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:49.856106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3606c0-3d53-47c9-9877-f14b083692c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:49.928807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5f3606c0-3d53-47c9-9877-f14b083692c9 None None] MFA Rules not processed for user `59d41b7300ca42159d1fa04d030bb37c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:49.930098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5f3606c0-3d53-47c9-9877-f14b083692c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.035018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:50.101544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1582/3177] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:50.125805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:50.127791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.291130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:50.291323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:50.291594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:50.303769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a4e1d-cc0c-4bb3-9e9b-6a1e45283f23 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:50.348346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed02d63-6487-4ca3-bd05-0cc1af3562c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:50.348979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed02d63-6487-4ca3-bd05-0cc1af3562c8 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:50.349252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed02d63-6487-4ca3-bd05-0cc1af3562c8 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:50.349558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed02d63-6487-4ca3-bd05-0cc1af3562c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:50.372363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed02d63-6487-4ca3-bd05-0cc1af3562c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:50.374086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed02d63-6487-4ca3-bd05-0cc1af3562c8 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:50.388118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] MFA Rules not processed for user `da76bbe04c22456f957be0675892359d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:50.389506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.405198 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1596/3178] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:50.524527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1583/3179] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:50.545078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb1d75-03fd-4cfd-8e62-51d9742f02ac None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:50.545367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb1d75-03fd-4cfd-8e62-51d9742f02ac None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:50.545670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb1d75-03fd-4cfd-8e62-51d9742f02ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:50.581676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:50.586860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.651361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-91eb1d75-03fd-4cfd-8e62-51d9742f02ac None None] MFA Rules not processed for user `da76bbe04c22456f957be0675892359d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:50.652661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-91eb1d75-03fd-4cfd-8e62-51d9742f02ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.703857 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1597/3180] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:50.711671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:50.711951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:50.712214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:50.776452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] MFA Rules not processed for user `da76bbe04c22456f957be0675892359d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:50.784233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.807693 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1598/3181] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:50.844628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:50.847602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.874968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338ddff-1ebc-46fa-9409-0b5700f19e21 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:50.890472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:50.891136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:50.891413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:50.894969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] PATH_INFO: `/v3/projects/5ed0bfddfd6241a2b5a8a58aed84b662` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:50.897899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed0bfddfd6241a2b5a8a58aed84b662)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:50.900463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:50.944710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72ce4b-9fbb-4b63-837c-d67bad1613e5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:50.974999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:51.000260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.000689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.001321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.027760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:51.028853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed0bfddfd6241a2b5a8a58aed84b662', 'request_id': 'req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.032948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:51.051066 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1599/3182] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:58:51.059148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed0bfddfd6241a2b5a8a58aed84b662', 'request_id': 'req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.079362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed0bfddfd6241a2b5a8a58aed84b662', 'request_id': 'req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.083091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:51.084712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:51.104229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] The token cache is being invalidate because project 5ed0bfddfd6241a2b5a8a58aed84b662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:51.106050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.109928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b47bd670-360c-4cfc-bcf1-8e4b7fb8dc81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.120138 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1584/3183] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:50 2022] DELETE /identity/v3/projects/5ed0bfddfd6241a2b5a8a58aed84b662 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:51.121786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df195a-8a37-4d6c-b500-2701bbd71af9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.122087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df195a-8a37-4d6c-b500-2701bbd71af9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.122375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df195a-8a37-4d6c-b500-2701bbd71af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.127446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa2107-6ddf-44c1-89da-3dc371c5069b service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:51.172020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:51.207548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e8df195a-8a37-4d6c-b500-2701bbd71af9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:51.208814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e8df195a-8a37-4d6c-b500-2701bbd71af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:51.296751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:51.300491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.301109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.302557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] PATH_INFO: `/v3/users/930853ac3d5c4d669b0379f1b0987aac` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.304388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=930853ac3d5c4d669b0379f1b0987aac)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:51.307460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:51.382105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1585/3184] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:51.384192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a44c3-369a-4678-bd76-1fa4e919ff2f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.386161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a44c3-369a-4678-bd76-1fa4e919ff2f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.386547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a44c3-369a-4678-bd76-1fa4e919ff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.427903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '930853ac3d5c4d669b0379f1b0987aac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.444622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ec5a44c3-369a-4678-bd76-1fa4e919ff2f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:51.449063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ec5a44c3-369a-4678-bd76-1fa4e919ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:51.460206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '930853ac3d5c4d669b0379f1b0987aac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.474263 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1586/3185] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:51.475567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a44c3-369a-4678-bd76-1fa4e919ff2f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:51.485656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '930853ac3d5c4d669b0379f1b0987aac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.486425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '930853ac3d5c4d669b0379f1b0987aac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.487256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c80d18ce-2f1d-4926-b7b1-961f2993bb93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '930853ac3d5c4d669b0379f1b0987aac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:51.502301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a44c3-369a-4678-bd76-1fa4e919ff2f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:51.517253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:51.517818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1600/3186] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:51 2022] DELETE /identity/v3/users/930853ac3d5c4d669b0379f1b0987aac => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:51.518430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.518779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.519123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.520112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:51.522089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.522491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.522862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.523566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:51.529096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:51.560723 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1587/3187] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:51.565004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:51.570264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:51.574784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:51.574933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:51.602972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12df9b-0f07-487f-b342-1a5027587bd8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:51.767443 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1601/3188] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:51.784544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:51.785782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.786055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.786318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.787472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:51.791283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:51.803123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:51.805270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:51.859152 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1588/3189] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:51 2022] POST /identity/v3/projects => generated 387 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:51.860394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:51.866119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:51.932492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2b6ab-23fd-4e75-9988-8b6d766e0104 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:52.023568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685e641-c293-4671-959a-2a41e06cd921 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:52.033075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:52.034169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:52.034455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:52.034726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:52.035403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:52.041695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:52.045041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:52.048361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:52.049158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:52.049793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:52.062586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:52.081960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:52.088288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:52.105237 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1589/3190] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:51 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:52.107858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:52.126910 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1602/3191] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:58:52.132259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d2364-b7b8-4929-80c0-666841f7932c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:52.150027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:52.150027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:52.150027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:52.150027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:52.150621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:52.150621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:52.176357 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1590/3192] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:58:52.177958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:52.188813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd90b3-d4eb-4870-a84f-2311564ae9f3 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:52.228484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccf419-fee6-4c90-bcaa-1d162bf97a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:52.229275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccf419-fee6-4c90-bcaa-1d162bf97a5e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:52.231837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccf419-fee6-4c90-bcaa-1d162bf97a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:52.232219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccf419-fee6-4c90-bcaa-1d162bf97a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:52.232918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccf419-fee6-4c90-bcaa-1d162bf97a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:52.236941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccf419-fee6-4c90-bcaa-1d162bf97a5e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:52.263939 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1603/3193] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:52.264942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:52.289144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baa750-2bb4-4e8f-872e-6985d073792c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:52.309685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:52.316241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:52.316543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:52.316832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] PATH_INFO: `/v3/projects/6fb2d4690e2e4fcb828a862219df93b0/users/632211c37c594b67a2316dab52068101/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:52.382960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb2d4690e2e4fcb828a862219df93b0, user_id=632211c37c594b67a2316dab52068101, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:52.399231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:52.424577 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1591/3194] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:52 2022] PUT /identity/v3/projects/6fb2d4690e2e4fcb828a862219df93b0/users/632211c37c594b67a2316dab52068101/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:58:52.439687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d9e5c-c4fb-4324-8010-3803574330be None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:52.439963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d9e5c-c4fb-4324-8010-3803574330be None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:52.440227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d9e5c-c4fb-4324-8010-3803574330be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:52.529454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8f1d9e5c-c4fb-4324-8010-3803574330be None None] MFA Rules not processed for user `632211c37c594b67a2316dab52068101`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:52.536894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f1d9e5c-c4fb-4324-8010-3803574330be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:52.771089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:52.771436 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1604/3195] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:52.818268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8e2f8-bfd4-49bb-8860-9bf1f44c1753 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:52.818566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8e2f8-bfd4-49bb-8860-9bf1f44c1753 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:52.818829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8e2f8-bfd4-49bb-8860-9bf1f44c1753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:52.843136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c9e8e2f8-bfd4-49bb-8860-9bf1f44c1753 None None] MFA Rules not processed for user `da76bbe04c22456f957be0675892359d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:52.844988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c9e8e2f8-bfd4-49bb-8860-9bf1f44c1753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:53.002762 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1605/3196] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:53.013612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41ae71-5866-41d3-a93e-35959b3b2923 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:53.022980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:53.025340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:53.028085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:53.028570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:53.029688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:53.037756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:53.113225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1592/3197] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:52 2022] POST /identity/v3/projects => generated 387 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:53.114409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8e2f8-bfd4-49bb-8860-9bf1f44c1753 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:53.129399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8e2f8-bfd4-49bb-8860-9bf1f44c1753 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:53.180661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:53.365394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:53.366754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:53.367267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:53.367727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:53.368545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:53.372836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:53.463207 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1606/3198] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:53 2022] POST /identity/v3/users => generated 300 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:53.466708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:53.486302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ebdbf-5fcf-4952-a459-1599bedd5cc2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:53.499267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:53.500133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:53.500572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:53.500981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:53.501828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:53.503845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:53.524728 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1593/3199] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:53.529354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:53.543194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bb1c7-fb7b-4bf3-bad7-8fced6a64053 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:53.558671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:53.560836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:53.561228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:53.562040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] PATH_INFO: `/v3/projects/365959ee2d354fb99d051ff01d187fed/users/41a2cabd597f4fbf81a7ccfb371ee30c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:53.599679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365959ee2d354fb99d051ff01d187fed, user_id=41a2cabd597f4fbf81a7ccfb371ee30c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:53.610895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:53.642575 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1607/3200] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:53 2022] PUT /identity/v3/projects/365959ee2d354fb99d051ff01d187fed/users/41a2cabd597f4fbf81a7ccfb371ee30c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:53.647034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:53.676362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e702e8-cf63-4cd5-aa5b-4e6938f6fe28 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:53.912607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:53.913294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:53.913676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:53.913935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:53.917345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:53.922571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:53.985165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:53.987437 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1594/3201] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:54.008211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c639099-1d4c-464d-b512-6d6762dfb9a4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:54.027532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93dcbd-d6bb-491e-82d9-2665e00a43f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:54.032296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93dcbd-d6bb-491e-82d9-2665e00a43f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:54.032671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93dcbd-d6bb-491e-82d9-2665e00a43f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:54.032961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93dcbd-d6bb-491e-82d9-2665e00a43f2 admin admin] PATH_INFO: `/v3/projects/365959ee2d354fb99d051ff01d187fed/users/41a2cabd597f4fbf81a7ccfb371ee30c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:54.043204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93dcbd-d6bb-491e-82d9-2665e00a43f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365959ee2d354fb99d051ff01d187fed, user_id=41a2cabd597f4fbf81a7ccfb371ee30c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:54.050778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93dcbd-d6bb-491e-82d9-2665e00a43f2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:54.093573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1608/3202] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:53 2022] PUT /identity/v3/projects/365959ee2d354fb99d051ff01d187fed/users/41a2cabd597f4fbf81a7ccfb371ee30c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:54.156846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:54.160345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:54.194982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0570c6-f52d-4836-a262-aac9b9f7d754 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:54.196186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0570c6-f52d-4836-a262-aac9b9f7d754 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:54.196801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0570c6-f52d-4836-a262-aac9b9f7d754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:54.224585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e1ddc-d4af-4ffb-baff-29aa31926ef5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:54.340838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9a0570c6-f52d-4836-a262-aac9b9f7d754 None None] MFA Rules not processed for user `41a2cabd597f4fbf81a7ccfb371ee30c`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:54.342158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9a0570c6-f52d-4836-a262-aac9b9f7d754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:54.356558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:54.446329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:54.446987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:54.447271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:54.447535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] PATH_INFO: `/v3/projects/b90e3cb8cd5e487f88be77242e41eb44` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:54.449633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90e3cb8cd5e487f88be77242e41eb44)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:54.451800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:54.509113 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1609/3203] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:54.509113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:54.509113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:54.509113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:54.555919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90e3cb8cd5e487f88be77242e41eb44', 'request_id': 'req-12b830a8-7814-46ef-9341-dec59a613457'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:54.566435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90e3cb8cd5e487f88be77242e41eb44', 'request_id': 'req-12b830a8-7814-46ef-9341-dec59a613457'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:54.570415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] MFA Rules not processed for user `87ee132f0e2f42f69b149570329b4855`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:54.571569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:54.593636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90e3cb8cd5e487f88be77242e41eb44', 'request_id': 'req-12b830a8-7814-46ef-9341-dec59a613457'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:54.611094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] The token cache is being invalidate because project b90e3cb8cd5e487f88be77242e41eb44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:54.611416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:54.614878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-12b830a8-7814-46ef-9341-dec59a613457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:54.620423 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1595/3204] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:54 2022] DELETE /identity/v3/projects/b90e3cb8cd5e487f88be77242e41eb44 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:54.625973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:54.625973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:54.626170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:54.764549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] MFA Rules not processed for user `da76bbe04c22456f957be0675892359d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:54.780498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:54.784364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:54.865302 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1610/3205] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:54.868972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:54.875954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:54.939135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1596/3206] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:54.940578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:54.947526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:55.058102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b147f4-ba35-4e4d-8c45-508dab3351cd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:55.066702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:55.068032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:55.068393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:55.069451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] PATH_INFO: `/v3/users/59d41b7300ca42159d1fa04d030bb37c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:55.071557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d41b7300ca42159d1fa04d030bb37c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:55.073969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:55.153281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:55.194630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ee45d-b580-4106-8f8e-05ab6813d8f1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:55.208396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:55.209196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:55.209602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:55.209993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:55.211492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d41b7300ca42159d1fa04d030bb37c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:55.215047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:55.249173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d41b7300ca42159d1fa04d030bb37c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:55.278131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d41b7300ca42159d1fa04d030bb37c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:55.278765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d41b7300ca42159d1fa04d030bb37c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:55.279432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d41b7300ca42159d1fa04d030bb37c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:55.302458 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1611/3207] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:54 2022] DELETE /identity/v3/users/59d41b7300ca42159d1fa04d030bb37c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:55.402984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:55.405394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:55.437346 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1597/3208] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:55.485060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:55.486446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:55.662015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:55.764865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37104405-6e15-44e8-a336-9b01d9e07553 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:55.780604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61210f-fae1-4d7a-a7d9-ce2a41d2529b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:55.781733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61210f-fae1-4d7a-a7d9-ce2a41d2529b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:55.782533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61210f-fae1-4d7a-a7d9-ce2a41d2529b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:55.783423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61210f-fae1-4d7a-a7d9-ce2a41d2529b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:55.788652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5b61210f-fae1-4d7a-a7d9-ce2a41d2529b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:55.952022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61210f-fae1-4d7a-a7d9-ce2a41d2529b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:55.959149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61210f-fae1-4d7a-a7d9-ce2a41d2529b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:55.990555 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1612/3209] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:56.170139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:56.189093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c652-1493-46a3-b21b-477f1d95d863 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:56.402849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:56.403530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:56.403803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:56.404085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] PATH_INFO: `/v3/projects/8d84be7f68134cfcaa15ab86d82d6722` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:56.410346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d84be7f68134cfcaa15ab86d82d6722)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:56.412321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:56.551367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d84be7f68134cfcaa15ab86d82d6722', 'request_id': 'req-96eab58e-7d67-4dbf-987f-22fb56ef2a89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:56.567578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d84be7f68134cfcaa15ab86d82d6722', 'request_id': 'req-96eab58e-7d67-4dbf-987f-22fb56ef2a89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:56.591966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d84be7f68134cfcaa15ab86d82d6722', 'request_id': 'req-96eab58e-7d67-4dbf-987f-22fb56ef2a89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:56.609713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] The token cache is being invalidate because project 8d84be7f68134cfcaa15ab86d82d6722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:58:56.610607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:56.628535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96eab58e-7d67-4dbf-987f-22fb56ef2a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:56.634407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1598/3210] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:58:56 2022] DELETE /identity/v3/projects/8d84be7f68134cfcaa15ab86d82d6722 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:56.639688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412614c-b5a5-4756-9085-aac3bca338f9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:56.640592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412614c-b5a5-4756-9085-aac3bca338f9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:56.640682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412614c-b5a5-4756-9085-aac3bca338f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:56.706200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e412614c-b5a5-4756-9085-aac3bca338f9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:56.710793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e412614c-b5a5-4756-9085-aac3bca338f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:56.892092 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1613/3211] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:56.899913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158b71f-981e-413e-901c-67e2fda1d6f1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:56.900217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158b71f-981e-413e-901c-67e2fda1d6f1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:56.900535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158b71f-981e-413e-901c-67e2fda1d6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:56.937298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6158b71f-981e-413e-901c-67e2fda1d6f1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:56.941483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6158b71f-981e-413e-901c-67e2fda1d6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:56.967804 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1599/3212] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:56.972060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412614c-b5a5-4756-9085-aac3bca338f9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:56.988675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412614c-b5a5-4756-9085-aac3bca338f9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:57.007250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:57.007887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:57.008170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:57.008441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:57.009107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:58:57.009586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:57.011216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:57.067881 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1614/3213] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:58:57.071916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158b71f-981e-413e-901c-67e2fda1d6f1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:57.092918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158b71f-981e-413e-901c-67e2fda1d6f1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:57.121701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:57.122343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:57.122676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:57.123029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:57.123645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:57.145607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:57.199930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:57.243788 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1600/3214] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:57 2022] POST /identity/v3/projects => generated 395 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:58:57.247716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:57.287183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c9ce2-1dad-4864-a693-ed647efdf42e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:57.568712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:57.570971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:57.571245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:57.571638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:57.572308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:57.574340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:57.675652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:57.694388 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1615/3215] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:57 2022] POST /identity/v3/users => generated 298 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:57.698419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78b2d2-28a5-436a-8278-d3461bab1fc7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:57.734461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:57.735122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:57.735405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:57.735777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:57.736574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:57.738337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:57.763941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1601/3216] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:57.768294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:57.798055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72d28-d705-48a3-a0c2-d90cdd7ad11a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:57.815982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33537788-184a-4f88-b49c-723ada28e475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:57.816689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33537788-184a-4f88-b49c-723ada28e475 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:57.816977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33537788-184a-4f88-b49c-723ada28e475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:57.817294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33537788-184a-4f88-b49c-723ada28e475 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:57.817989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33537788-184a-4f88-b49c-723ada28e475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:57.819761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33537788-184a-4f88-b49c-723ada28e475 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:57.861799 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1616/3217] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:57.866010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:57.893767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9d61d-5c56-471f-b7f4-8c978ba39e53 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:57.908474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:57.909160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:57.909700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:57.910430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] PATH_INFO: `/v3/projects/0e56ac292d5b4ca4b33eb14177bae033/users/dd724397e60b45fba695b208f1d50506/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:57.950398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e56ac292d5b4ca4b33eb14177bae033, user_id=dd724397e60b45fba695b208f1d50506, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:57.954327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:58.004627 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1602/3218] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:57 2022] PUT /identity/v3/projects/0e56ac292d5b4ca4b33eb14177bae033/users/dd724397e60b45fba695b208f1d50506/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:58.008895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fee19-fb34-47dc-b406-ff599bfa6df8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:58.009185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fee19-fb34-47dc-b406-ff599bfa6df8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:58.009512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fee19-fb34-47dc-b406-ff599bfa6df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:58.023992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:58.121932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ed9fee19-fb34-47dc-b406-ff599bfa6df8 None None] MFA Rules not processed for user `dd724397e60b45fba695b208f1d50506`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:58.126031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ed9fee19-fb34-47dc-b406-ff599bfa6df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:58.165061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:58.170866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:58.279086 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1617/3219] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:58.281048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fee19-fb34-47dc-b406-ff599bfa6df8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:58.444011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78561d4-06b6-494e-a908-1cef2489006c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:58.452725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:58.454404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:58.455638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:58.456126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] PATH_INFO: `/v3/users/632211c37c594b67a2316dab52068101` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:58.459159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] RBAC: Authorizing `identity:delete_user(user_id=632211c37c594b67a2316dab52068101)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:58.464149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:58.500064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fee19-fb34-47dc-b406-ff599bfa6df8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:58.513027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:58.514147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:58.514847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:58.515496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:58.516153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:58.518155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:58.576511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632211c37c594b67a2316dab52068101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:58.577438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632211c37c594b67a2316dab52068101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:58.592232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632211c37c594b67a2316dab52068101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:58.593388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632211c37c594b67a2316dab52068101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:58.621374 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1618/3220] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:58:58 2022] POST /identity/v3/projects => generated 395 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:58.625913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:58.631189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e860123b-edc7-4bae-bf13-cae426ea355e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632211c37c594b67a2316dab52068101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:58:58.656311 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1603/3221] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:58:58 2022] DELETE /identity/v3/users/632211c37c594b67a2316dab52068101 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:58.659993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:58.661783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bfaa5-72d9-423b-8ecc-49af46da3f7d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:58.663167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bfaa5-72d9-423b-8ecc-49af46da3f7d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:58.665965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bfaa5-72d9-423b-8ecc-49af46da3f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:58.700421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d32bfaa5-72d9-423b-8ecc-49af46da3f7d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:58.702703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d32bfaa5-72d9-423b-8ecc-49af46da3f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:58.785017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:58.815631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09ffe-a853-4fff-afd8-80fedd8d1c59 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:58.822009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:58.822971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:58.823462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:58.824944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:58.826313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:58.829554 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1604/3222] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:58.835283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:58.882581 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1619/3223] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:58:58 2022] POST /identity/v3/users => generated 304 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:58:58.886530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bfaa5-72d9-423b-8ecc-49af46da3f7d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:58.907782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bfaa5-72d9-423b-8ecc-49af46da3f7d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:58.915602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:58.916564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:58.916845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:58.917164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:58.918055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:58.919823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:58.936621 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1605/3224] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:58.938336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:58.954629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb70964-ba5c-424f-92c2-b54789ad1032 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:58.967062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:58.967764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:58.968042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:58.968314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] PATH_INFO: `/v3/projects/e1e31caed814450689517dccbafb3da5/users/2d5ea0d6d9c94fc9a71aa6dc05cb43c5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:58.976025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:58.999219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:58.999450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e31caed814450689517dccbafb3da5, user_id=2d5ea0d6d9c94fc9a71aa6dc05cb43c5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:59.001797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:59.005084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:59.041100 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1620/3225] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:58 2022] PUT /identity/v3/projects/e1e31caed814450689517dccbafb3da5/users/2d5ea0d6d9c94fc9a71aa6dc05cb43c5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:59.045263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:59.137501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058973a-2043-467e-afb7-660c030bde64 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:59.195908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbad87-8ad4-4be9-ad5e-f7b8f328827d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:59.209210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bbbb0-b590-431a-bd8c-32822247e3f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:59.210159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bbbb0-b590-431a-bd8c-32822247e3f3 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:59.211423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bbbb0-b590-431a-bd8c-32822247e3f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:59.211701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bbbb0-b590-431a-bd8c-32822247e3f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:59.211907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:59.212631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:59.212920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:59.213191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:59.214004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:59.215620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:59.223455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bbbb0-b590-431a-bd8c-32822247e3f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:59.225677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bbbb0-b590-431a-bd8c-32822247e3f3 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:59.244412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1621/3226] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:58:59.245764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:59.246888 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1606/3227] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:58:59.271656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384d4d9-9d09-43a2-9b41-9e8473f4c205 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:59.288352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07eb47-534c-4048-9041-612d0926f7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:59.293334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07eb47-534c-4048-9041-612d0926f7b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:59.293692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07eb47-534c-4048-9041-612d0926f7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:59.293976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07eb47-534c-4048-9041-612d0926f7b0 admin admin] PATH_INFO: `/v3/projects/e1e31caed814450689517dccbafb3da5/users/2d5ea0d6d9c94fc9a71aa6dc05cb43c5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:59.297363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07eb47-534c-4048-9041-612d0926f7b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e31caed814450689517dccbafb3da5, user_id=2d5ea0d6d9c94fc9a71aa6dc05cb43c5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:59.301113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07eb47-534c-4048-9041-612d0926f7b0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:59.335941 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1622/3228] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:58:59 2022] PUT /identity/v3/projects/e1e31caed814450689517dccbafb3da5/users/2d5ea0d6d9c94fc9a71aa6dc05cb43c5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:58:59.337493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:59.337734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:59.338014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:59.390750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] MFA Rules not processed for user `2d5ea0d6d9c94fc9a71aa6dc05cb43c5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:59.392082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:59.496442 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1607/3229] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:58:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:59.633067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:59.633387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:59.633750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:59.668137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:59.669631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:59.795948 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1623/3230] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 20:58:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:59.825198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:59.829997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:59.870742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:59.871637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:59.873076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:00.068288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8dfef-0be1-4df3-a27d-a47dafd8849a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:00.073950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170b203-6c7b-4f40-80a4-a80cfb35e73c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:00.074128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ee71-b997-41a7-9be3-706aa057f706 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:00.074769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ee71-b997-41a7-9be3-706aa057f706 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:00.075112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ee71-b997-41a7-9be3-706aa057f706 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:00.075528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ee71-b997-41a7-9be3-706aa057f706 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:00.086431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:00.088898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:00.089892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:00.090317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:00.100705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ee71-b997-41a7-9be3-706aa057f706 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:00.103475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ee71-b997-41a7-9be3-706aa057f706 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:00.113288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:00.119481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:00.129792 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1608/3231] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:58:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:59:00.135314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:00.157573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1624/3232] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:59 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:59:01.294833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3b81-92b1-4c28-99bb-5f987a10ee5b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:01.295285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3b81-92b1-4c28-99bb-5f987a10ee5b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:01.295285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a3b81-92b1-4c28-99bb-5f987a10ee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:01.351723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7e3a3b81-92b1-4c28-99bb-5f987a10ee5b None None] MFA Rules not processed for user `6e33e0b756e14af28a90cc5c76d346e6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:01.352984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e3a3b81-92b1-4c28-99bb-5f987a10ee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:01.469742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:01.474287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:01.493633 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1609/3233] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:01.499210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a373-72b5-407f-9f5f-84dba0ae22d4 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:01.506713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:01.507653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:01.508120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:01.508548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:01.510584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:01.621810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:01.623686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:01.636949 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1625/3234] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:01 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:59:01.666855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a3b81-92b1-4c28-99bb-5f987a10ee5b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:01.779165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a3b81-92b1-4c28-99bb-5f987a10ee5b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:01.797320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:01.798235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:01.798589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:01.798927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] PATH_INFO: `/v3/users/dd724397e60b45fba695b208f1d50506` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:01.800333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd724397e60b45fba695b208f1d50506)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:01.803205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:01.901782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd724397e60b45fba695b208f1d50506'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:01.902354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd724397e60b45fba695b208f1d50506'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:01.904744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:01.921934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:01.923598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd724397e60b45fba695b208f1d50506'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:01.924418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd724397e60b45fba695b208f1d50506'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:01.925269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:01.962483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf39e8b-79a2-4c88-96ae-4eaec06de61c service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:01.964015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a457752-fd79-415a-b056-96e67ce64b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd724397e60b45fba695b208f1d50506'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:01.976901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:01.979677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:01.980086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:01.980620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] PATH_INFO: `/v3/projects/6fb2d4690e2e4fcb828a862219df93b0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:01.983818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb2d4690e2e4fcb828a862219df93b0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:01.993236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:02.002198 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1610/3235] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:59:01 2022] DELETE /identity/v3/users/dd724397e60b45fba695b208f1d50506 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:02.006965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:02.007360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:02.007876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:02.072446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:02.076349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:02.136940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb2d4690e2e4fcb828a862219df93b0', 'request_id': 'req-2da1971f-422f-4800-9b30-e3b8ece15c59'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.168242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb2d4690e2e4fcb828a862219df93b0', 'request_id': 'req-2da1971f-422f-4800-9b30-e3b8ece15c59'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.191031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb2d4690e2e4fcb828a862219df93b0', 'request_id': 'req-2da1971f-422f-4800-9b30-e3b8ece15c59'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.219883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] The token cache is being invalidate because project 6fb2d4690e2e4fcb828a862219df93b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:02.220556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.221674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.226074 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1626/3236] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:01 2022] DELETE /identity/v3/projects/6fb2d4690e2e4fcb828a862219df93b0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:02.229189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:02.232833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:02.255501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1611/3237] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:02.277030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:02.278678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:02.384323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeda087-0d15-440f-b260-e2a663b3d98a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:02.421844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1971f-422f-4800-9b30-e3b8ece15c59 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:02.439381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:02.441806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:02.443726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:02.446249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] PATH_INFO: `/v3/users/41a2cabd597f4fbf81a7ccfb371ee30c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:02.451497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a2cabd597f4fbf81a7ccfb371ee30c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:02.460986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:02.474548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:02.475387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:02.475966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:02.476387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:02.490029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:02.492362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:02.500000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:02.508733 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1612/3238] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:02.559079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a2cabd597f4fbf81a7ccfb371ee30c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.587152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a2cabd597f4fbf81a7ccfb371ee30c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.605150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a2cabd597f4fbf81a7ccfb371ee30c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.606243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a2cabd597f4fbf81a7ccfb371ee30c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.607250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a2cabd597f4fbf81a7ccfb371ee30c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:02.633564 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1627/3239] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:59:02 2022] DELETE /identity/v3/users/41a2cabd597f4fbf81a7ccfb371ee30c => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:03.749116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:03.756907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:03.764230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:03.766308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:03.787510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca201-d4d4-4db2-abd0-d229816e5226 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:03.849528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:03.959192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:03.960708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:03.961263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:03.962922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] PATH_INFO: `/v3/projects/0e56ac292d5b4ca4b33eb14177bae033` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:03.965068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e56ac292d5b4ca4b33eb14177bae033)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:03.967828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:04.003244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3719-72b0-4ac3-9a42-cd87b8b243f8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:04.012469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:04.014337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:04.015596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:04.016115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] PATH_INFO: `/v3/projects/365959ee2d354fb99d051ff01d187fed` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:04.022995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=365959ee2d354fb99d051ff01d187fed)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:04.026884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:04.076612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e56ac292d5b4ca4b33eb14177bae033', 'request_id': 'req-dac68317-08c7-4d93-9db7-777cb581c847'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.095109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e56ac292d5b4ca4b33eb14177bae033', 'request_id': 'req-dac68317-08c7-4d93-9db7-777cb581c847'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.115354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e56ac292d5b4ca4b33eb14177bae033', 'request_id': 'req-dac68317-08c7-4d93-9db7-777cb581c847'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.141482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] The token cache is being invalidate because project 0e56ac292d5b4ca4b33eb14177bae033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:04.142269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.142962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.146665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365959ee2d354fb99d051ff01d187fed', 'request_id': 'req-ed2402b6-cf6f-43f0-9f95-f902376490bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.149876 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1613/3240] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:03 2022] DELETE /identity/v3/projects/0e56ac292d5b4ca4b33eb14177bae033 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:04.150923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:04.153035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:04.177971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365959ee2d354fb99d051ff01d187fed', 'request_id': 'req-ed2402b6-cf6f-43f0-9f95-f902376490bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.192328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365959ee2d354fb99d051ff01d187fed', 'request_id': 'req-ed2402b6-cf6f-43f0-9f95-f902376490bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.222425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] The token cache is being invalidate because project 365959ee2d354fb99d051ff01d187fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:04.222728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.223866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ed2402b6-cf6f-43f0-9f95-f902376490bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.228274 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1628/3241] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:03 2022] DELETE /identity/v3/projects/365959ee2d354fb99d051ff01d187fed => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:04.233433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd3c24-aa17-4eec-8a0e-df9237b39481 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:04.233728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd3c24-aa17-4eec-8a0e-df9237b39481 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:04.234195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd3c24-aa17-4eec-8a0e-df9237b39481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:04.266719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68317-08c7-4d93-9db7-777cb581c847 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:04.283671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-50cd3c24-aa17-4eec-8a0e-df9237b39481 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:04.285098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-50cd3c24-aa17-4eec-8a0e-df9237b39481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:04.374878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:04.452548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:04.453310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:04.454166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:04.454519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] PATH_INFO: `/v3/users/2d5ea0d6d9c94fc9a71aa6dc05cb43c5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:04.455881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5ea0d6d9c94fc9a71aa6dc05cb43c5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:04.458310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:04.462437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:04.466263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1629/3242] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:04.474165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920ee13-8488-4594-b936-56e9102d090e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:04.474630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920ee13-8488-4594-b936-56e9102d090e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:04.475008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920ee13-8488-4594-b936-56e9102d090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:04.567412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f920ee13-8488-4594-b936-56e9102d090e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:04.569817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f920ee13-8488-4594-b936-56e9102d090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:04.588898 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1630/3243] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:04.594790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920ee13-8488-4594-b936-56e9102d090e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:04.627892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5ea0d6d9c94fc9a71aa6dc05cb43c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.628714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920ee13-8488-4594-b936-56e9102d090e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:04.629294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5ea0d6d9c94fc9a71aa6dc05cb43c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.640975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:04.640975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:04.641506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:04.641853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:04.642611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:59:04.643001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:04.645412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:04.649199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5ea0d6d9c94fc9a71aa6dc05cb43c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.650186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5ea0d6d9c94fc9a71aa6dc05cb43c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.708658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0fe16d0f-a3cf-4e07-9997-ade28e9b1b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5ea0d6d9c94fc9a71aa6dc05cb43c5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:04.720181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1631/3244] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:59:04.733759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:04.738615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:04.776842 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1614/3245] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:59:04 2022] DELETE /identity/v3/users/2d5ea0d6d9c94fc9a71aa6dc05cb43c5 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:04.796725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91706-0a39-467c-969e-6033963f0614 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:04.851299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc37c-0af1-4838-8f45-c1dc39140aac None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:04.851699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc37c-0af1-4838-8f45-c1dc39140aac None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:04.852049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc37c-0af1-4838-8f45-c1dc39140aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:04.895159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d29bc37c-0af1-4838-8f45-c1dc39140aac None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:04.898820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d29bc37c-0af1-4838-8f45-c1dc39140aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:04.951982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:04.952883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:04.954284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:04.954918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:04.956099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:04.960995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:05.027132 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1632/3246] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:04 2022] POST /identity/v3/projects => generated 375 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:05.032224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:05.048301 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1615/3247] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:05.174868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28273456-a2e9-455f-8593-34ef021e05df admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:05.181261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:05.182122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.182506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.182997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.183828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:05.186816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:05.219646 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1633/3248] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:05 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:05.221236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc37c-0af1-4838-8f45-c1dc39140aac None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:05.233640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc37c-0af1-4838-8f45-c1dc39140aac None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:05.248337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:05.249217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.249324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.249661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.250381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:05.253128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:05.274754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:05.284800 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1616/3249] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:05.290492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855b33b-a6ff-45bf-9761-b1d692118393 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:05.298935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7f68f-c307-499e-b65d-35ef87bdb3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:05.299721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f68f-c307-499e-b65d-35ef87bdb3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.300873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f68f-c307-499e-b65d-35ef87bdb3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.301264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f68f-c307-499e-b65d-35ef87bdb3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.306392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f68f-c307-499e-b65d-35ef87bdb3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:05.314613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f68f-c307-499e-b65d-35ef87bdb3e4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:05.330456 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1634/3250] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:05.335043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:05.350000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bea8f2-93e7-4a92-9d07-74e800d71276 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:05.357354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:05.358381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.358490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.358748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] PATH_INFO: `/v3/projects/a0c2cdfddf5b4d35a0ca3c7554271365/users/1fff4805e6fb45a0b7c8b4dc08c4dedb/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.390121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c2cdfddf5b4d35a0ca3c7554271365, user_id=1fff4805e6fb45a0b7c8b4dc08c4dedb, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:05.393038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:05.426365 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1617/3251] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:05 2022] PUT /identity/v3/projects/a0c2cdfddf5b4d35a0ca3c7554271365/users/1fff4805e6fb45a0b7c8b4dc08c4dedb/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:05.428746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.429124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.429516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.480808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:05.487948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] MFA Rules not processed for user `1fff4805e6fb45a0b7c8b4dc08c4dedb`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:05.490609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:05.618201 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1635/3252] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:05.625882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:05.663452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:05.666718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:05.753233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:05.775555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688f2e6-987c-4d2c-822b-924171285aa1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:05.783878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:05.785949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.786419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.788186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.793682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:05.795577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c399f9-e027-4eaf-b4ff-a6b5d1ecc40d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:05.796633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:05.805657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:05.806678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.807022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.807360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] PATH_INFO: `/v3/projects/e1e31caed814450689517dccbafb3da5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.808604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e31caed814450689517dccbafb3da5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:05.810992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:05.861533 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1618/3253] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:05 2022] POST /identity/v3/projects => generated 377 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:05.866596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:05.888371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e31caed814450689517dccbafb3da5', 'request_id': 'req-07cd7605-ba92-4e0c-bd31-f2b3cd566319'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:05.907754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e31caed814450689517dccbafb3da5', 'request_id': 'req-07cd7605-ba92-4e0c-bd31-f2b3cd566319'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:05.919284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e31caed814450689517dccbafb3da5', 'request_id': 'req-07cd7605-ba92-4e0c-bd31-f2b3cd566319'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:05.934672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] The token cache is being invalidate because project e1e31caed814450689517dccbafb3da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:05.935067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:05.935845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-07cd7605-ba92-4e0c-bd31-f2b3cd566319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:05.938473 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1636/3254] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:05 2022] DELETE /identity/v3/projects/e1e31caed814450689517dccbafb3da5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:05.942833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d9cb-3e8e-40c9-8dae-a42e5f2c8ae7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:05.943628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d9cb-3e8e-40c9-8dae-a42e5f2c8ae7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:05.944054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d9cb-3e8e-40c9-8dae-a42e5f2c8ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:05.970286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecaef0-d65c-48ec-91b8-74fe297aeae0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:05.971848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ac47d9cb-3e8e-40c9-8dae-a42e5f2c8ae7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:05.973326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ac47d9cb-3e8e-40c9-8dae-a42e5f2c8ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:06.010238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:06.063247 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1637/3255] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:06.063414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.063414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.063414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.063414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.063755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.063755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.066401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab538b-7f16-48eb-a310-9d200077b8d0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.066752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab538b-7f16-48eb-a310-9d200077b8d0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.067186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab538b-7f16-48eb-a310-9d200077b8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.096955 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1619/3256] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:05 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:06.103904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.108880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:06.113069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-72ab538b-7f16-48eb-a310-9d200077b8d0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:06.114273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-72ab538b-7f16-48eb-a310-9d200077b8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:06.125535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cc925-a6b6-4b47-b8be-0be1be2e8c16 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.132327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.132946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.133218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.133894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.134549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.136170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.146442 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1638/3257] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:06.147644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab538b-7f16-48eb-a310-9d200077b8d0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.152582 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1620/3258] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:06.153929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.166793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab538b-7f16-48eb-a310-9d200077b8d0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.167895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b7985-60f4-48db-9212-26727102a261 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.173279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.174738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.175025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.175509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] PATH_INFO: `/v3/projects/c81f6785bd004f7c8d04b6d5979b4aa4/users/0cae9638f3e04011b9afe8c334303e12/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.178546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.179334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.179608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.179871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.180522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:59:06.180815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.184838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.192895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81f6785bd004f7c8d04b6d5979b4aa4, user_id=0cae9638f3e04011b9afe8c334303e12, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.207134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.218390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1639/3259] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:59:06.249294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1621/3260] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:06 2022] PUT /identity/v3/projects/c81f6785bd004f7c8d04b6d5979b4aa4/users/0cae9638f3e04011b9afe8c334303e12/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:59:06.251187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.251295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.269605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70488403-f021-4c6c-8943-59fc248e34d1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.340777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:06.363845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.364465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.364873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.365150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.365944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.368171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.403238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061fb3-7fed-4004-a1c7-a3cd71d7bb5a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.406421 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1640/3261] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:06 2022] POST /identity/v3/projects => generated 403 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:59:06.410565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.416432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.417080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.417363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.417665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.418323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.419989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.450388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1622/3262] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:59:06.455301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.472744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f603a-3c7b-4299-90ae-74261f22d7f5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.566792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129f3cc-bef7-4e53-9068-afa2a45279ee admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.587694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.590186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.591256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.592208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.593634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.601623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.604688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ca1ab-536c-458c-a2a2-9cb955f432a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.605515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ca1ab-536c-458c-a2a2-9cb955f432a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.605941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ca1ab-536c-458c-a2a2-9cb955f432a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.606544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ca1ab-536c-458c-a2a2-9cb955f432a9 admin admin] PATH_INFO: `/v3/projects/c81f6785bd004f7c8d04b6d5979b4aa4/users/0cae9638f3e04011b9afe8c334303e12/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.609004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ca1ab-536c-458c-a2a2-9cb955f432a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81f6785bd004f7c8d04b6d5979b4aa4, user_id=0cae9638f3e04011b9afe8c334303e12, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.612464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ca1ab-536c-458c-a2a2-9cb955f432a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.632267 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1623/3263] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:06 2022] PUT /identity/v3/projects/c81f6785bd004f7c8d04b6d5979b4aa4/users/0cae9638f3e04011b9afe8c334303e12/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:59:06.645223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a11ac-9469-42d1-bd87-956c04115de0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.645612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a11ac-9469-42d1-bd87-956c04115de0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.645944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a11ac-9469-42d1-bd87-956c04115de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.648477 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1641/3264] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:06 2022] POST /identity/v3/users => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:06.652468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.666299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf586d-9afd-47c7-8c93-6714a9a5cb43 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.716536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4b8a11ac-9469-42d1-bd87-956c04115de0 None None] MFA Rules not processed for user `0cae9638f3e04011b9afe8c334303e12`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:06.722581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b8a11ac-9469-42d1-bd87-956c04115de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:06.754689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:06.858018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea406b3-54c5-47d3-8412-4450ef011155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.859505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea406b3-54c5-47d3-8412-4450ef011155 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.859792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea406b3-54c5-47d3-8412-4450ef011155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.860148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea406b3-54c5-47d3-8412-4450ef011155 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.861780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea406b3-54c5-47d3-8412-4450ef011155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.863716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea406b3-54c5-47d3-8412-4450ef011155 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.904428 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1642/3265] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:06.906385 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1624/3266] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:06.907031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a8284-e81b-4ed4-9826-825e8532da9f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.907318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a8284-e81b-4ed4-9826-825e8532da9f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.907592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a8284-e81b-4ed4-9826-825e8532da9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.910505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a11ac-9469-42d1-bd87-956c04115de0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:06.933516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a11ac-9469-42d1-bd87-956c04115de0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:06.940105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dc8b9-1e63-4205-8b8f-4d00fd4ab4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:06.940777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc8b9-1e63-4205-8b8f-4d00fd4ab4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.941260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc8b9-1e63-4205-8b8f-4d00fd4ab4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.941564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc8b9-1e63-4205-8b8f-4d00fd4ab4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.942220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272dc8b9-1e63-4205-8b8f-4d00fd4ab4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:06.944751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272dc8b9-1e63-4205-8b8f-4d00fd4ab4a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:06.961607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-959a8284-e81b-4ed4-9826-825e8532da9f None None] MFA Rules not processed for user `0d48260054d84784b89a52d9b766d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:06.962243 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1625/3267] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:06.963644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-959a8284-e81b-4ed4-9826-825e8532da9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:06.963935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:06.964185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:06.964659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:06.987643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] MFA Rules not processed for user `1fff4805e6fb45a0b7c8b4dc08c4dedb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:06.989671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:07.074340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:07.075618 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1643/3268] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:07.076574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a8284-e81b-4ed4-9826-825e8532da9f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.091285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a8284-e81b-4ed4-9826-825e8532da9f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.104626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1626/3269] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:07.115826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.119506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:07.128777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2ef4a-b3bd-45bc-b9d2-bfb3338b2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:07.129737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ef4a-b3bd-45bc-b9d2-bfb3338b2a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.130249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ef4a-b3bd-45bc-b9d2-bfb3338b2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.130670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2ef4a-b3bd-45bc-b9d2-bfb3338b2a3b admin admin] PATH_INFO: `/v3/projects/6b36aa6378b248c786edfa7fa3816d86/users/130da60ae25c4da2b70be30d77e3c435/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.149623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2ef4a-b3bd-45bc-b9d2-bfb3338b2a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b36aa6378b248c786edfa7fa3816d86, user_id=130da60ae25c4da2b70be30d77e3c435, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:07.154060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2ef4a-b3bd-45bc-b9d2-bfb3338b2a3b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:07.180786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1644/3270] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:07 2022] PUT /identity/v3/projects/6b36aa6378b248c786edfa7fa3816d86/users/130da60ae25c4da2b70be30d77e3c435/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:07.193100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba187b-66ee-4421-b816-d35d38bfa052 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.193541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba187b-66ee-4421-b816-d35d38bfa052 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.193827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba187b-66ee-4421-b816-d35d38bfa052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.252676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-80ba187b-66ee-4421-b816-d35d38bfa052 None None] MFA Rules not processed for user `130da60ae25c4da2b70be30d77e3c435`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:07.258315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-80ba187b-66ee-4421-b816-d35d38bfa052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:07.319507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31337-ed07-4849-8cd2-ead9c3709e2a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.401764 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1645/3271] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:07 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:07.403478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba187b-66ee-4421-b816-d35d38bfa052 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.409673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:07.410742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.411163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.411569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.413849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:07.486933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:07.490495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:07.492924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:07.522185 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1627/3272] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:07.542360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba187b-66ee-4421-b816-d35d38bfa052 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.548430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:07.549224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.549659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.550022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.551188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:07.553632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:07.605512 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1646/3273] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:07 2022] POST /identity/v3/projects => generated 403 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:07.607109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.617876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c489609-17b6-4a5a-97ff-15f2f163d11e service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.645005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:07.734569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:07.735542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.735967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.736381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.737331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:07.739445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:07.776132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.778227 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1628/3274] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:07 2022] POST /identity/v3/users => generated 308 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:07.788368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a6184-63be-4d27-ae99-62230aaf32be admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.794813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:07.795427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.795699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.797206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.797206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:07.798902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:07.809430 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1647/3275] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:07.813625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.824860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa10a3-9ec0-4cb2-b9f1-fcf3021b8e1d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.830060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:07.830729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.831000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.831266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] PATH_INFO: `/v3/projects/2142b9af3d894f4caecea660a1be854f/users/6d16dcc6227a48dab66c34b3c00cfa9a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.846995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2142b9af3d894f4caecea660a1be854f, user_id=6d16dcc6227a48dab66c34b3c00cfa9a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:07.849669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:07.867673 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1629/3276] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:07 2022] PUT /identity/v3/projects/2142b9af3d894f4caecea660a1be854f/users/6d16dcc6227a48dab66c34b3c00cfa9a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:59:07.872251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.886155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615640b3-dc33-4add-bb7d-f084fad3299b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.957737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1223338-52e0-4143-8f49-19350b503309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:07.958686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1223338-52e0-4143-8f49-19350b503309 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:07.958969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1223338-52e0-4143-8f49-19350b503309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:07.959245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1223338-52e0-4143-8f49-19350b503309 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:07.959910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1223338-52e0-4143-8f49-19350b503309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:07.962290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1223338-52e0-4143-8f49-19350b503309 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:07.974712 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1648/3277] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:07.978834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:07.991116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5746-220c-4e71-a0b0-1bb6744a1087 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:07.999403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c695-5357-4ec3-930e-e1579a5b5038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:08.000144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c695-5357-4ec3-930e-e1579a5b5038 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:08.000470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c695-5357-4ec3-930e-e1579a5b5038 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:08.000788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c695-5357-4ec3-930e-e1579a5b5038 admin admin] PATH_INFO: `/v3/projects/2142b9af3d894f4caecea660a1be854f/users/6d16dcc6227a48dab66c34b3c00cfa9a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:08.003228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c695-5357-4ec3-930e-e1579a5b5038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2142b9af3d894f4caecea660a1be854f, user_id=6d16dcc6227a48dab66c34b3c00cfa9a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:08.005901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410c695-5357-4ec3-930e-e1579a5b5038 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:08.019118 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1630/3278] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:07 2022] PUT /identity/v3/projects/2142b9af3d894f4caecea660a1be854f/users/6d16dcc6227a48dab66c34b3c00cfa9a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:59:08.031451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46253bfc-5709-4854-90a7-30f62358ba81 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:08.031734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46253bfc-5709-4854-90a7-30f62358ba81 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:08.032011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46253bfc-5709-4854-90a7-30f62358ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:08.081548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-46253bfc-5709-4854-90a7-30f62358ba81 None None] MFA Rules not processed for user `6d16dcc6227a48dab66c34b3c00cfa9a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:08.083768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-46253bfc-5709-4854-90a7-30f62358ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:08.209616 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1649/3279] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:08.293371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:08.293978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:08.294282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:08.335480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] MFA Rules not processed for user `1fff4805e6fb45a0b7c8b4dc08c4dedb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:08.336741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:08.438704 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1631/3280] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:08.558806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46253bfc-5709-4854-90a7-30f62358ba81 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:08.629308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:08.772806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46253bfc-5709-4854-90a7-30f62358ba81 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:08.805251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:08.809886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:08.809886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:08.810852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:08.851420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:08.853600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:08.895115 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1650/3281] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:08 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:10.742651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:10.744668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:10.771114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c095721-2916-4301-9c4c-73d8f6d525c6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:10.779960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:10.781253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:10.781750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:10.782656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:10.865928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:10.867882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:10.879131 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1632/3282] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:10 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:10.917139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:11.019443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:11.020995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:11.060165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:11.078387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679965-7d95-4918-82d1-5064885df566 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:11.085811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:11.087534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.088022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.088333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] PATH_INFO: `/v3/users/130da60ae25c4da2b70be30d77e3c435` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.089865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=130da60ae25c4da2b70be30d77e3c435)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:11.092534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:11.147536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c31d0-b981-4c70-81de-871e611677b0 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:11.152589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:11.153250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.153914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.154200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] PATH_INFO: `/v3/users/87ee132f0e2f42f69b149570329b4855` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.155410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ee132f0e2f42f69b149570329b4855)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:11.158069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:11.165947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '130da60ae25c4da2b70be30d77e3c435'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.189329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '130da60ae25c4da2b70be30d77e3c435'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.203323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '130da60ae25c4da2b70be30d77e3c435'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.204655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '130da60ae25c4da2b70be30d77e3c435'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.205667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-20715e34-69f6-4658-955b-b19f28c6dc47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '130da60ae25c4da2b70be30d77e3c435'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.236481 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1651/3283] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:59:10 2022] DELETE /identity/v3/users/130da60ae25c4da2b70be30d77e3c435 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:11.241597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.242210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.242659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.245328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ee132f0e2f42f69b149570329b4855'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.246085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ee132f0e2f42f69b149570329b4855'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.259020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ee132f0e2f42f69b149570329b4855'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.260367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ee132f0e2f42f69b149570329b4855'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.266548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:11.267174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:11.283313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ee132f0e2f42f69b149570329b4855'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:11.301111 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1633/3284] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:59:11 2022] DELETE /identity/v3/users/87ee132f0e2f42f69b149570329b4855 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:59:11.305907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:11.342811 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1652/3285] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:11.368733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:11.370430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:11.550464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e864ce-95ef-4ced-a3b6-4d9245e90441 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:11.554352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:11.555293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.555403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.555645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.624292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:11.667017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:11.668693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:11.710614 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1634/3286] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:12.657512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:12.659725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:12.682230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c916931-7dba-425e-9d00-26faf2b9a06f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:12.691070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:12.695239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:12.704507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:12.705717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:12.706440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:12.707673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] PATH_INFO: `/v3/projects/6b36aa6378b248c786edfa7fa3816d86` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:12.710170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b36aa6378b248c786edfa7fa3816d86)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:12.714480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:12.717348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533045a-25a5-4cfe-b431-1a39a64e7414 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:12.732003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:12.733636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:12.733636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:12.734874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] PATH_INFO: `/v3/projects/d957f65ae29b4829b478e118ed2f89d5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:12.736424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d957f65ae29b4829b478e118ed2f89d5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:12.740534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:12.811211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b36aa6378b248c786edfa7fa3816d86', 'request_id': 'req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.827702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b36aa6378b248c786edfa7fa3816d86', 'request_id': 'req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.837642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b36aa6378b248c786edfa7fa3816d86', 'request_id': 'req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.854780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] The token cache is being invalidate because project 6b36aa6378b248c786edfa7fa3816d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:12.856141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.858343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.860375 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1653/3287] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:12 2022] DELETE /identity/v3/projects/6b36aa6378b248c786edfa7fa3816d86 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:59:12.864385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:12.866195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:12.885018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd957f65ae29b4829b478e118ed2f89d5', 'request_id': 'req-eb6e8b01-07ba-4969-b986-4a6a47ead985'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.903986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd957f65ae29b4829b478e118ed2f89d5', 'request_id': 'req-eb6e8b01-07ba-4969-b986-4a6a47ead985'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.925685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd957f65ae29b4829b478e118ed2f89d5', 'request_id': 'req-eb6e8b01-07ba-4969-b986-4a6a47ead985'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.944005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] The token cache is being invalidate because project d957f65ae29b4829b478e118ed2f89d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:12.945055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.948676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb6e8b01-07ba-4969-b986-4a6a47ead985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:12.952633 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1635/3288] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:12 2022] DELETE /identity/v3/projects/d957f65ae29b4829b478e118ed2f89d5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:59:12.959874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcf7d7-7514-405f-b104-bdce9647e3f6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:12.960180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcf7d7-7514-405f-b104-bdce9647e3f6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:12.960459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcf7d7-7514-405f-b104-bdce9647e3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.001293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-20fcf7d7-7514-405f-b104-bdce9647e3f6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:13.002583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-20fcf7d7-7514-405f-b104-bdce9647e3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:13.027321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c467-1a0f-4b10-8e2e-b70c5e4f116e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:13.150310 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1636/3289] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:13.160590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2d2eb-ecbf-4a61-afc5-4c2f92aba975 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.160898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2d2eb-ecbf-4a61-afc5-4c2f92aba975 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.161179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2d2eb-ecbf-4a61-afc5-4c2f92aba975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.212972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-21b2d2eb-ecbf-4a61-afc5-4c2f92aba975 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:13.215540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21b2d2eb-ecbf-4a61-afc5-4c2f92aba975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:13.246597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:13.247358 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1637/3290] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:13.248200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.248633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.249181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] PATH_INFO: `/v3/users/6d16dcc6227a48dab66c34b3c00cfa9a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.251178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2d2eb-ecbf-4a61-afc5-4c2f92aba975 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:13.254025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d16dcc6227a48dab66c34b3c00cfa9a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:13.257303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:13.270161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2d2eb-ecbf-4a61-afc5-4c2f92aba975 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:13.274527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:13.286100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:13.287311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.287311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.287311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.288222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:59:13.288593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:13.292850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:13.322269 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1638/3291] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:59:13.330081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:13.363334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99661-5efc-4c00-bcf8-250faadb5c4f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:13.373164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d16dcc6227a48dab66c34b3c00cfa9a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:13.376484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:13.378527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.379464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.379828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.381217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:13.387135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:13.405693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d16dcc6227a48dab66c34b3c00cfa9a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:13.424680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d16dcc6227a48dab66c34b3c00cfa9a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:13.425801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d16dcc6227a48dab66c34b3c00cfa9a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:13.426591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d16dcc6227a48dab66c34b3c00cfa9a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:13.455436 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1639/3292] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:13 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:13.458952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1654/3293] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:59:12 2022] DELETE /identity/v3/users/6d16dcc6227a48dab66c34b3c00cfa9a => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:13.460319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:13.463208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:13.540902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:13.589356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda3805-0eb8-4143-b335-a8a5616727d6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:13.664239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:13.664482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.664867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.666179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.666179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:13.669295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:13.708678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1640/3294] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:13 2022] POST /identity/v3/users => generated 287 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:59:13.712731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:13.724679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723b022-6db6-49a1-a7f5-e7deea5baa59 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:13.731537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:13.732560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.732984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.733397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.734248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:13.736335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:13.754122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1655/3295] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:59:13.757944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:13.769784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb1b67-4fea-4062-abee-0f140a2c471b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:13.774867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eafda4-130d-4796-acfd-570e529be7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:13.776200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eafda4-130d-4796-acfd-570e529be7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.777208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eafda4-130d-4796-acfd-570e529be7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.777208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eafda4-130d-4796-acfd-570e529be7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.778301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eafda4-130d-4796-acfd-570e529be7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:13.780499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eafda4-130d-4796-acfd-570e529be7b6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:13.814312 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1641/3296] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:13.819306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:13.834402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01202a9-3971-4d3d-bb2f-b0d4b0d255b4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:13.848328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:13.849298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.850718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.851435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] PATH_INFO: `/v3/projects/123af01a2abf4b24920c623a0bbe6330/users/88c9912ab49942f7943cd766ba040cc5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.893966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=123af01a2abf4b24920c623a0bbe6330, user_id=88c9912ab49942f7943cd766ba040cc5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:13.897324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:13.922718 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1656/3297] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:13 2022] PUT /identity/v3/projects/123af01a2abf4b24920c623a0bbe6330/users/88c9912ab49942f7943cd766ba040cc5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:13.928886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:13.929678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:13.930183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:13.958139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:13.960452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:14.045020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1642/3298] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:14.076540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:14.078636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:14.226632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed41822-7aac-4737-b4a8-81d3b34cd7fa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:14.233486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1e23-0d70-487c-8446-8086182ba39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:14.234847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1e23-0d70-487c-8446-8086182ba39e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:14.235382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1e23-0d70-487c-8446-8086182ba39e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:14.235853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1e23-0d70-487c-8446-8086182ba39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:14.249715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1e23-0d70-487c-8446-8086182ba39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:14.252709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1e23-0d70-487c-8446-8086182ba39e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:14.269549 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1657/3299] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:15.235148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:15.236862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:15.249830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a38e7-ac59-4ff2-80ed-fa2b690bc80c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:15.256447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:15.257220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:15.257887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:15.258397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] PATH_INFO: `/v3/projects/2142b9af3d894f4caecea660a1be854f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:15.259814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2142b9af3d894f4caecea660a1be854f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:15.262967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:15.328150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2142b9af3d894f4caecea660a1be854f', 'request_id': 'req-7509f824-1a62-4f52-95a1-f8cbb2f68c49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:15.330171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:15.339055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2142b9af3d894f4caecea660a1be854f', 'request_id': 'req-7509f824-1a62-4f52-95a1-f8cbb2f68c49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:15.353134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2142b9af3d894f4caecea660a1be854f', 'request_id': 'req-7509f824-1a62-4f52-95a1-f8cbb2f68c49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:15.377511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] The token cache is being invalidate because project 2142b9af3d894f4caecea660a1be854f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:15.377511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:15.379123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7509f824-1a62-4f52-95a1-f8cbb2f68c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:15.386273 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1643/3300] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:15 2022] DELETE /identity/v3/projects/2142b9af3d894f4caecea660a1be854f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:15.388080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00feaa0-4577-4fe0-9e5a-1d638b15eb09 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:15.388538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00feaa0-4577-4fe0-9e5a-1d638b15eb09 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:15.388706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00feaa0-4577-4fe0-9e5a-1d638b15eb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:15.417126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f00feaa0-4577-4fe0-9e5a-1d638b15eb09 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:15.418877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f00feaa0-4577-4fe0-9e5a-1d638b15eb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:15.500565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:15.529538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ab7bd-a838-411d-a06a-42dec746b366 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:15.529867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ab7bd-a838-411d-a06a-42dec746b366 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:15.530122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ab7bd-a838-411d-a06a-42dec746b366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:15.532820 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1658/3301] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:15.580897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ff1ab7bd-a838-411d-a06a-42dec746b366 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:15.582182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff1ab7bd-a838-411d-a06a-42dec746b366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:15.592160 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1644/3302] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:15.596755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00feaa0-4577-4fe0-9e5a-1d638b15eb09 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:15.619485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00feaa0-4577-4fe0-9e5a-1d638b15eb09 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:15.627217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:15.628082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:15.628082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:15.628392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:15.629894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:59:15.630206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:15.631963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:15.706932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ab7bd-a838-411d-a06a-42dec746b366 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:15.711696 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1659/3303] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:59:15.722005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ab7bd-a838-411d-a06a-42dec746b366 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:15.739929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:15.740998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:15.741522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:15.741922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:15.742784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:15.748816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:15.827951 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1645/3304] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:15 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:59:15.829178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:15.844227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc603832-dd5b-4f77-8fe5-cb833e528c80 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:15.948764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:15.949393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:15.949697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:15.950630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:15.950820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:15.953060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:15.989509 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1660/3305] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:15 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:15.994322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:16.009516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab44ab-d170-4c67-8f4f-ce871b0e7ae5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:16.015953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:16.016956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:16.017421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:16.017733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:16.018393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:16.020042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:16.034428 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1646/3306] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:59:16.035980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:16.047967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020387-d07d-4ae8-ae46-36d488180a8c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:16.053259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a88bf-8ee0-41c6-9e95-571517094187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:16.053919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a88bf-8ee0-41c6-9e95-571517094187 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:16.054195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a88bf-8ee0-41c6-9e95-571517094187 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:16.054466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a88bf-8ee0-41c6-9e95-571517094187 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:16.055288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a88bf-8ee0-41c6-9e95-571517094187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:16.057144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a88bf-8ee0-41c6-9e95-571517094187 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:16.069266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1661/3307] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:16.073560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:16.085073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01700b21-14b8-4a46-a8ab-b55552c53a3d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:16.096944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:16.098122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:16.099833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:16.100425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] PATH_INFO: `/v3/projects/f8813aaee4604882b4cd7a2a8df58077/users/da5d41f7e95d44d3898f6306bb093ed7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:16.123281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8813aaee4604882b4cd7a2a8df58077, user_id=da5d41f7e95d44d3898f6306bb093ed7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:16.126293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:16.144163 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1647/3308] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:16 2022] PUT /identity/v3/projects/f8813aaee4604882b4cd7a2a8df58077/users/da5d41f7e95d44d3898f6306bb093ed7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:16.149388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0ce0-9f6b-4980-a914-55d2306a0641 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:16.149731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0ce0-9f6b-4980-a914-55d2306a0641 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:16.150007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0ce0-9f6b-4980-a914-55d2306a0641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:16.171882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-465b0ce0-9f6b-4980-a914-55d2306a0641 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:16.171882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-465b0ce0-9f6b-4980-a914-55d2306a0641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:16.277108 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1662/3309] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:16.364719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:16.366512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:16.489831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d754-0a6f-4a89-87c0-5b2ff3541f88 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:16.495676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864d957-846c-4386-a4d6-17eadb6fcf50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:16.497114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d957-846c-4386-a4d6-17eadb6fcf50 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:16.497498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d957-846c-4386-a4d6-17eadb6fcf50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:16.497956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d957-846c-4386-a4d6-17eadb6fcf50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:16.511343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864d957-846c-4386-a4d6-17eadb6fcf50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:16.512764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864d957-846c-4386-a4d6-17eadb6fcf50 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:16.524048 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1648/3310] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:26.268862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:26.519176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=59,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:28.556057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:28.556615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:28.557242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:28.669313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] MFA Rules not processed for user `88c9912ab49942f7943cd766ba040cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:28.673540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:28.880738 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1663/3311] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:28.881846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecd02d-7a81-4c25-9aa9-3e8a3eed728d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:28.882720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecd02d-7a81-4c25-9aa9-3e8a3eed728d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:28.883152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecd02d-7a81-4c25-9aa9-3e8a3eed728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:28.927932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1fecd02d-7a81-4c25-9aa9-3e8a3eed728d None None] MFA Rules not processed for user `88c9912ab49942f7943cd766ba040cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:28.934198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1fecd02d-7a81-4c25-9aa9-3e8a3eed728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:28.978551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1649/3312] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:29.278498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:29.280980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:29.489700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8b33e-64df-4adc-8a5b-e11909e1b2b6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:29.505620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:29.505620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:29.505620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:29.505620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:29.530555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:29.532325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:29.544415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1664/3313] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:30.364155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db20042-dcce-4a74-a910-30898cd9b595 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:30.364716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db20042-dcce-4a74-a910-30898cd9b595 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:30.365116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db20042-dcce-4a74-a910-30898cd9b595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:30.430877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2db20042-dcce-4a74-a910-30898cd9b595 None None] MFA Rules not processed for user `da5d41f7e95d44d3898f6306bb093ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:30.432549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2db20042-dcce-4a74-a910-30898cd9b595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:30.560592 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1650/3314] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:30.561925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:30.575370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23161eee-8cb1-4040-a3a2-74d07e2a018f service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:30.581213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:30.581972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:30.582242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:30.582589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:30.583724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:30.585750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:30.649735 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1665/3315] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:30 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:59:30.651494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db20042-dcce-4a74-a910-30898cd9b595 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:30.698055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db20042-dcce-4a74-a910-30898cd9b595 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:30.830463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:30.831087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:30.831375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:30.831645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:30.832293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:30.834271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:30.879880 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1651/3316] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:30 2022] POST /identity/v3/users => generated 299 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:59:30.883856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:30.901880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544636fa-bcf9-46fa-a619-2b56c7de7420 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:30.913571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:30.914575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:30.914856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:30.915132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:30.916533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:30.919822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:30.956332 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1666/3317] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:59:30.961022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:30.975208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37aa2c-0e1c-4e64-a909-fa3ea95f63b8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:30.992060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:30.992764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:30.993027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:30.993299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] PATH_INFO: `/v3/projects/21cb941f09fb43ddb7d5e921bab41181/users/fdbe8a5b76e54af0b3cfc40a7ae5dcd2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:31.028047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cb941f09fb43ddb7d5e921bab41181, user_id=fdbe8a5b76e54af0b3cfc40a7ae5dcd2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:31.030666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:31.086453 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1652/3318] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:30 2022] PUT /identity/v3/projects/21cb941f09fb43ddb7d5e921bab41181/users/fdbe8a5b76e54af0b3cfc40a7ae5dcd2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:31.094525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:31.118620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415692e8-da09-4ec2-a81c-ab9e881edc5d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:31.195742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:31.267113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2ebc8-4a43-4313-8dcd-fec5552ce333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:31.268426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2ebc8-4a43-4313-8dcd-fec5552ce333 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:31.270086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2ebc8-4a43-4313-8dcd-fec5552ce333 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:31.270739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2ebc8-4a43-4313-8dcd-fec5552ce333 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:31.271767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2ebc8-4a43-4313-8dcd-fec5552ce333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:31.276139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2ebc8-4a43-4313-8dcd-fec5552ce333 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:31.289332 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1667/3319] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:31.293391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:31.326510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bc682-2e21-401d-bdd2-f6a02a4e443b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:31.349156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed354fda-05c2-45d6-82c7-aec75bb121b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:31.350006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed354fda-05c2-45d6-82c7-aec75bb121b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:31.350374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed354fda-05c2-45d6-82c7-aec75bb121b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:31.350656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed354fda-05c2-45d6-82c7-aec75bb121b4 admin admin] PATH_INFO: `/v3/projects/21cb941f09fb43ddb7d5e921bab41181/users/fdbe8a5b76e54af0b3cfc40a7ae5dcd2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:31.383567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed354fda-05c2-45d6-82c7-aec75bb121b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cb941f09fb43ddb7d5e921bab41181, user_id=fdbe8a5b76e54af0b3cfc40a7ae5dcd2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:31.386334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed354fda-05c2-45d6-82c7-aec75bb121b4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:31.454891 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1653/3320] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:31 2022] PUT /identity/v3/projects/21cb941f09fb43ddb7d5e921bab41181/users/fdbe8a5b76e54af0b3cfc40a7ae5dcd2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:33.735474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7441b43-d865-4088-975c-2e0b7b05c9d8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:33.736182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7441b43-d865-4088-975c-2e0b7b05c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:33.736182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7441b43-d865-4088-975c-2e0b7b05c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:33.788355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e7441b43-d865-4088-975c-2e0b7b05c9d8 None None] MFA Rules not processed for user `0d48260054d84784b89a52d9b766d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:33.789867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e7441b43-d865-4088-975c-2e0b7b05c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:33.932008 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1668/3321] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:41.427933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:41.562040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:41.562734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:41.562860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:41.625522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] MFA Rules not processed for user `1fff4805e6fb45a0b7c8b4dc08c4dedb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:41.627213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:41.802862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1654/3322] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:42.958217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:42.958682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:42.958760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:42.998918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] MFA Rules not processed for user `fdbe8a5b76e54af0b3cfc40a7ae5dcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:43.000356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:43.087487 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1669/3323] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:43.107288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:43.111754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:43.184071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dfe7e-fd03-4fc8-b63f-7d368a6dab69 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:43.188547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:43.189294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:43.189704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:43.190226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:43.202151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:43.204441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:43.216689 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1655/3324] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:52.153726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:52.153726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:52.287724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef4de4-bc1d-4fe8-b092-fc56b24a023e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:52.294151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:52.295340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:52.295825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:52.296381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] PATH_INFO: `/v3/users/6e33e0b756e14af28a90cc5c76d346e6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:52.298546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e33e0b756e14af28a90cc5c76d346e6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:52.301086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:52.317637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:52.370326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e33e0b756e14af28a90cc5c76d346e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:52.392568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e33e0b756e14af28a90cc5c76d346e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:52.407225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e33e0b756e14af28a90cc5c76d346e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:52.408118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e33e0b756e14af28a90cc5c76d346e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:52.408976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c9898db-9ec8-4309-83de-1c1e91d0679c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e33e0b756e14af28a90cc5c76d346e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:52.429895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1670/3325] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 20:59:52 2022] DELETE /identity/v3/users/6e33e0b756e14af28a90cc5c76d346e6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:52.753554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:52.769540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e66789-c5e6-4040-b19e-69344caf78d0 service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:52.923905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:52.925046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:52.925794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:52.926502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] PATH_INFO: `/v3/projects/2a4117ca05454573bbd93fbca50431d2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:52.929107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4117ca05454573bbd93fbca50431d2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:52.932932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:53.035981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4117ca05454573bbd93fbca50431d2', 'request_id': 'req-f6c71dc5-8151-4e81-a595-53e4a0590cf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:53.037537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:53.051827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4117ca05454573bbd93fbca50431d2', 'request_id': 'req-f6c71dc5-8151-4e81-a595-53e4a0590cf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:53.065399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4117ca05454573bbd93fbca50431d2', 'request_id': 'req-f6c71dc5-8151-4e81-a595-53e4a0590cf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:53.083973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] The token cache is being invalidate because project 2a4117ca05454573bbd93fbca50431d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 20:59:53.084267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:53.085200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6c71dc5-8151-4e81-a595-53e4a0590cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 20:59:53.087913 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1656/3326] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 20:59:52 2022] DELETE /identity/v3/projects/2a4117ca05454573bbd93fbca50431d2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:53.093105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd55667-1bb7-4b05-a5d5-09091327b97a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.093327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd55667-1bb7-4b05-a5d5-09091327b97a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.093639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd55667-1bb7-4b05-a5d5-09091327b97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.124793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bdd55667-1bb7-4b05-a5d5-09091327b97a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:53.126398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bdd55667-1bb7-4b05-a5d5-09091327b97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:53.214967 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1671/3327] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:53.221078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb14b8c-6bf2-4fce-b854-b49a6dc28d12 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.221359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb14b8c-6bf2-4fce-b854-b49a6dc28d12 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.221849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb14b8c-6bf2-4fce-b854-b49a6dc28d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.265335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6eb14b8c-6bf2-4fce-b854-b49a6dc28d12 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:53.266620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6eb14b8c-6bf2-4fce-b854-b49a6dc28d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:53.274747 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1657/3328] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:53.279480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd55667-1bb7-4b05-a5d5-09091327b97a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:53.291664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd55667-1bb7-4b05-a5d5-09091327b97a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:53.296897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:53.297625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.297967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.298427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.299302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:59:53.299668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:53.302026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:53.328296 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1672/3329] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 20:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:59:53.332761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb14b8c-6bf2-4fce-b854-b49a6dc28d12 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:53.346573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb14b8c-6bf2-4fce-b854-b49a6dc28d12 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:53.351597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:53.352222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.352498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.352769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.353721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:53.355800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:53.405040 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1658/3330] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 20:59:53 2022] POST /identity/v3/projects => generated 397 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:53.406780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:53.427125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456471-1237-4dfa-9dd3-4e2c00ef67d9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:53.520337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:53.521397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.522039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.522455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.523225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:53.525596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:53.555453 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1673/3331] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 20:59:53 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:59:53.556707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:53.573638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10f6f7-8fcd-4d4a-9927-62088ad66ed8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:53.582882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:53.583956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.584329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.584613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.585487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:53.588296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:53.606418 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1659/3332] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:59:53.610170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:53.618973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:53.623122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b203b-ea3b-43a5-9fa7-56578c6fc887 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:53.632606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcfa87-aa92-4c1b-a832-8ac4e95a23e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:53.634273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcfa87-aa92-4c1b-a832-8ac4e95a23e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.634845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcfa87-aa92-4c1b-a832-8ac4e95a23e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.635351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcfa87-aa92-4c1b-a832-8ac4e95a23e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.636218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcfa87-aa92-4c1b-a832-8ac4e95a23e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:53.639726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcfa87-aa92-4c1b-a832-8ac4e95a23e4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:53.658844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1674/3333] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 20:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:59:53.663712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:53.704072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390e91a-c4d0-421c-81ab-97581b3f97af admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:53.712577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:53.713258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.715885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.715885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] PATH_INFO: `/v3/projects/004f77fba4b14080bca4ae36c857add3/users/22306a93c9404db393b92a267d6258d4/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.733994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004f77fba4b14080bca4ae36c857add3, user_id=22306a93c9404db393b92a267d6258d4, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:53.738337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:53.763203 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1660/3334] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 20:59:53 2022] PUT /identity/v3/projects/004f77fba4b14080bca4ae36c857add3/users/22306a93c9404db393b92a267d6258d4/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:59:53.770830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2feb40-c67b-4035-a1d5-660e0b995be3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:53.771118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2feb40-c67b-4035-a1d5-660e0b995be3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:53.771394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2feb40-c67b-4035-a1d5-660e0b995be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:53.816250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9f2feb40-c67b-4035-a1d5-660e0b995be3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:53.818127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9f2feb40-c67b-4035-a1d5-660e0b995be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:53.933849 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1675/3335] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:53.945110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:53.947115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:54.047832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed17185-9901-4282-a199-8341dc2ac699 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:54.051639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56865005-5230-409b-8085-6385685ce373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:54.052254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56865005-5230-409b-8085-6385685ce373 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:54.052537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56865005-5230-409b-8085-6385685ce373 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:54.052807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56865005-5230-409b-8085-6385685ce373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:54.062874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56865005-5230-409b-8085-6385685ce373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:54.064313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56865005-5230-409b-8085-6385685ce373 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:54.074802 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1661/3336] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:56.212892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f507e-fa11-490a-8d26-52a738b445c0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:56.214050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f507e-fa11-490a-8d26-52a738b445c0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:56.214050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f507e-fa11-490a-8d26-52a738b445c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:56.238394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5c4f507e-fa11-490a-8d26-52a738b445c0 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:56.239934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5c4f507e-fa11-490a-8d26-52a738b445c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:56.334942 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1676/3337] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 20:59:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:56.366948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56865005-5230-409b-8085-6385685ce373 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:56.369212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-56865005-5230-409b-8085-6385685ce373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:56.546414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56865005-5230-409b-8085-6385685ce373 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:56.552565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477157f-e0a9-428e-a4fb-072927e03fe1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:56.557356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477157f-e0a9-428e-a4fb-072927e03fe1 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:56.557356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477157f-e0a9-428e-a4fb-072927e03fe1 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:56.557356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477157f-e0a9-428e-a4fb-072927e03fe1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:56.570170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477157f-e0a9-428e-a4fb-072927e03fe1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:56.572349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477157f-e0a9-428e-a4fb-072927e03fe1 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:56.585400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1662/3338] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 20:59:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:59:57.220762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1ddd-87bb-4dc3-ba5e-877cf427f834 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:57.221249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1ddd-87bb-4dc3-ba5e-877cf427f834 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:57.221350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1ddd-87bb-4dc3-ba5e-877cf427f834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:57.275108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a7ac1ddd-87bb-4dc3-ba5e-877cf427f834 None None] MFA Rules not processed for user `0d48260054d84784b89a52d9b766d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:57.283302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7ac1ddd-87bb-4dc3-ba5e-877cf427f834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:57.423064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1677/3339] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 20:59:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:00:01.917940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:01.917940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:01.954655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:01.971324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:02.026240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] MFA Rules not processed for user `22306a93c9404db393b92a267d6258d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:02.027942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:02.178236 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1663/3340] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:02.179748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:02.180261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:02.180261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:02.211491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] MFA Rules not processed for user `22306a93c9404db393b92a267d6258d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:02.213533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:02.223495 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1678/3341] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:02.650779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:02.652574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:02.833867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f126a06-ecb3-4130-b8dd-4735455be25d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:02.841489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773dd0fc-31d5-43d3-b3a3-fbac2a0199ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:02.842244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd0fc-31d5-43d3-b3a3-fbac2a0199ef service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:02.842579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd0fc-31d5-43d3-b3a3-fbac2a0199ef service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:02.842909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd0fc-31d5-43d3-b3a3-fbac2a0199ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:02.862654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773dd0fc-31d5-43d3-b3a3-fbac2a0199ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:02.864324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773dd0fc-31d5-43d3-b3a3-fbac2a0199ef service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:02.889925 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1664/3342] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:02 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:07.215035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:07.218029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:07.239051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61118556-aff1-470e-85a0-63a96c0a8221 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:07.252774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:07.253882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:07.254366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:07.254868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:07.257892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:07.340232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:07.415108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:07.420802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:07.444954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1679/3343] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:10.666226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608d6d3-dd88-4b75-bc70-d07d07560db4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:10.667204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608d6d3-dd88-4b75-bc70-d07d07560db4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:10.667668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608d6d3-dd88-4b75-bc70-d07d07560db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:10.706660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2608d6d3-dd88-4b75-bc70-d07d07560db4 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:10.715265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2608d6d3-dd88-4b75-bc70-d07d07560db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:10.742136 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1665/3344] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:00:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:10.758774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:10.776547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f56486-2c59-4faf-8989-3dbf03398768 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:10.782840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850978c-3685-456b-beb5-00b609294242 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:10.784620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850978c-3685-456b-beb5-00b609294242 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:10.785114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850978c-3685-456b-beb5-00b609294242 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:10.785636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850978c-3685-456b-beb5-00b609294242 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:10.797296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850978c-3685-456b-beb5-00b609294242 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:10.799140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850978c-3685-456b-beb5-00b609294242 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:10.809950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1680/3345] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:00:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:00:17.748704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608d6d3-dd88-4b75-bc70-d07d07560db4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:17.768511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608d6d3-dd88-4b75-bc70-d07d07560db4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:17.785566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:17.786220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:17.786497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:17.786766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:17.788557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:17.900534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:17.902338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:17.924513 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1666/3346] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:20.805546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=19 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:20.896409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850978c-3685-456b-beb5-00b609294242 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:20.898209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4850978c-3685-456b-beb5-00b609294242 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:20.964371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850978c-3685-456b-beb5-00b609294242 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:20.988563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:20.989729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:20.992359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:20.994173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] PATH_INFO: `/v3/users/88c9912ab49942f7943cd766ba040cc5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:20.996324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c9912ab49942f7943cd766ba040cc5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:21.001819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:21.183704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c9912ab49942f7943cd766ba040cc5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:21.240906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c9912ab49942f7943cd766ba040cc5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:21.261202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c9912ab49942f7943cd766ba040cc5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:21.261859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c9912ab49942f7943cd766ba040cc5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:21.262472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b05b99f7-8d3e-4a6c-b92c-d57f998e8bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c9912ab49942f7943cd766ba040cc5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:21.296545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1681/3347] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:00:20 2022] DELETE /identity/v3/users/88c9912ab49942f7943cd766ba040cc5 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:21.834416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:21.855763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0325a2-0b13-46f7-9002-26ff32c250ad service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:21.897726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:21.982011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:21.983034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:21.983473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:21.983892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] PATH_INFO: `/v3/projects/123af01a2abf4b24920c623a0bbe6330` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:21.986060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] RBAC: Authorizing `identity:delete_project(project_id=123af01a2abf4b24920c623a0bbe6330)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:21.990078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:22.102756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123af01a2abf4b24920c623a0bbe6330', 'request_id': 'req-7a8a9f8f-1760-4595-8c71-2466fdceccce'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:22.114149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123af01a2abf4b24920c623a0bbe6330', 'request_id': 'req-7a8a9f8f-1760-4595-8c71-2466fdceccce'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:22.130940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123af01a2abf4b24920c623a0bbe6330', 'request_id': 'req-7a8a9f8f-1760-4595-8c71-2466fdceccce'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:22.148171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] The token cache is being invalidate because project 123af01a2abf4b24920c623a0bbe6330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:00:22.148758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:22.149692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7a8a9f8f-1760-4595-8c71-2466fdceccce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:22.155358 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1667/3348] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:00:21 2022] DELETE /identity/v3/projects/123af01a2abf4b24920c623a0bbe6330 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:22.156746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437a250-b85b-40f3-bda4-b2fa8190beba None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.157243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437a250-b85b-40f3-bda4-b2fa8190beba None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.157529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437a250-b85b-40f3-bda4-b2fa8190beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.189973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0437a250-b85b-40f3-bda4-b2fa8190beba None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:22.194729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0437a250-b85b-40f3-bda4-b2fa8190beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:22.218698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c210166-8727-4984-aedf-f782587b4d7a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.218986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c210166-8727-4984-aedf-f782587b4d7a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.219262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c210166-8727-4984-aedf-f782587b4d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.251651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6c210166-8727-4984-aedf-f782587b4d7a None None] MFA Rules not processed for user `da5d41f7e95d44d3898f6306bb093ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:22.254245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6c210166-8727-4984-aedf-f782587b4d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:22.349097 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1682/3349] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:00:22.359454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b5be-ceac-4c5c-96c4-29adf1522f2b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.360006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b5be-ceac-4c5c-96c4-29adf1522f2b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.360444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b5be-ceac-4c5c-96c4-29adf1522f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.388176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1668/3350] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:22.398945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ad41b5be-ceac-4c5c-96c4-29adf1522f2b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:22.400237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ad41b5be-ceac-4c5c-96c4-29adf1522f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:22.416559 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1683/3351] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:22.418547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c210166-8727-4984-aedf-f782587b4d7a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:22.431340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c210166-8727-4984-aedf-f782587b4d7a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:22.436147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:22.436766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.437043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.437313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.438005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:00:22.438306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:22.440189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:22.461265 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1669/3352] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:00:22.462486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41b5be-ceac-4c5c-96c4-29adf1522f2b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:22.473744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41b5be-ceac-4c5c-96c4-29adf1522f2b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:22.481177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:22.481840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.482149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.482419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.483065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:22.486618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:22.530196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1684/3353] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:00:22 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:00:22.535691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:22.546903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47f7f1-7da1-4f11-8437-7325fac5022d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:22.603019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:22.675743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:22.676558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.676838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.677106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.677772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:22.679706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:22.733967 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1670/3354] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:00:22 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:00:22.737885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:22.760364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a6605-b32c-45ca-a81d-96f6f5b39a72 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:22.783502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:22.784170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.784449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.786694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.787702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:22.790088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:22.807652 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1685/3355] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:00:22.811445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:22.829816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cc6bc-87f9-4473-99df-2f87d6aab642 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:22.849053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0afcb-deb4-4c33-870b-d78df128b4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:22.850103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0afcb-deb4-4c33-870b-d78df128b4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.850103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0afcb-deb4-4c33-870b-d78df128b4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.850288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0afcb-deb4-4c33-870b-d78df128b4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.850870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0afcb-deb4-4c33-870b-d78df128b4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:22.852479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0afcb-deb4-4c33-870b-d78df128b4f6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:22.871542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:22.878688 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1671/3356] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:00:22.883514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8396-edb2-40cf-a796-00eab3b9882b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:22.889932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:22.890735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.891081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.891517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] PATH_INFO: `/v3/projects/36ce01157eec446dbcc93b2948b4be90/users/33fa2d463884464fbe90666c98da0fd7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:22.924520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ce01157eec446dbcc93b2948b4be90, user_id=33fa2d463884464fbe90666c98da0fd7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:22.931627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:22.951413 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1686/3357] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:00:22 2022] PUT /identity/v3/projects/36ce01157eec446dbcc93b2948b4be90/users/33fa2d463884464fbe90666c98da0fd7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:22.957786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:22.958139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:22.958479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:23.027195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:23.028723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:23.185618 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1672/3358] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:23.193105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:23.195836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:23.213077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:23.279244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb9754-bed3-43b2-bcee-fabd2d8d642d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:23.283868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:23.284639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:23.284810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:23.285082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:23.299034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:23.300959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:23.318104 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1687/3359] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:26.129222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:26.130876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:26.368638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428461c-e809-4ada-9eee-e669a8832681 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:26.386248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed5cb4-0410-43eb-b5ea-9d6a6269f67a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:26.386898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed5cb4-0410-43eb-b5ea-9d6a6269f67a service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:26.387182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed5cb4-0410-43eb-b5ea-9d6a6269f67a service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:26.387816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed5cb4-0410-43eb-b5ea-9d6a6269f67a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:26.390018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1ed5cb4-0410-43eb-b5ea-9d6a6269f67a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:26.501716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:26.541680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed5cb4-0410-43eb-b5ea-9d6a6269f67a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:26.543506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed5cb4-0410-43eb-b5ea-9d6a6269f67a service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:26.581890 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1673/3360] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:30.312980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:30.315245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:30.334610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce1d3a-593b-4720-8fbd-53beba6c2297 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:30.362941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:30.363993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:30.364478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:30.364954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:30.369929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:30.413599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:30.415243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:30.440748 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1688/3361] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:31.593884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bccf5-75e7-40ae-95d0-eac0a6a4f28e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:31.594905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bccf5-75e7-40ae-95d0-eac0a6a4f28e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:31.595369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bccf5-75e7-40ae-95d0-eac0a6a4f28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:31.639567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-109bccf5-75e7-40ae-95d0-eac0a6a4f28e None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:31.641825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-109bccf5-75e7-40ae-95d0-eac0a6a4f28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:31.772710 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1674/3362] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:31.803252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:31.811528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:32.009275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234a499-3124-4f1d-8753-1509b365f156 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:32.030306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba6ec4-9091-4f9f-a247-020d3d281f51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:32.031229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6ec4-9091-4f9f-a247-020d3d281f51 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:32.031660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6ec4-9091-4f9f-a247-020d3d281f51 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:32.032094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6ec4-9091-4f9f-a247-020d3d281f51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:32.067586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba6ec4-9091-4f9f-a247-020d3d281f51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:32.069501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba6ec4-9091-4f9f-a247-020d3d281f51 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:32.093756 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1689/3363] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:00:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:00:33.537339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bccf5-75e7-40ae-95d0-eac0a6a4f28e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:33.552602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bccf5-75e7-40ae-95d0-eac0a6a4f28e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:33.566992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10759ee2-9872-4025-81bb-5085c75c98b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:33.567748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759ee2-9872-4025-81bb-5085c75c98b2 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:33.568042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759ee2-9872-4025-81bb-5085c75c98b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:33.568549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759ee2-9872-4025-81bb-5085c75c98b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:33.570969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-10759ee2-9872-4025-81bb-5085c75c98b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:33.618310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10759ee2-9872-4025-81bb-5085c75c98b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:33.619896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10759ee2-9872-4025-81bb-5085c75c98b2 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:33.654657 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1675/3364] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:36.166842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec2539-5b56-4503-a8c4-9c7f8ee33476 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:36.167350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec2539-5b56-4503-a8c4-9c7f8ee33476 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:36.167350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec2539-5b56-4503-a8c4-9c7f8ee33476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:36.245218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-93ec2539-5b56-4503-a8c4-9c7f8ee33476 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:36.246487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-93ec2539-5b56-4503-a8c4-9c7f8ee33476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:36.446279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:36.446788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:36.446788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:36.451061 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1690/3365] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:36.483970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:36.485359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:36.506784 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1676/3366] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:37.294590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec2539-5b56-4503-a8c4-9c7f8ee33476 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:37.316679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec2539-5b56-4503-a8c4-9c7f8ee33476 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:37.334380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc270e-5963-4242-8af6-aa46d4b728aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:37.335020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc270e-5963-4242-8af6-aa46d4b728aa service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:37.335301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc270e-5963-4242-8af6-aa46d4b728aa service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:37.335566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc270e-5963-4242-8af6-aa46d4b728aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:37.353340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc270e-5963-4242-8af6-aa46d4b728aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:37.355422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc270e-5963-4242-8af6-aa46d4b728aa service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:37.379189 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1691/3367] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:38.506613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:38.508182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:38.697995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119706d-70d1-4ed4-9143-9356836ee408 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:38.705671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:38.706648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:38.707075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:38.707538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:38.710576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:38.736461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:38.738532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:38.773911 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1677/3368] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:39.920451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06640-98ad-4e13-a230-a38dcf0dacf3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:39.920949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06640-98ad-4e13-a230-a38dcf0dacf3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:39.921185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06640-98ad-4e13-a230-a38dcf0dacf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:39.945985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:40.052291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-88c06640-98ad-4e13-a230-a38dcf0dacf3 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:40.054917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-88c06640-98ad-4e13-a230-a38dcf0dacf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:40.108528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:40.119780 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1692/3369] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:40.142709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29b802-1ab1-41a2-bc81-bb238e299fad service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:40.155878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:40.156495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:40.157292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:40.157620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:40.169440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:40.171890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:40.187101 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1678/3370] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:00:43.946952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06640-98ad-4e13-a230-a38dcf0dacf3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:43.980103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06640-98ad-4e13-a230-a38dcf0dacf3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:44.011690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d15004-0c5d-4d2d-a1cc-dd765e9c722d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:44.013749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d15004-0c5d-4d2d-a1cc-dd765e9c722d service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.015495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d15004-0c5d-4d2d-a1cc-dd765e9c722d service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.015997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d15004-0c5d-4d2d-a1cc-dd765e9c722d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.021268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-91d15004-0c5d-4d2d-a1cc-dd765e9c722d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:44.065245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d15004-0c5d-4d2d-a1cc-dd765e9c722d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:44.070856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d15004-0c5d-4d2d-a1cc-dd765e9c722d service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:44.170509 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1693/3371] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:45.315807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:45.329736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04b737-f888-4583-a4a7-bd3742ba2200 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:45.359245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:45.360135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:45.360645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:45.361413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:45.362163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:45.364993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:45.365389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:45.371186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.392938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:45.394785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:45.406824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:45.414310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.420263 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1679/3372] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:45.629281 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1694/3373] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:45.861966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:45.871802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b6e35-efcf-4c9f-b0e4-e53952777ac1 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:45.887272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:45.890092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:45.890605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:45.891088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:45.894924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.922606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:45.924210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:45.934317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1680/3374] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:46.293882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:46.295423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:46.321679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa56c01-273b-44e3-9dd8-6091c5919de1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:46.326949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:46.328393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:46.328897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:46.329274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] PATH_INFO: `/v3/users/da76bbe04c22456f957be0675892359d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:46.330931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da76bbe04c22456f957be0675892359d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:46.336809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:46.413308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da76bbe04c22456f957be0675892359d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.438084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da76bbe04c22456f957be0675892359d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.454292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da76bbe04c22456f957be0675892359d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.455027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da76bbe04c22456f957be0675892359d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.459615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47e2a88e-f1f2-428b-b5e7-dfbc00785979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da76bbe04c22456f957be0675892359d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.479931 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1695/3375] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:00:46 2022] DELETE /identity/v3/users/da76bbe04c22456f957be0675892359d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:47.628464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:47.646122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7410a02-8c0f-4730-a301-bf8abf322a64 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:47.665503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:47.799139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:47.800046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:47.800506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:47.800995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] PATH_INFO: `/v3/projects/9b45a150f2db4c2bb4871cd1d1cab898` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:47.802438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b45a150f2db4c2bb4871cd1d1cab898)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:47.804819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:47.905593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b45a150f2db4c2bb4871cd1d1cab898', 'request_id': 'req-fe650dfc-30ed-492f-92de-142eb7eab3ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:47.959447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b45a150f2db4c2bb4871cd1d1cab898', 'request_id': 'req-fe650dfc-30ed-492f-92de-142eb7eab3ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:48.007990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b45a150f2db4c2bb4871cd1d1cab898', 'request_id': 'req-fe650dfc-30ed-492f-92de-142eb7eab3ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:48.059549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] The token cache is being invalidate because project 9b45a150f2db4c2bb4871cd1d1cab898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:00:48.063385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:48.066141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:48.075574 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1681/3376] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:00:47 2022] DELETE /identity/v3/projects/9b45a150f2db4c2bb4871cd1d1cab898 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:48.076425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12011ad7-bb05-42e1-98a8-5c3d0bd139ea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:48.077506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12011ad7-bb05-42e1-98a8-5c3d0bd139ea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:48.077990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12011ad7-bb05-42e1-98a8-5c3d0bd139ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:48.134522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-12011ad7-bb05-42e1-98a8-5c3d0bd139ea None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:48.142197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-12011ad7-bb05-42e1-98a8-5c3d0bd139ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:48.146447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:48.147981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:48.222792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:48.302775 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1696/3377] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:48.304374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ff90-811e-4b28-bfe8-779ca8bf522e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:48.304763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ff90-811e-4b28-bfe8-779ca8bf522e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:48.305508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ff90-811e-4b28-bfe8-779ca8bf522e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:48.336023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0868ff90-811e-4b28-bfe8-779ca8bf522e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:48.337872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0868ff90-811e-4b28-bfe8-779ca8bf522e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:48.361849 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1697/3378] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:48.364778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ff90-811e-4b28-bfe8-779ca8bf522e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:48.376667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650dfc-30ed-492f-92de-142eb7eab3ae admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:48.389559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:48.390202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:48.390834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:48.391121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:48.392985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:48.396011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ff90-811e-4b28-bfe8-779ca8bf522e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:48.407268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:48.410338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:48.410461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:48.410822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:48.411582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:00:48.415973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:48.417919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:48.418381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:48.420007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:48.448101 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1698/3379] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:00:48.454481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:48.468512 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1682/3380] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:00:48.502615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4854c2f-f0ab-4180-9495-b139c8a15247 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:48.518417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:48.519023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:48.519323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:48.519654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:48.520309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:48.523021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:48.614033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:48.615184 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1699/3381] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:00:48 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:00:48.630109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7e827-da12-4862-ab6b-4e8e7a937833 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:48.774627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:48.847687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:48.850104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:48.850286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:48.850491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:48.852439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:48.861221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:48.943187 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1683/3382] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:00:48 2022] POST /identity/v3/users => generated 290 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:00:48.947678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:48.964203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06fb9-da21-48eb-8342-62facbd58271 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:48.978420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:48.979794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:48.981299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:48.982887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:48.984190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:48.989674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:49.018622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1700/3383] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:00:49.023699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:49.037294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678de2-848c-4816-a8bd-193030449c8c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:49.045579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f851-b6f5-4de1-a770-c08b7b2137bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:49.046648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f851-b6f5-4de1-a770-c08b7b2137bc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:49.047070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f851-b6f5-4de1-a770-c08b7b2137bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:49.047626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f851-b6f5-4de1-a770-c08b7b2137bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:49.049035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562f851-b6f5-4de1-a770-c08b7b2137bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:49.050991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562f851-b6f5-4de1-a770-c08b7b2137bc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:49.078675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:49.080253 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1684/3384] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:00:49.098410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3073a5-516a-4130-93d0-dd106cc7a29e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:49.106632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:49.107886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:49.108394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:49.108907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] PATH_INFO: `/v3/projects/77e4ba38e25a4143adfe864a8408c524/users/d1bc1e85c16e49c695f073f2a7b40a52/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:49.134059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e4ba38e25a4143adfe864a8408c524, user_id=d1bc1e85c16e49c695f073f2a7b40a52, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:49.140173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:49.168407 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1701/3385] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:00:49 2022] PUT /identity/v3/projects/77e4ba38e25a4143adfe864a8408c524/users/d1bc1e85c16e49c695f073f2a7b40a52/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:49.170515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c17bd-bb0b-4137-8b6b-1cb781ede668 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:49.170809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c17bd-bb0b-4137-8b6b-1cb781ede668 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:49.171099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c17bd-bb0b-4137-8b6b-1cb781ede668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:49.199902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dc1c17bd-bb0b-4137-8b6b-1cb781ede668 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:49.202831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc1c17bd-bb0b-4137-8b6b-1cb781ede668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:49.370897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1685/3386] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:49.391919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:49.394009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:49.511681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f642c-2fb8-49b4-97df-21340515b5d1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:49.517579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0eb9fb-1918-4113-8e13-83b518b8f3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:49.518399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0eb9fb-1918-4113-8e13-83b518b8f3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:49.519214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0eb9fb-1918-4113-8e13-83b518b8f3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:49.519214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0eb9fb-1918-4113-8e13-83b518b8f3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:49.530856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0eb9fb-1918-4113-8e13-83b518b8f3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:49.532403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0eb9fb-1918-4113-8e13-83b518b8f3f9 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:49.556259 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1702/3387] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:52.863581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f0d7c-7947-49b4-aa21-fa7dc41e5850 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:52.864040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f0d7c-7947-49b4-aa21-fa7dc41e5850 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:52.864118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f0d7c-7947-49b4-aa21-fa7dc41e5850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:52.905695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-283f0d7c-7947-49b4-aa21-fa7dc41e5850 None None] MFA Rules not processed for user `22306a93c9404db393b92a267d6258d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:52.907128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-283f0d7c-7947-49b4-aa21-fa7dc41e5850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:53.008303 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1686/3388] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:58.748392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542febe-baec-41f5-bb55-a287f4139538 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:58.748959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542febe-baec-41f5-bb55-a287f4139538 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:58.748959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542febe-baec-41f5-bb55-a287f4139538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:58.788426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3542febe-baec-41f5-bb55-a287f4139538 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:58.789743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3542febe-baec-41f5-bb55-a287f4139538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:58.854019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:58.864135 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1703/3389] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:00:59.629735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:59.630307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:59.630405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:59.662410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] MFA Rules not processed for user `d1bc1e85c16e49c695f073f2a7b40a52`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:59.663949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:59.721128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:59.749701 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1687/3390] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:59.759807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:59.760762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:59.761431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:59.782876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] MFA Rules not processed for user `d1bc1e85c16e49c695f073f2a7b40a52`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:59.784273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:59.792716 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1704/3391] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:00:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:00.716558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:00.718349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:00.936819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857df1b-2409-4c01-8cc6-56734be30384 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:00.952070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:00.952875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:00.953352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:00.953701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:00.965016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:00.967926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:00.990073 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1688/3392] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:01:09.776520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:10.975022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=40 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:11.049839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:11.052540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:11.076585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ccc7f-d3a3-4711-8e50-7aba305977b1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:11.083762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:11.084756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:11.085241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:11.085770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] PATH_INFO: `/v3/users/0d48260054d84784b89a52d9b766d7c1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:11.087317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d48260054d84784b89a52d9b766d7c1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:11.090295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:11.212022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d48260054d84784b89a52d9b766d7c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:11.215801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:11.218122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:11.243444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9c1da-feb9-4492-9a69-a7333acee4d3 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:11.256743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:11.257659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:11.258788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:11.259352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:11.262605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:11.262875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d48260054d84784b89a52d9b766d7c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:11.277188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d48260054d84784b89a52d9b766d7c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:11.278383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d48260054d84784b89a52d9b766d7c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:11.279227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-513bb109-0266-4485-a6da-23b9fda584ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d48260054d84784b89a52d9b766d7c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:11.281220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:11.282868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:11.295875 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1705/3393] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:01:11 2022] DELETE /identity/v3/users/0d48260054d84784b89a52d9b766d7c1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:11.300555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:11.301837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:11.302115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:11.306087 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1689/3394] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:01:11.334214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:11.335753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:11.461571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1706/3395] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:11.475260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:11.487676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc556c-32c0-4a48-be21-6d0e97834611 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:11.552327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:11.554953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:11.554953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:11.554953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:11.564361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:11.566579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:11.580432 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1690/3396] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:01:12.596249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:12.597654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:12.625898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0b362-8306-4d4b-9160-88abb5febfab None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:12.645825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:12.647329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:12.649334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:12.649334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] PATH_INFO: `/v3/projects/3515123097e744ef855ef1129fb59cf1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:12.649334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3515123097e744ef855ef1129fb59cf1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:12.655422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:12.756879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3515123097e744ef855ef1129fb59cf1', 'request_id': 'req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:12.785509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3515123097e744ef855ef1129fb59cf1', 'request_id': 'req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:12.828697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3515123097e744ef855ef1129fb59cf1', 'request_id': 'req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:12.836299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:12.863049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] The token cache is being invalidate because project 3515123097e744ef855ef1129fb59cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:01:12.863349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:12.864333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:12.869669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1707/3397] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:01:12 2022] DELETE /identity/v3/projects/3515123097e744ef855ef1129fb59cf1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:12.870839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:12.882317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115d402-ff46-499c-b709-6d93ed1461ef service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:12.966435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:12.967277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:12.967593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:12.967890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] PATH_INFO: `/v3/users/d3738b013fa24822a134fee91bffa9c1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:12.969014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3738b013fa24822a134fee91bffa9c1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:12.971325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:13.018476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:13.020445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:13.093393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3738b013fa24822a134fee91bffa9c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.096006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3738b013fa24822a134fee91bffa9c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.108020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3738b013fa24822a134fee91bffa9c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.109046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3738b013fa24822a134fee91bffa9c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.129773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3738b013fa24822a134fee91bffa9c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.152236 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1691/3398] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:01:12 2022] DELETE /identity/v3/users/d3738b013fa24822a134fee91bffa9c1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:13.173266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed7b9-7ddc-480b-ad34-5f4d980026ac admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:13.294472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64fdfd-15b9-44db-93ec-043ca813fcaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:13.295509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64fdfd-15b9-44db-93ec-043ca813fcaa service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:13.295883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64fdfd-15b9-44db-93ec-043ca813fcaa service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:13.296585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64fdfd-15b9-44db-93ec-043ca813fcaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:13.298892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0f64fdfd-15b9-44db-93ec-043ca813fcaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:13.314379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64fdfd-15b9-44db-93ec-043ca813fcaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:13.316424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64fdfd-15b9-44db-93ec-043ca813fcaa service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:13.334270 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1708/3399] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:01:13.428175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:13.429749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:13.460338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:13.627916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7d80-4c5b-49cc-94b5-94b4d6551878 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:13.632769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:13.633426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:13.633737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:13.634008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] PATH_INFO: `/v3/projects/e0fca99ea853470d9fa1df0bd87fcfcc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:13.635429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fca99ea853470d9fa1df0bd87fcfcc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:13.637971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:13.711891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fca99ea853470d9fa1df0bd87fcfcc', 'request_id': 'req-66795aa3-6403-40fa-b0d0-dc7c761bf702'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.722959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fca99ea853470d9fa1df0bd87fcfcc', 'request_id': 'req-66795aa3-6403-40fa-b0d0-dc7c761bf702'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.740768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fca99ea853470d9fa1df0bd87fcfcc', 'request_id': 'req-66795aa3-6403-40fa-b0d0-dc7c761bf702'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.758098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] The token cache is being invalidate because project e0fca99ea853470d9fa1df0bd87fcfcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:01:13.758895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.760190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-66795aa3-6403-40fa-b0d0-dc7c761bf702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:13.763051 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1692/3400] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:01:13 2022] DELETE /identity/v3/projects/e0fca99ea853470d9fa1df0bd87fcfcc => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:13.769629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247392-131d-4832-8c70-48c021e4ec66 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:13.770883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247392-131d-4832-8c70-48c021e4ec66 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:13.771400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247392-131d-4832-8c70-48c021e4ec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:13.798175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ef247392-131d-4832-8c70-48c021e4ec66 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:13.800911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ef247392-131d-4832-8c70-48c021e4ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:13.929334 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1709/3401] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:13.937989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcdf24-d2cd-49a6-ada9-9416b19b60dd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:13.939007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcdf24-d2cd-49a6-ada9-9416b19b60dd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:13.939286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcdf24-d2cd-49a6-ada9-9416b19b60dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:13.962904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f8fcdf24-d2cd-49a6-ada9-9416b19b60dd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:13.964453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f8fcdf24-d2cd-49a6-ada9-9416b19b60dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:13.972964 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1693/3402] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:13.977191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef247392-131d-4832-8c70-48c021e4ec66 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:13.987917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef247392-131d-4832-8c70-48c021e4ec66 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:13.994677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:13.995067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:13.995439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:13.995831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:13.996666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:01:13.997074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:13.999279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:14.017016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcdf24-d2cd-49a6-ada9-9416b19b60dd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:14.028591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1710/3403] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:01:14.034435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcdf24-d2cd-49a6-ada9-9416b19b60dd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:14.059197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:14.059838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:14.060112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:14.060393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:14.061269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:14.063379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:14.106107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:14.106702 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1694/3404] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:01:14 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:14.119142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41540cec-ec20-4c38-9063-50c38c55b4aa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:14.137549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:14.211676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:14.212454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:14.212730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:14.213004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:14.213679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:14.215636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:14.244820 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1711/3405] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:01:14 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:14.246308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:14.258549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdee74d-5c23-44ef-aff8-92482e4f2dbb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:14.265425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:14.266165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:14.266478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:14.266845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:14.267735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:14.269339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:14.292815 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1695/3406] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:14.297059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:14.315093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bb39e-fd60-48c6-ace1-0ba7a90a2da6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:14.321501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3816932-49df-420d-9dac-8926da0b14bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:14.322275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816932-49df-420d-9dac-8926da0b14bc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:14.322640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816932-49df-420d-9dac-8926da0b14bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:14.322968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816932-49df-420d-9dac-8926da0b14bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:14.323682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3816932-49df-420d-9dac-8926da0b14bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:14.325558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3816932-49df-420d-9dac-8926da0b14bc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:14.339837 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1712/3407] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:14.344015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:14.356467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c535c-7542-4ba0-ab3d-e3ac347ba27d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:14.362292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:14.363249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:14.363590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:14.363933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] PATH_INFO: `/v3/projects/0d42cba2664646f6adb4202574b7fa5a/users/7fc9559a966444d98e54edb4f948ae1a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:14.397247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d42cba2664646f6adb4202574b7fa5a, user_id=7fc9559a966444d98e54edb4f948ae1a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:14.400337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:14.430064 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1696/3408] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:01:14 2022] PUT /identity/v3/projects/0d42cba2664646f6adb4202574b7fa5a/users/7fc9559a966444d98e54edb4f948ae1a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:14.442259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb00a-f7d5-4eba-a126-394b290f6d21 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:14.442440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb00a-f7d5-4eba-a126-394b290f6d21 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:14.442657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb00a-f7d5-4eba-a126-394b290f6d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:14.485733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c7fcb00a-f7d5-4eba-a126-394b290f6d21 None None] MFA Rules not processed for user `7fc9559a966444d98e54edb4f948ae1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:14.487198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c7fcb00a-f7d5-4eba-a126-394b290f6d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:14.620415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1713/3409] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:14.636971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:14.638817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:14.688107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:14.739962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f91ba-5b7a-4c42-b9ca-d3d560bd2032 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:14.744218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:14.744843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:14.745222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:14.745603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:14.754853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:14.756333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:14.766693 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1697/3410] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:01:15.927409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd998-0ae4-4828-b2cb-0feddb0793f7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:15.928016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd998-0ae4-4828-b2cb-0feddb0793f7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:15.928016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd998-0ae4-4828-b2cb-0feddb0793f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:15.947872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b32fd998-0ae4-4828-b2cb-0feddb0793f7 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:15.949081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b32fd998-0ae4-4828-b2cb-0feddb0793f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:16.008433 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1714/3411] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:16.030506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:16.032095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:16.118749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3e52b-c338-47ab-8f24-37bbc461ae6f service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:16.124160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:16.124762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:16.125055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:16.125308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:16.133938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:16.135600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:16.144388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1698/3412] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:01:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:01:17.913427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:17.915341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:17.915898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:17.943809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:17.945438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:17.971204 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1715/3413] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:17.972465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:17.989804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3d2ca-c4fb-4ca2-a506-bb1fec49cbc6 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:18.018421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:18.019178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:18.019551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:18.019890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:18.039388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:18.041215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:18.062424 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1699/3414] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:01:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:01:19.412079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:19.414154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:19.435117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:19.490922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025e3ec-3604-46ed-a271-b895a7d2c75d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:19.497386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:19.498134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:19.498421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:19.498693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] PATH_INFO: `/v3/users/55e8e491f8844f7696f93d8ec5a7171b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:19.499976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e8e491f8844f7696f93d8ec5a7171b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:19.501823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:19.565727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e8e491f8844f7696f93d8ec5a7171b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.578020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e8e491f8844f7696f93d8ec5a7171b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.587247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e8e491f8844f7696f93d8ec5a7171b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.587930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e8e491f8844f7696f93d8ec5a7171b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.588604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e8e491f8844f7696f93d8ec5a7171b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.601248 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1716/3415] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:01:19 2022] DELETE /identity/v3/users/55e8e491f8844f7696f93d8ec5a7171b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:19.780557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:19.795672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2adee3-ef06-437a-82f7-e189a334ae7c service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:19.868564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:19.869201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:19.869508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:19.869787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] PATH_INFO: `/v3/projects/822801d3963d45c1b4176259fb5c5b19` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:19.870946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=822801d3963d45c1b4176259fb5c5b19)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:19.872852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:19.908564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:19.926930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822801d3963d45c1b4176259fb5c5b19', 'request_id': 'req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.934059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822801d3963d45c1b4176259fb5c5b19', 'request_id': 'req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.942914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822801d3963d45c1b4176259fb5c5b19', 'request_id': 'req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.953910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] The token cache is being invalidate because project 822801d3963d45c1b4176259fb5c5b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:01:19.954324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.954950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b88e9c26-1e4a-4ca1-8fb8-39f5759b2ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:19.956452 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1700/3416] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:01:19 2022] DELETE /identity/v3/projects/822801d3963d45c1b4176259fb5c5b19 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:19.960605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:19.962093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:19.975585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f185d1-4de1-4cdc-9319-dc0b0cfab0d4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:20.042196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:20.042980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:20.043268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:20.043537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] PATH_INFO: `/v3/users/35e971080bf140cabf769606048f1436` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:20.044914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e971080bf140cabf769606048f1436)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:20.047127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:20.070800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:20.071258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:20.071666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:20.092765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:20.094306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:20.095805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:20.107463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e971080bf140cabf769606048f1436'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.125115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e971080bf140cabf769606048f1436'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.134405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e971080bf140cabf769606048f1436'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.135491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e971080bf140cabf769606048f1436'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.136654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e971080bf140cabf769606048f1436'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.164894 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1717/3417] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:01:19 2022] DELETE /identity/v3/users/35e971080bf140cabf769606048f1436 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:01:20.185201 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1701/3418] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:01:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:20.199933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:20.202387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:20.294772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb02df-72da-4302-bab9-6ab9330d2b20 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:20.303481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c57c8-e04e-4e31-bbfc-d4bd4b3f6e91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:20.304131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c57c8-e04e-4e31-bbfc-d4bd4b3f6e91 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:20.304544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c57c8-e04e-4e31-bbfc-d4bd4b3f6e91 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:20.304818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c57c8-e04e-4e31-bbfc-d4bd4b3f6e91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:20.392074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:20.393955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:20.419090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3815d27-abd4-485d-99e8-452c7957f111 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:20.420491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c57c8-e04e-4e31-bbfc-d4bd4b3f6e91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:20.422952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c57c8-e04e-4e31-bbfc-d4bd4b3f6e91 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:20.438779 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1718/3419] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:01:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:01:20.534440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:20.536177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:20.536650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:20.537136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] PATH_INFO: `/v3/projects/3da703231acd4485b6620f22891ad268` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:20.539720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3da703231acd4485b6620f22891ad268)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:20.542933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:20.631654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3da703231acd4485b6620f22891ad268', 'request_id': 'req-9e733d0c-5379-4f05-830c-3cd1768bb227'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.639545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3da703231acd4485b6620f22891ad268', 'request_id': 'req-9e733d0c-5379-4f05-830c-3cd1768bb227'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.661693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3da703231acd4485b6620f22891ad268', 'request_id': 'req-9e733d0c-5379-4f05-830c-3cd1768bb227'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.664457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:20.688891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] The token cache is being invalidate because project 3da703231acd4485b6620f22891ad268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:01:20.689190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.689916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9e733d0c-5379-4f05-830c-3cd1768bb227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:20.695692 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1702/3420] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:01:20 2022] DELETE /identity/v3/projects/3da703231acd4485b6620f22891ad268 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:20.704173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94a68b-ed5f-409c-9c59-6dbd3c49a8e6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:20.704520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94a68b-ed5f-409c-9c59-6dbd3c49a8e6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:20.704855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94a68b-ed5f-409c-9c59-6dbd3c49a8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:20.742901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4c94a68b-ed5f-409c-9c59-6dbd3c49a8e6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:20.746813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4c94a68b-ed5f-409c-9c59-6dbd3c49a8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:20.757490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18030-ec55-4ad8-9ca3-1786af8d6ee6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:20.757793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18030-ec55-4ad8-9ca3-1786af8d6ee6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:20.758067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18030-ec55-4ad8-9ca3-1786af8d6ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:20.788617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-48e18030-ec55-4ad8-9ca3-1786af8d6ee6 None None] MFA Rules not processed for user `7fc9559a966444d98e54edb4f948ae1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:20.789878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-48e18030-ec55-4ad8-9ca3-1786af8d6ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:20.866418 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1703/3421] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:20.878910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:20.900612 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1719/3422] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:20.909432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fcf71-fade-4004-86ee-5cbee42308a0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:20.909746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fcf71-fade-4004-86ee-5cbee42308a0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:20.910736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fcf71-fade-4004-86ee-5cbee42308a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:20.964031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-414fcf71-fade-4004-86ee-5cbee42308a0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:20.966090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-414fcf71-fade-4004-86ee-5cbee42308a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:20.985368 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1704/3423] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:20.989946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94a68b-ed5f-409c-9c59-6dbd3c49a8e6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:21.001990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94a68b-ed5f-409c-9c59-6dbd3c49a8e6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:21.007044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:21.008160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.008160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.008362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.009975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:01:21.009975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:21.011424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:21.025732 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1720/3424] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:01:21.032292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fcf71-fade-4004-86ee-5cbee42308a0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:21.064364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fcf71-fade-4004-86ee-5cbee42308a0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:21.078108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:21.078846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.079314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.080084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.080789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:21.083456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:21.163983 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1705/3425] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:01:21 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:21.165483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:21.180968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e19e5-2555-44bc-9b50-da3949eb5c32 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:21.289517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:21.290625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.291952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.292383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.293202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:21.295417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:21.330839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:21.331460 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1721/3426] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:01:21 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:21.347911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793e87f-399f-42eb-a936-869d0c6c1752 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:21.356781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:21.357945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.358445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.358722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.359377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:21.361064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:21.382846 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1706/3427] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:21.384998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:21.400540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0e244-cdfe-483c-9a37-7ad893263995 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:21.406488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28706c41-39cb-447c-aa3b-7f4da351e3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:21.407111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28706c41-39cb-447c-aa3b-7f4da351e3ff admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.407389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28706c41-39cb-447c-aa3b-7f4da351e3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.407660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28706c41-39cb-447c-aa3b-7f4da351e3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.408353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28706c41-39cb-447c-aa3b-7f4da351e3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:21.410054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28706c41-39cb-447c-aa3b-7f4da351e3ff admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:21.422704 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1722/3428] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:21.428298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:21.471208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd0d03-bba1-4ac2-8a86-0b4a717ca29c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:21.476685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2c5d1-4847-409d-8159-a0c47407fa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:21.477531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c5d1-4847-409d-8159-a0c47407fa16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.479866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c5d1-4847-409d-8159-a0c47407fa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.480238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c5d1-4847-409d-8159-a0c47407fa16 admin admin] PATH_INFO: `/v3/projects/3e48e3c2f7f74636a75679fdf10ef33a/users/2ef547e071fd43ec96c745c26eca119f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.500174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2c5d1-4847-409d-8159-a0c47407fa16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e48e3c2f7f74636a75679fdf10ef33a, user_id=2ef547e071fd43ec96c745c26eca119f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:21.502966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2c5d1-4847-409d-8159-a0c47407fa16 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:21.526765 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1707/3429] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:01:21 2022] PUT /identity/v3/projects/3e48e3c2f7f74636a75679fdf10ef33a/users/2ef547e071fd43ec96c745c26eca119f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:01:21.535026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.535391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.535731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.559041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:21.560247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:21.654180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b874851-9da7-4fd5-a899-703c15b7e73d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.654630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b874851-9da7-4fd5-a899-703c15b7e73d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.655051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b874851-9da7-4fd5-a899-703c15b7e73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.660739 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1723/3430] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:21.677482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:21.678185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9b874851-9da7-4fd5-a899-703c15b7e73d None None] MFA Rules not processed for user `7fc9559a966444d98e54edb4f948ae1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:21.679386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b874851-9da7-4fd5-a899-703c15b7e73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:21.681902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:21.754882 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1708/3431] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:21.803450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d21dd-5708-42e3-9fdc-44c235151a4d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:21.807969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c118e-8d86-4c20-861a-fe331f68164b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:21.808816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c118e-8d86-4c20-861a-fe331f68164b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:21.809169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c118e-8d86-4c20-861a-fe331f68164b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:21.809534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c118e-8d86-4c20-861a-fe331f68164b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:21.822020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c118e-8d86-4c20-861a-fe331f68164b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:21.823770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c118e-8d86-4c20-861a-fe331f68164b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:21.837753 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1724/3432] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:01:26.879387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe387a0-01db-463e-b6b7-2f08858d9652 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:26.880586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe387a0-01db-463e-b6b7-2f08858d9652 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:26.881070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe387a0-01db-463e-b6b7-2f08858d9652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:26.918114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-efe387a0-01db-463e-b6b7-2f08858d9652 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:26.921845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-efe387a0-01db-463e-b6b7-2f08858d9652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:27.001949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:27.060084 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1709/3433] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:30.524992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd41b0-bb6e-4ba1-9c82-befe5705a044 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:30.526234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd41b0-bb6e-4ba1-9c82-befe5705a044 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:30.526697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd41b0-bb6e-4ba1-9c82-befe5705a044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:30.536379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:30.560476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-51bd41b0-bb6e-4ba1-9c82-befe5705a044 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:30.562085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-51bd41b0-bb6e-4ba1-9c82-befe5705a044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:30.580863 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1725/3434] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:31.012683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:31.013229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:31.013347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:31.065276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] MFA Rules not processed for user `22306a93c9404db393b92a267d6258d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:31.068086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:31.178407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1710/3435] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:33.294607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d939-be4e-4689-9db3-b4a278235e09 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:33.296042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d939-be4e-4689-9db3-b4a278235e09 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:33.296522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d939-be4e-4689-9db3-b4a278235e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:33.358564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d501d939-be4e-4689-9db3-b4a278235e09 None None] MFA Rules not processed for user `2ef547e071fd43ec96c745c26eca119f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:33.360166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d501d939-be4e-4689-9db3-b4a278235e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:33.462625 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1726/3436] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:33.490660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:33.492196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:33.507285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74320c92-eebc-4a6b-a856-3677488d2bd4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:33.512634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:33.513284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:33.513598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:33.513864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] PATH_INFO: `/v3/users/d1bc1e85c16e49c695f073f2a7b40a52` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:33.515061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1bc1e85c16e49c695f073f2a7b40a52)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:33.517339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:33.574221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1bc1e85c16e49c695f073f2a7b40a52'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:33.575250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1bc1e85c16e49c695f073f2a7b40a52'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:33.585242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1bc1e85c16e49c695f073f2a7b40a52'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:33.586016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1bc1e85c16e49c695f073f2a7b40a52'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:33.603158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1bc1e85c16e49c695f073f2a7b40a52'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:33.620928 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1711/3437] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:01:33 2022] DELETE /identity/v3/users/d1bc1e85c16e49c695f073f2a7b40a52 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:33.977243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d939-be4e-4689-9db3-b4a278235e09 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:33.997111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d939-be4e-4689-9db3-b4a278235e09 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:34.053310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:34.054794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:34.130997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:34.131874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:34.132296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:34.132786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] PATH_INFO: `/v3/projects/77e4ba38e25a4143adfe864a8408c524` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:34.134155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e4ba38e25a4143adfe864a8408c524)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:34.136640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:34.195267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:34.205261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e4ba38e25a4143adfe864a8408c524', 'request_id': 'req-728e4e7a-5dc2-4058-b066-cfa0185c8746'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:34.225239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e4ba38e25a4143adfe864a8408c524', 'request_id': 'req-728e4e7a-5dc2-4058-b066-cfa0185c8746'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:34.226912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9470fa-b4c3-4632-aa9b-9953b9914f9c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:34.233795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e4ba38e25a4143adfe864a8408c524', 'request_id': 'req-728e4e7a-5dc2-4058-b066-cfa0185c8746'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:34.278322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] The token cache is being invalidate because project 77e4ba38e25a4143adfe864a8408c524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:01:34.278614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:34.279721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-728e4e7a-5dc2-4058-b066-cfa0185c8746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:34.282447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1727/3438] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:01:33 2022] DELETE /identity/v3/projects/77e4ba38e25a4143adfe864a8408c524 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:34.287522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b5b4-93dd-4729-aa8f-95a2396aa4dd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:34.287817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b5b4-93dd-4729-aa8f-95a2396aa4dd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:34.288094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b5b4-93dd-4729-aa8f-95a2396aa4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:34.348119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fe91b5b4-93dd-4729-aa8f-95a2396aa4dd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:34.351244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe91b5b4-93dd-4729-aa8f-95a2396aa4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:34.371272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:34.415173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:34.415830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:34.416116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:34.416392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:34.424829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:34.552411 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1728/3439] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:34.561212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd01706-4570-430f-8d5e-fd24a92f85ce None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:34.561875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd01706-4570-430f-8d5e-fd24a92f85ce None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:34.562379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd01706-4570-430f-8d5e-fd24a92f85ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:34.596635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9bd01706-4570-430f-8d5e-fd24a92f85ce None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:34.598263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9bd01706-4570-430f-8d5e-fd24a92f85ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:34.613938 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1729/3440] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:34.617641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd01706-4570-430f-8d5e-fd24a92f85ce None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:34.631725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd01706-4570-430f-8d5e-fd24a92f85ce None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:34.638869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:34.639620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:34.640492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:34.641070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:34.641257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:34.641833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:34.642770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:01:34.643243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:34.645384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:34.664392 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1730/3441] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:01:34.669071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:34.669259 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1712/3442] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:01:34.692315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4d0d8-a087-4691-9da6-8d8db8c9494c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:34.702319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:34.703686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:34.705733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:34.706323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:34.708280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:34.712597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:34.793624 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1731/3443] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:01:34 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:34.794884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:34.812295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e60d0-9a7b-4cec-8547-729339a786c8 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:34.995047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:34.995740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:34.996018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:34.996292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:35.001652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:35.006967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:35.066040 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1713/3444] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:01:34 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:35.067391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:35.083215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bf667-38da-4242-a1e0-5d444c2b38f6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:35.095473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:35.096659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:35.097009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:35.097298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:35.098243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:35.099995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:35.135463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:35.136284 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1732/3445] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:35.156950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e512-dc53-41af-894f-8498d2e02b83 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:35.179078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c90452-08a8-4a1e-8d9d-85a855e6c9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:35.179724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90452-08a8-4a1e-8d9d-85a855e6c9ab admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:35.180036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90452-08a8-4a1e-8d9d-85a855e6c9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:35.180264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90452-08a8-4a1e-8d9d-85a855e6c9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:35.180987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c90452-08a8-4a1e-8d9d-85a855e6c9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:35.185648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c90452-08a8-4a1e-8d9d-85a855e6c9ab admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:35.217604 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1714/3446] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:01:35.223585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:35.240467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0f2d9-b4d3-4796-89f5-10309f7db91c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:35.254188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d7c9-1228-4b6c-90d1-b8be699791a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:35.254876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d7c9-1228-4b6c-90d1-b8be699791a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:35.255331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d7c9-1228-4b6c-90d1-b8be699791a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:35.255612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d7c9-1228-4b6c-90d1-b8be699791a6 admin admin] PATH_INFO: `/v3/projects/6e147bacb13f42c9867f0c28daa93b47/users/81da2270eef945508fd99c0761f27cb5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:35.288135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d7c9-1228-4b6c-90d1-b8be699791a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e147bacb13f42c9867f0c28daa93b47, user_id=81da2270eef945508fd99c0761f27cb5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:35.291694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d7c9-1228-4b6c-90d1-b8be699791a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:35.314352 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1733/3447] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:01:35 2022] PUT /identity/v3/projects/6e147bacb13f42c9867f0c28daa93b47/users/81da2270eef945508fd99c0761f27cb5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:35.319363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:35.319649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:35.319924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:35.350450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:35.352486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:35.366638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:35.395500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20a4bc-305b-401f-acc2-56b0c0e452ec None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:35.396087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20a4bc-305b-401f-acc2-56b0c0e452ec None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:35.396087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20a4bc-305b-401f-acc2-56b0c0e452ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:35.435559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2d20a4bc-305b-401f-acc2-56b0c0e452ec None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:35.436876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d20a4bc-305b-401f-acc2-56b0c0e452ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:35.511291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1715/3448] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:35.542057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:35.543918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:35.584597 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1734/3449] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:35.683326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b342a-1bbb-453e-a4d1-95e6c70194a4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:35.690932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbda3d-7f8f-463b-981e-ad838954c166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:35.693578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbda3d-7f8f-463b-981e-ad838954c166 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:35.693873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbda3d-7f8f-463b-981e-ad838954c166 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:35.694147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbda3d-7f8f-463b-981e-ad838954c166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:35.713433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbda3d-7f8f-463b-981e-ad838954c166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:35.715225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbda3d-7f8f-463b-981e-ad838954c166 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:35.732063 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1716/3450] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:01:37.434635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37575b71-a8c0-4436-8aea-b2d462c06026 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:37.435277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37575b71-a8c0-4436-8aea-b2d462c06026 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:37.435646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37575b71-a8c0-4436-8aea-b2d462c06026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:37.495530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-37575b71-a8c0-4436-8aea-b2d462c06026 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:37.497246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-37575b71-a8c0-4436-8aea-b2d462c06026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:37.521904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1735/3451] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:40.271891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900caf-7817-447c-9e51-b7ecfc1cd326 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:40.272977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900caf-7817-447c-9e51-b7ecfc1cd326 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:40.273267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900caf-7817-447c-9e51-b7ecfc1cd326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:40.312003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-90900caf-7817-447c-9e51-b7ecfc1cd326 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:40.315268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-90900caf-7817-447c-9e51-b7ecfc1cd326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:40.337739 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1717/3452] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:41.161947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:41.162440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:41.162606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:41.216846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] MFA Rules not processed for user `2ef547e071fd43ec96c745c26eca119f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:41.218992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:41.277533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:41.337125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1736/3453] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:41.366440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974df334-99a3-4f74-8de5-57cabbc31502 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:41.368152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974df334-99a3-4f74-8de5-57cabbc31502 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:41.369554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974df334-99a3-4f74-8de5-57cabbc31502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:41.397798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-974df334-99a3-4f74-8de5-57cabbc31502 None None] MFA Rules not processed for user `3009c5feb1ef43a08b07996d8f63bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:41.398953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-974df334-99a3-4f74-8de5-57cabbc31502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:41.479161 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1718/3454] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:42.279064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:42.282059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:42.306895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2f5fc-82c0-4b34-9f38-5cfb0d4f9428 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:42.312879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed735d-0b7c-4e82-9812-e84592f69efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:42.313813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed735d-0b7c-4e82-9812-e84592f69efd service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:42.314096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed735d-0b7c-4e82-9812-e84592f69efd service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:42.314702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed735d-0b7c-4e82-9812-e84592f69efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:42.317009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-27ed735d-0b7c-4e82-9812-e84592f69efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:42.332537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed735d-0b7c-4e82-9812-e84592f69efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:42.335014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed735d-0b7c-4e82-9812-e84592f69efd service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:42.350890 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1737/3455] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:01:43.494634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae53f2-888b-4b0d-bf45-4660bcd2ba97 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:43.494634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae53f2-888b-4b0d-bf45-4660bcd2ba97 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:43.495235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae53f2-888b-4b0d-bf45-4660bcd2ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:43.589111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ddae53f2-888b-4b0d-bf45-4660bcd2ba97 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:43.599585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ddae53f2-888b-4b0d-bf45-4660bcd2ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:43.640060 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1719/3456] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:45.314405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770632-b569-4090-b2af-cb8aded998ca None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:45.314932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770632-b569-4090-b2af-cb8aded998ca None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:45.315037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef770632-b569-4090-b2af-cb8aded998ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:45.356680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ef770632-b569-4090-b2af-cb8aded998ca None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:45.359771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ef770632-b569-4090-b2af-cb8aded998ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:45.390442 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1738/3457] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:48.110760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:48.111328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:48.111328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:48.152853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:48.154368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:48.170617 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1720/3458] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:48.889361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98615a-fbed-4811-a4ef-1667b58f017c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:48.894773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98615a-fbed-4811-a4ef-1667b58f017c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:48.895933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98615a-fbed-4811-a4ef-1667b58f017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:49.011893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-de98615a-fbed-4811-a4ef-1667b58f017c None None] MFA Rules not processed for user `81da2270eef945508fd99c0761f27cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:49.013162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-de98615a-fbed-4811-a4ef-1667b58f017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:49.184360 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1739/3459] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:49.783400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:49.785010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:49.816292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dd759-5077-4c31-9842-5d18e6b9314f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:49.821727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:49.822384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:49.822660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:49.822935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] PATH_INFO: `/v3/users/1fff4805e6fb45a0b7c8b4dc08c4dedb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:49.825245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fff4805e6fb45a0b7c8b4dc08c4dedb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:49.827329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:49.929331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:49.947564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fff4805e6fb45a0b7c8b4dc08c4dedb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:49.949282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fff4805e6fb45a0b7c8b4dc08c4dedb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:49.976244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fff4805e6fb45a0b7c8b4dc08c4dedb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:49.977542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fff4805e6fb45a0b7c8b4dc08c4dedb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:50.006184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fff4805e6fb45a0b7c8b4dc08c4dedb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:50.034349 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1721/3460] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:01:49 2022] DELETE /identity/v3/users/1fff4805e6fb45a0b7c8b4dc08c4dedb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:50.039313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a497-24bb-4e75-9f16-955ed9cd1e2f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:50.039906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a497-24bb-4e75-9f16-955ed9cd1e2f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:50.039906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a497-24bb-4e75-9f16-955ed9cd1e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:50.068710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1029a497-24bb-4e75-9f16-955ed9cd1e2f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:50.071832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1029a497-24bb-4e75-9f16-955ed9cd1e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:50.145365 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1740/3461] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:50.158727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:50.160273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:50.256586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4ee91-8d80-4659-90a4-55bd676feb18 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:50.264203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:50.265189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:50.265694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:50.266755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:50.283357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:50.285430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:50.331856 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1722/3462] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:01:50.418229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:50.418840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:50.419277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:50.444173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:50.445842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:50.504396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:50.516181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1741/3463] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:50.533081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:50.534578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:50.573386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:50.577146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:50.599873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6c89c-d5f8-4515-b828-ff43037bf308 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:50.605067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:50.606154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:50.606604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:50.607012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] PATH_INFO: `/v3/projects/a0c2cdfddf5b4d35a0ca3c7554271365` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:50.608688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c2cdfddf5b4d35a0ca3c7554271365)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:50.611655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:50.711800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c2cdfddf5b4d35a0ca3c7554271365', 'request_id': 'req-fdad7faf-f67f-4606-b439-4a79dcddb971'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:50.714806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba886-c176-4c98-b76d-1bdecde1aeeb service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:50.726086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c2cdfddf5b4d35a0ca3c7554271365', 'request_id': 'req-fdad7faf-f67f-4606-b439-4a79dcddb971'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:50.753618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c2cdfddf5b4d35a0ca3c7554271365', 'request_id': 'req-fdad7faf-f67f-4606-b439-4a79dcddb971'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:50.777368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] The token cache is being invalidate because project a0c2cdfddf5b4d35a0ca3c7554271365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:01:50.777691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:50.778412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:50.781820 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1742/3464] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:01:50 2022] DELETE /identity/v3/projects/a0c2cdfddf5b4d35a0ca3c7554271365 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:50.785562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:50.787480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:50.824069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:50.847224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ea943-1cbf-4422-aac3-37204d2cfd95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:50.847856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea943-1cbf-4422-aac3-37204d2cfd95 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:50.848128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea943-1cbf-4422-aac3-37204d2cfd95 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:50.848436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ea943-1cbf-4422-aac3-37204d2cfd95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:50.852587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-078ea943-1cbf-4422-aac3-37204d2cfd95 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:50.922996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad7faf-f67f-4606-b439-4a79dcddb971 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:50.934345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:50.934933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:50.935223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:50.935496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] PATH_INFO: `/v3/users/0cae9638f3e04011b9afe8c334303e12` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:50.936676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cae9638f3e04011b9afe8c334303e12)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:50.939058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:50.995086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ea943-1cbf-4422-aac3-37204d2cfd95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:51.000527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ea943-1cbf-4422-aac3-37204d2cfd95 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:51.027667 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1723/3465] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:01:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:01:51.032215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17df7-ebe4-48dc-87b1-f36574de1717 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:51.034540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17df7-ebe4-48dc-87b1-f36574de1717 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:51.035449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17df7-ebe4-48dc-87b1-f36574de1717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:51.044338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cae9638f3e04011b9afe8c334303e12'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:51.104221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cae9638f3e04011b9afe8c334303e12'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:51.108168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-96b17df7-ebe4-48dc-87b1-f36574de1717 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:51.109412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-96b17df7-ebe4-48dc-87b1-f36574de1717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:51.131134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cae9638f3e04011b9afe8c334303e12'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:51.132383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cae9638f3e04011b9afe8c334303e12'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:51.133025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cae9638f3e04011b9afe8c334303e12'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:51.155066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:51.179272 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1743/3466] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:01:50 2022] DELETE /identity/v3/users/0cae9638f3e04011b9afe8c334303e12 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:51.353055 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1724/3467] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:51.570794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:51.572278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:51.781396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60453098-27f0-4c0b-8682-29ca57fe2f69 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:51.789973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:51.790908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:51.791370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:51.791999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] PATH_INFO: `/v3/projects/c81f6785bd004f7c8d04b6d5979b4aa4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:51.795628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81f6785bd004f7c8d04b6d5979b4aa4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:51.798888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:51.963392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81f6785bd004f7c8d04b6d5979b4aa4', 'request_id': 'req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:51.981636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81f6785bd004f7c8d04b6d5979b4aa4', 'request_id': 'req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:52.004414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81f6785bd004f7c8d04b6d5979b4aa4', 'request_id': 'req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:52.036426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] The token cache is being invalidate because project c81f6785bd004f7c8d04b6d5979b4aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:01:52.038099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:52.039119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-769b9185-7c9f-4c80-b7c8-71ccabdd0c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:01:52.042029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1744/3468] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:01:51 2022] DELETE /identity/v3/projects/c81f6785bd004f7c8d04b6d5979b4aa4 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:52.048189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f7a64-d65e-4f3b-a1ba-cae29625abbe None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.048707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f7a64-d65e-4f3b-a1ba-cae29625abbe None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.048707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f7a64-d65e-4f3b-a1ba-cae29625abbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.108465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a00f7a64-d65e-4f3b-a1ba-cae29625abbe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:52.117515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a00f7a64-d65e-4f3b-a1ba-cae29625abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:52.275143 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1725/3469] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:52.284713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899c47-9968-4247-86f9-57e7425203dd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.285915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899c47-9968-4247-86f9-57e7425203dd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.286424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899c47-9968-4247-86f9-57e7425203dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.322450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-77899c47-9968-4247-86f9-57e7425203dd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:52.325009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-77899c47-9968-4247-86f9-57e7425203dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:52.341028 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1745/3470] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:01:52.341904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f7a64-d65e-4f3b-a1ba-cae29625abbe None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:52.361098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f7a64-d65e-4f3b-a1ba-cae29625abbe None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:52.366556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:52.367753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.368274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.368766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.369694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:01:52.370227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:52.372153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:52.404225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1726/3471] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:01:52.414829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899c47-9968-4247-86f9-57e7425203dd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:52.428981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899c47-9968-4247-86f9-57e7425203dd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:52.434211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:52.434940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.435288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.435629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.437142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:52.440302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:52.483449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:52.487377 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1746/3472] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:01:52 2022] POST /identity/v3/projects => generated 391 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:52.498426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be1263-685b-43a7-ab41-dfa409c4a593 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:52.587624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:52.663400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:52.664817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.665187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.666091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.682130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:52.684738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:52.732585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:52.741291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1727/3473] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:01:52 2022] POST /identity/v3/users => generated 296 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:01:52.749748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0f2fe-1b34-4f78-a3b4-597149003632 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:52.769991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:52.770821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.771238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.771668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.772637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:52.775358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:52.789954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1747/3474] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:52.794153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:52.809803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a46f8-14b0-471e-b306-eee331438b90 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:52.815856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23aed4-8f3e-4324-aebf-aacfe03bea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:52.816492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23aed4-8f3e-4324-aebf-aacfe03bea60 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.816818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23aed4-8f3e-4324-aebf-aacfe03bea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.817094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23aed4-8f3e-4324-aebf-aacfe03bea60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.817798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23aed4-8f3e-4324-aebf-aacfe03bea60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:52.820506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23aed4-8f3e-4324-aebf-aacfe03bea60 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:52.835173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1728/3475] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:52.840187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:52.859642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf472f99-6a02-4cfe-972a-0989ffaee454 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:52.886810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:52.887662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:52.888050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:52.889060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] PATH_INFO: `/v3/projects/1894770a3dfd4dd2aaa4a2b02f9a7f36/users/e570c2dfca474a089d281ac21a859e98/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:52.950187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1894770a3dfd4dd2aaa4a2b02f9a7f36, user_id=e570c2dfca474a089d281ac21a859e98, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:52.954246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:52.991419 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1748/3476] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:01:52 2022] PUT /identity/v3/projects/1894770a3dfd4dd2aaa4a2b02f9a7f36/users/e570c2dfca474a089d281ac21a859e98/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:53.002958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bdac8-fe0e-475c-91af-d34d8c734592 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:53.003377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bdac8-fe0e-475c-91af-d34d8c734592 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:53.003750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bdac8-fe0e-475c-91af-d34d8c734592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:53.056213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e85bdac8-fe0e-475c-91af-d34d8c734592 None None] MFA Rules not processed for user `e570c2dfca474a089d281ac21a859e98`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:53.058238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e85bdac8-fe0e-475c-91af-d34d8c734592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:53.214367 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1729/3477] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:53.217984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:53.300063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:53.366191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05761868-eca6-4b42-919d-714c6ff95272 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:53.374707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:53.377348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:53.377785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:53.378483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:53.379183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:53.383124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:53.459060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1749/3478] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:01:53 2022] POST /identity/v3/projects => generated 393 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:01:53.463195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bdac8-fe0e-475c-91af-d34d8c734592 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:53.485142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bdac8-fe0e-475c-91af-d34d8c734592 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:53.661762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:53.662408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:53.662687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:53.662958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:53.663596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:53.665612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:53.746760 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1730/3479] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:01:53 2022] POST /identity/v3/users => generated 303 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:01:53.751083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:53.770186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e109914-2902-4af6-96ef-ad176ba8eacc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:53.781185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:53.782174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:53.782174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:53.782385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:53.783759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:53.787038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:53.808950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1750/3480] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:53.813648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:53.824956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0869bf2-8e30-49eb-bcfb-1a3adb3e18b3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:53.844102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:53.845274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:53.845947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:53.850265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] PATH_INFO: `/v3/projects/c2d9a9f0412940ee9c92d54c7ac16d4e/users/9128af1af1684c82ae2db52f2ccaa0b9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:53.891374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d9a9f0412940ee9c92d54c7ac16d4e, user_id=9128af1af1684c82ae2db52f2ccaa0b9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:53.899121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:53.948892 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1731/3481] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:01:53 2022] PUT /identity/v3/projects/c2d9a9f0412940ee9c92d54c7ac16d4e/users/9128af1af1684c82ae2db52f2ccaa0b9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:53.953668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:53.976548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d078e04-51f3-4b0f-9c6e-2fa32976befe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:54.124304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe890e0-c506-4e5b-b7f8-df0c525176f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:54.124994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe890e0-c506-4e5b-b7f8-df0c525176f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:54.125882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe890e0-c506-4e5b-b7f8-df0c525176f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:54.125882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe890e0-c506-4e5b-b7f8-df0c525176f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:54.127778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe890e0-c506-4e5b-b7f8-df0c525176f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:54.132814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe890e0-c506-4e5b-b7f8-df0c525176f2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:54.166080 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1751/3482] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:01:54.168587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:54.178012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:54.187647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35328c96-d7af-4d43-be00-b770bba0291b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:54.199208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7aa92-31d9-4ec8-b20f-4849c3ab5b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:54.199842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7aa92-31d9-4ec8-b20f-4849c3ab5b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:54.209598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7aa92-31d9-4ec8-b20f-4849c3ab5b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:54.211192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7aa92-31d9-4ec8-b20f-4849c3ab5b89 admin admin] PATH_INFO: `/v3/projects/c2d9a9f0412940ee9c92d54c7ac16d4e/users/9128af1af1684c82ae2db52f2ccaa0b9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:54.218016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7aa92-31d9-4ec8-b20f-4849c3ab5b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d9a9f0412940ee9c92d54c7ac16d4e, user_id=9128af1af1684c82ae2db52f2ccaa0b9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:54.224265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7aa92-31d9-4ec8-b20f-4849c3ab5b89 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:54.244185 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1732/3483] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:01:54 2022] PUT /identity/v3/projects/c2d9a9f0412940ee9c92d54c7ac16d4e/users/9128af1af1684c82ae2db52f2ccaa0b9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:01:54.256209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:54.256551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:54.256832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:54.326302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] MFA Rules not processed for user `9128af1af1684c82ae2db52f2ccaa0b9`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:54.328945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:54.431362 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1752/3484] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:54.444633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15dd84-2968-4f5d-a6cc-c2a4dbaf5855 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:54.445127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15dd84-2968-4f5d-a6cc-c2a4dbaf5855 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:54.445703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15dd84-2968-4f5d-a6cc-c2a4dbaf5855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:54.501576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ac15dd84-2968-4f5d-a6cc-c2a4dbaf5855 None None] MFA Rules not processed for user `e570c2dfca474a089d281ac21a859e98`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:54.506957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac15dd84-2968-4f5d-a6cc-c2a4dbaf5855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:54.655460 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1733/3485] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:01:55.389078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:01:55.392011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:55.439122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:55.542591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101e1a9-37e7-45d4-9801-d6dcda94db16 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:01:55.553506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffa750-30de-4c15-906f-157ffe1bcf70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:01:55.554525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffa750-30de-4c15-906f-157ffe1bcf70 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:55.554965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffa750-30de-4c15-906f-157ffe1bcf70 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:55.555351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffa750-30de-4c15-906f-157ffe1bcf70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:55.566888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffa750-30de-4c15-906f-157ffe1bcf70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:01:55.569112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffa750-30de-4c15-906f-157ffe1bcf70 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:01:55.580667 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1753/3486] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:01:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:01:56.928568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:01:56.929130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:01:56.929438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:01:56.966328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:01:56.970248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:01:57.139289 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1734/3487] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:01:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:00.298164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:00.298767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:00.298767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:00.355908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:00.362313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:00.529009 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1754/3488] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:00.548596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:00.550232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:00.757744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab1fa-bdfd-4d29-adaf-6c18d7a6c1d6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:00.774425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:00.775937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:00.776437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:00.776923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:00.787925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:00.790726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:00.806322 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1735/3489] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:02:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:02:01.517796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:01.520215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:01.649326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbbe9f-0579-40ca-b658-48f1917f198a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:01.654746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:01.655671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:01.656581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:01.657052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] PATH_INFO: `/v3/users/81da2270eef945508fd99c0761f27cb5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:01.659338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=81da2270eef945508fd99c0761f27cb5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:01.663533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:01.752501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81da2270eef945508fd99c0761f27cb5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:01.765185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:01.788185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81da2270eef945508fd99c0761f27cb5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:01.800733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81da2270eef945508fd99c0761f27cb5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:01.801473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81da2270eef945508fd99c0761f27cb5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:01.802193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81da2270eef945508fd99c0761f27cb5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:01.817635 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1755/3490] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:01 2022] DELETE /identity/v3/users/81da2270eef945508fd99c0761f27cb5 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:02.258925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:02.275569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87115e-a012-4234-8ea8-f6cd814097c1 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:02.356509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:02.408164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:02.409205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:02.409715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:02.410141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] PATH_INFO: `/v3/projects/6e147bacb13f42c9867f0c28daa93b47` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:02.412777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e147bacb13f42c9867f0c28daa93b47)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:02.415816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:02.527326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:02.529396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:02.529993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e147bacb13f42c9867f0c28daa93b47', 'request_id': 'req-aa1f0a66-77e3-4550-a5a1-c3ead906da06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:02.552466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e147bacb13f42c9867f0c28daa93b47', 'request_id': 'req-aa1f0a66-77e3-4550-a5a1-c3ead906da06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:02.609948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e147bacb13f42c9867f0c28daa93b47', 'request_id': 'req-aa1f0a66-77e3-4550-a5a1-c3ead906da06'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:02.628025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adec1e-217d-4f00-ad92-d3f9ecc9d7ba admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:02.657137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] The token cache is being invalidate because project 6e147bacb13f42c9867f0c28daa93b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:02.657448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:02.661025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa1f0a66-77e3-4550-a5a1-c3ead906da06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:02.666783 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1736/3491] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:02 2022] DELETE /identity/v3/projects/6e147bacb13f42c9867f0c28daa93b47 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:02.672249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3d06-2d59-476f-b7a3-18de40f92c5e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:02.675481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3d06-2d59-476f-b7a3-18de40f92c5e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:02.675947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3d06-2d59-476f-b7a3-18de40f92c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:02.751455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-427e3d06-2d59-476f-b7a3-18de40f92c5e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:02.753920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-427e3d06-2d59-476f-b7a3-18de40f92c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:02.842271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:02.843412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:02.843877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:02.844309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:02.847088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:02.881516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1737/3492] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:02.885675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af3778-e2fe-4b38-b552-093d888b55e1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:02.886035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af3778-e2fe-4b38-b552-093d888b55e1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:02.886370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af3778-e2fe-4b38-b552-093d888b55e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:02.928621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b8af3778-e2fe-4b38-b552-093d888b55e1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:02.930326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b8af3778-e2fe-4b38-b552-093d888b55e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:02.940746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1738/3493] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:02.949681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af3778-e2fe-4b38-b552-093d888b55e1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:02.963101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af3778-e2fe-4b38-b552-093d888b55e1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:02.972755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:02.972755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:02.973086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:02.973414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:02.974242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:02.974595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:02.976999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:02.992668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:02.993861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1739/3494] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:02.994507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:02.998091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:03.017394 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1756/3495] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:03.030421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869c61-5eee-4e8d-9a9a-6657564d73eb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:03.044087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:03.044737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:03.045022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:03.046003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:03.050279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:03.054221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:03.142473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1740/3496] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:02 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:03.146450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:03.151815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:03.163438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715870c4-4367-47b4-b8a0-de621aa1de91 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:03.269419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:03.270250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:03.271057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:03.271057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:03.271522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:03.273699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:03.309184 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1757/3497] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:03 2022] POST /identity/v3/users => generated 283 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:03.313630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:03.333908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ae349-469e-48b1-97ff-e8f7827b8911 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:03.348389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:03.348389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:03.348389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:03.348389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:03.348905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:03.348905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:03.381292 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1741/3498] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:03.386345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:03.428187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049873f0-d507-4bd4-8ba2-f19a1211080d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:03.454497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cd994-05ee-43b9-ac0d-2e52086b4bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:03.455159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cd994-05ee-43b9-ac0d-2e52086b4bad admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:03.455429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cd994-05ee-43b9-ac0d-2e52086b4bad admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:03.455699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cd994-05ee-43b9-ac0d-2e52086b4bad admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:03.456398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cd994-05ee-43b9-ac0d-2e52086b4bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:03.458236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cd994-05ee-43b9-ac0d-2e52086b4bad admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:03.489570 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1758/3499] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:03.490454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:03.539769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ede58-dc11-4ee6-8c2b-0c567a689f54 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:03.552131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:03.552833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:03.555094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:03.555616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] PATH_INFO: `/v3/projects/6ad3935ef1a84c1698b0c5e55e237b40/users/1a71c359c22d4ad5a234a315f0383802/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:03.574689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:03.619754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ad3935ef1a84c1698b0c5e55e237b40, user_id=1a71c359c22d4ad5a234a315f0383802, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:03.622774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:03.698613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1742/3500] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:03 2022] PUT /identity/v3/projects/6ad3935ef1a84c1698b0c5e55e237b40/users/1a71c359c22d4ad5a234a315f0383802/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:03.712610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7a383-c74d-4d46-abee-e22d597ada33 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:03.713642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7a383-c74d-4d46-abee-e22d597ada33 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:03.714435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7a383-c74d-4d46-abee-e22d597ada33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:03.781803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2ac7a383-c74d-4d46-abee-e22d597ada33 None None] MFA Rules not processed for user `1a71c359c22d4ad5a234a315f0383802`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:03.783195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2ac7a383-c74d-4d46-abee-e22d597ada33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:03.901660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1759/3501] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:04.228665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:04.230514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:04.360237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfd837-ac27-4647-bd3e-a82111e6d296 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:04.370202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e662a-9b4c-43c3-a4da-3863aab282fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:04.374299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e662a-9b4c-43c3-a4da-3863aab282fe service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:04.374588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e662a-9b4c-43c3-a4da-3863aab282fe service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:04.374875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e662a-9b4c-43c3-a4da-3863aab282fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:04.388017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e662a-9b4c-43c3-a4da-3863aab282fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:04.390451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e662a-9b4c-43c3-a4da-3863aab282fe service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:04.412191 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1743/3502] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:04.837052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7a383-c74d-4d46-abee-e22d597ada33 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:05.114590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7a383-c74d-4d46-abee-e22d597ada33 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:05.123774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:05.124847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:05.125958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:05.126420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] PATH_INFO: `/v3/users/1a71c359c22d4ad5a234a315f0383802` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:05.128213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a71c359c22d4ad5a234a315f0383802)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:05.130647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:05.150939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:05.220492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a71c359c22d4ad5a234a315f0383802'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:05.258411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a71c359c22d4ad5a234a315f0383802'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:05.275202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a71c359c22d4ad5a234a315f0383802'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:05.275921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a71c359c22d4ad5a234a315f0383802'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:05.276692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a71c359c22d4ad5a234a315f0383802'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:05.300130 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1760/3503] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:04 2022] DELETE /identity/v3/users/1a71c359c22d4ad5a234a315f0383802 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:05.304450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:05.304794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:05.305073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:05.334107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:05.339100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:05.523956 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1744/3504] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:05.549318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:05.551304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:05.667157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efb520-577e-464a-ae4b-1d7542c18e82 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:05.691099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7296-501f-4a0e-85f1-8a1751ce04d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:05.692091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7296-501f-4a0e-85f1-8a1751ce04d8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:05.692681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7296-501f-4a0e-85f1-8a1751ce04d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:05.693479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7296-501f-4a0e-85f1-8a1751ce04d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:05.714702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7296-501f-4a0e-85f1-8a1751ce04d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:05.718010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7296-501f-4a0e-85f1-8a1751ce04d8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:05.750669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1761/3505] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:06.286738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:06.289800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:06.344252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae6da9-f13b-44f7-9912-804a64bf58fa None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:06.356186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:06.356852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:06.357136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:06.361054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] PATH_INFO: `/v3/projects/6ad3935ef1a84c1698b0c5e55e237b40` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:06.363609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ad3935ef1a84c1698b0c5e55e237b40)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:06.370170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:06.479526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ad3935ef1a84c1698b0c5e55e237b40', 'request_id': 'req-1f07b2ff-854d-422a-89d2-780d86f25aa3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:06.491174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ad3935ef1a84c1698b0c5e55e237b40', 'request_id': 'req-1f07b2ff-854d-422a-89d2-780d86f25aa3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:06.511591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ad3935ef1a84c1698b0c5e55e237b40', 'request_id': 'req-1f07b2ff-854d-422a-89d2-780d86f25aa3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:06.521556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:06.545921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] The token cache is being invalidate because project 6ad3935ef1a84c1698b0c5e55e237b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:06.547152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:06.548116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1f07b2ff-854d-422a-89d2-780d86f25aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:06.554033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1745/3506] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:06 2022] DELETE /identity/v3/projects/6ad3935ef1a84c1698b0c5e55e237b40 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:06.556337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a5e43-d3c6-4ebd-a355-7b1b248c9f29 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:06.557447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a5e43-d3c6-4ebd-a355-7b1b248c9f29 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:06.557974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a5e43-d3c6-4ebd-a355-7b1b248c9f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:06.588784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-832a5e43-d3c6-4ebd-a355-7b1b248c9f29 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:06.590605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-832a5e43-d3c6-4ebd-a355-7b1b248c9f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:06.713955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1762/3507] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:06.715365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f87f-5f92-405a-9928-a00a255a99d1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:06.715650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f87f-5f92-405a-9928-a00a255a99d1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:06.715922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f87f-5f92-405a-9928-a00a255a99d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:06.751695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9660f87f-5f92-405a-9928-a00a255a99d1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:06.752846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9660f87f-5f92-405a-9928-a00a255a99d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:06.777069 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1746/3508] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:06.783413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a5e43-d3c6-4ebd-a355-7b1b248c9f29 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:06.802785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a5e43-d3c6-4ebd-a355-7b1b248c9f29 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:06.813495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:06.814605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:06.817688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:06.818094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:06.818894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:06.820421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:06.824498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:06.850639 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1763/3509] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:06.855368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f87f-5f92-405a-9928-a00a255a99d1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:06.867534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f87f-5f92-405a-9928-a00a255a99d1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:06.882100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:06.882862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:06.883102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:06.883376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:06.884117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:06.886462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:06.965409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:06.966678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1747/3510] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:06 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:06.982143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06ff64-b05b-40cf-95a3-b8cc94a13ccf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:07.017167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:07.108757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:07.111837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.112476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.113228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.115084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:07.141454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:07.206240 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1764/3511] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:06 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:07.211315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:07.225204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a4e55-fdaf-4666-b05e-bacbece9b211 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:07.242414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:07.243059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.243334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.243607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.244270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:07.245922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:07.267593 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1748/3512] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:07.271289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:07.282092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060008f0-ea6f-44fd-8e2e-131d48201a60 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:07.293032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9405ee-2201-439e-93bd-72f232c1c7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:07.293792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9405ee-2201-439e-93bd-72f232c1c7dd admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.294066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9405ee-2201-439e-93bd-72f232c1c7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.294344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9405ee-2201-439e-93bd-72f232c1c7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.294998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9405ee-2201-439e-93bd-72f232c1c7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:07.296626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9405ee-2201-439e-93bd-72f232c1c7dd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:07.309910 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1765/3513] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:07.315315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:07.328130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30804135-0816-4ad4-96d1-96b6ca388f6a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:07.334280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:07.335079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.335457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.335930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] PATH_INFO: `/v3/projects/79c766c4e65e4af69d9f096d5d5c73be/users/5efa7e46d0004e72abf5b53994dfcfb9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.375104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c766c4e65e4af69d9f096d5d5c73be, user_id=5efa7e46d0004e72abf5b53994dfcfb9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:07.378013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:07.426770 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1749/3514] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:07 2022] PUT /identity/v3/projects/79c766c4e65e4af69d9f096d5d5c73be/users/5efa7e46d0004e72abf5b53994dfcfb9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:07.431492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432f706-e94f-446d-8f88-99e7be00e76c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.432472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432f706-e94f-446d-8f88-99e7be00e76c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.432858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432f706-e94f-446d-8f88-99e7be00e76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.503794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c432f706-e94f-446d-8f88-99e7be00e76c None None] MFA Rules not processed for user `5efa7e46d0004e72abf5b53994dfcfb9`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:07.506956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c432f706-e94f-446d-8f88-99e7be00e76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:07.608926 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1766/3515] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:07.620754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:07.623695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:07.741987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8b1a6-7018-4747-be88-c96f42774e35 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:07.748685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:07.749603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.750027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.750632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.763442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:07.765318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:07.782225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1750/3516] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:07.795924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e060d-ad84-4e4e-8d92-761073c36339 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.798720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e060d-ad84-4e4e-8d92-761073c36339 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.798720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e060d-ad84-4e4e-8d92-761073c36339 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.799412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1767/3517] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:07.811094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 21:02:07.835617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:07.853876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895dca8-d32d-4955-84a9-2b240b17b292 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:07.863736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5efa7e46d0004e72abf5b53994dfcfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79c766c4e65e4af69d9f096d5d5c73be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:07.863736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.863736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.864175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] PATH_INFO: `/v3/projects/79c766c4e65e4af69d9f096d5d5c73be` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.864665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorizing `identity:get_project(project_id=79c766c4e65e4af69d9f096d5d5c73be)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:07.871298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:07.873364 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1751/3518] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 21:02:07 2022] GET /identity/v3/projects/79c766c4e65e4af69d9f096d5d5c73be => generated 365 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:07.907014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f558faf-13d9-4e66-9ccf-a9f2b627410f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.907419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f558faf-13d9-4e66-9ccf-a9f2b627410f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.907799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f558faf-13d9-4e66-9ccf-a9f2b627410f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.909513 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1768/3519] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:07.924698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 21:02:07.944613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:07.955579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6a238-2c2f-4ad0-a4d5-18bf051ba3c7 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:07.963407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5efa7e46d0004e72abf5b53994dfcfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79c766c4e65e4af69d9f096d5d5c73be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:07.965935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:07.966355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:07.966792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] PATH_INFO: `/v3/projects/79c766c4e65e4af69d9f096d5d5c73be` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:07.968364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorizing `identity:get_project(project_id=79c766c4e65e4af69d9f096d5d5c73be)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:07.970448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:07.986169 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1752/3520] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 21:02:07 2022] GET /identity/v3/projects/79c766c4e65e4af69d9f096d5d5c73be => generated 365 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:08.019949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3db6-b85c-46af-995f-150823236291 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.020406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3db6-b85c-46af-995f-150823236291 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.020792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3db6-b85c-46af-995f-150823236291 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.022918 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1769/3521] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:08.030978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 21:02:08.048723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:08.063588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55161b5-c8b8-4f62-bd44-9ce3bc15be01 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:08.074582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5efa7e46d0004e72abf5b53994dfcfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79c766c4e65e4af69d9f096d5d5c73be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:08.075588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.076017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.076431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] PATH_INFO: `/v3/projects/79c766c4e65e4af69d9f096d5d5c73be` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.078099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorizing `identity:get_project(project_id=79c766c4e65e4af69d9f096d5d5c73be)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:08.080232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:08.082715 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1753/3522] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 21:02:08 2022] GET /identity/v3/projects/79c766c4e65e4af69d9f096d5d5c73be => generated 365 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:08.139972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75cdf2-23a4-4024-86b0-1c2742044ec9 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.141524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75cdf2-23a4-4024-86b0-1c2742044ec9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.142481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75cdf2-23a4-4024-86b0-1c2742044ec9 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.148806 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1770/3523] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:02:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:08.149689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 21:02:08.168102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:08.188662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476b94e-3b10-4d11-8796-d6491120eaf6 tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:08.206716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871301e-cb7e-402c-9718-dbcac89b5ceb tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5efa7e46d0004e72abf5b53994dfcfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79c766c4e65e4af69d9f096d5d5c73be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:08.207061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871301e-cb7e-402c-9718-dbcac89b5ceb tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.207343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871301e-cb7e-402c-9718-dbcac89b5ceb tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.207617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871301e-cb7e-402c-9718-dbcac89b5ceb tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] PATH_INFO: `/v3/projects/79c766c4e65e4af69d9f096d5d5c73be` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.209307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871301e-cb7e-402c-9718-dbcac89b5ceb tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorizing `identity:get_project(project_id=79c766c4e65e4af69d9f096d5d5c73be)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:08.211290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871301e-cb7e-402c-9718-dbcac89b5ceb tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:08.213127 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1754/3524] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 21:02:08 2022] GET /identity/v3/projects/79c766c4e65e4af69d9f096d5d5c73be => generated 365 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:08.432381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6eb80-1570-4c4d-b331-4c7fcd116406 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.432676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6eb80-1570-4c4d-b331-4c7fcd116406 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.432958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6eb80-1570-4c4d-b331-4c7fcd116406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.434994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fecb22-05ec-4c58-8f4b-51300ef0052c None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.435277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fecb22-05ec-4c58-8f4b-51300ef0052c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.435534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fecb22-05ec-4c58-8f4b-51300ef0052c None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.439491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1755/3525] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:02:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:08.469760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2ce6eb80-1570-4c4d-b331-4c7fcd116406 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:08.470871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2ce6eb80-1570-4c4d-b331-4c7fcd116406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:08.473848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fecb22-05ec-4c58-8f4b-51300ef0052c None None] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 21:02:08.475698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-99fecb22-05ec-4c58-8f4b-51300ef0052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:08.507622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fecb22-05ec-4c58-8f4b-51300ef0052c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:08.530699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fecb22-05ec-4c58-8f4b-51300ef0052c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:08.538855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5efa7e46d0004e72abf5b53994dfcfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79c766c4e65e4af69d9f096d5d5c73be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:08.542049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.542350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.543701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] PATH_INFO: `/v3/projects/79c766c4e65e4af69d9f096d5d5c73be` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.545322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorizing `identity:get_project(project_id=79c766c4e65e4af69d9f096d5d5c73be)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:08.547393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:08.553413 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1756/3526] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 21:02:08 2022] GET /identity/v3/projects/79c766c4e65e4af69d9f096d5d5c73be => generated 365 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:02:08.631563 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1771/3527] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:08.666827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:08.772923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:08.863893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e1230-a58d-4c3d-8c49-88aa8843554d tempest-QuotasTestJSON-1488837861 tempest-QuotasTestJSON-1488837861-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:08.877107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:08.877852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:08.878128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:08.878627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] PATH_INFO: `/v3/users/5efa7e46d0004e72abf5b53994dfcfb9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:08.880246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5efa7e46d0004e72abf5b53994dfcfb9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:08.892314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:08.999345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5efa7e46d0004e72abf5b53994dfcfb9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:09.000391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5efa7e46d0004e72abf5b53994dfcfb9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:09.031064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5efa7e46d0004e72abf5b53994dfcfb9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:09.032183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5efa7e46d0004e72abf5b53994dfcfb9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:09.054565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5efa7e46d0004e72abf5b53994dfcfb9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:09.072757 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1757/3528] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:08 2022] DELETE /identity/v3/users/5efa7e46d0004e72abf5b53994dfcfb9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:09.077668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13cd5e-057a-40e5-9245-5407129988be None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:09.078298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13cd5e-057a-40e5-9245-5407129988be None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:09.078790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13cd5e-057a-40e5-9245-5407129988be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:09.113524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bd13cd5e-057a-40e5-9245-5407129988be None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:09.127276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bd13cd5e-057a-40e5-9245-5407129988be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:09.192234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:09.281127 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1772/3529] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:09.310768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:09.312394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:09.423373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1390df-4b3d-4f56-8385-76952524c9f1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:09.424005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1390df-4b3d-4f56-8385-76952524c9f1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:09.424451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1390df-4b3d-4f56-8385-76952524c9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:09.464888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8e1390df-4b3d-4f56-8385-76952524c9f1 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:09.466556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8e1390df-4b3d-4f56-8385-76952524c9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:09.494773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfd78-9f5b-4d32-9c7a-e01cb42ca5e2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:09.515253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:09.516014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:09.516297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:09.516576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:09.542780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:09.544283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:09.566435 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1758/3530] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:09.571373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1773/3531] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:11.081275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:11.082881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:11.109132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa3383-1f79-4adf-b838-19ffe19a63bd service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:11.120224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:11.121268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:11.121794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:11.122259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] PATH_INFO: `/v3/projects/79c766c4e65e4af69d9f096d5d5c73be` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:11.124442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c766c4e65e4af69d9f096d5d5c73be)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:11.127302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:11.216956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:11.219414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c766c4e65e4af69d9f096d5d5c73be', 'request_id': 'req-848121b4-8d2f-402e-ae00-3e9a73aea1de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:11.231079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c766c4e65e4af69d9f096d5d5c73be', 'request_id': 'req-848121b4-8d2f-402e-ae00-3e9a73aea1de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:11.246312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c766c4e65e4af69d9f096d5d5c73be', 'request_id': 'req-848121b4-8d2f-402e-ae00-3e9a73aea1de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:11.268783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] The token cache is being invalidate because project 79c766c4e65e4af69d9f096d5d5c73be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:11.269270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:11.270316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-848121b4-8d2f-402e-ae00-3e9a73aea1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:11.273401 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1759/3532] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:11 2022] DELETE /identity/v3/projects/79c766c4e65e4af69d9f096d5d5c73be => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:11.286576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc553cf-655a-4452-af30-82ec68f056d6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:11.287431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc553cf-655a-4452-af30-82ec68f056d6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:11.287862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc553cf-655a-4452-af30-82ec68f056d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:11.345572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6dc553cf-655a-4452-af30-82ec68f056d6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:11.349181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6dc553cf-655a-4452-af30-82ec68f056d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:11.515035 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1774/3533] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:11.523277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2bf3c-4e29-4571-a9e4-b5d2fb0200a4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:11.523709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2bf3c-4e29-4571-a9e4-b5d2fb0200a4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:11.523993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2bf3c-4e29-4571-a9e4-b5d2fb0200a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:11.557302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a8b2bf3c-4e29-4571-a9e4-b5d2fb0200a4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:11.558871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a8b2bf3c-4e29-4571-a9e4-b5d2fb0200a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:11.570781 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1760/3534] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:11.574785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc553cf-655a-4452-af30-82ec68f056d6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:11.589128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc553cf-655a-4452-af30-82ec68f056d6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:11.619176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:11.620286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:11.620644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:11.620983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:11.622000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:11.622369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:11.626291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:11.660943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2bf3c-4e29-4571-a9e4-b5d2fb0200a4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:11.670797 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1775/3535] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:11.686286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2bf3c-4e29-4571-a9e4-b5d2fb0200a4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:11.696304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:11.696955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:11.697230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:11.697523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:11.698192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:11.702493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:11.767809 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1761/3536] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:11 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:11.775211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:11.786426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95babc45-f47c-4b15-8167-f7641422db12 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:11.899761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:11.900409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:11.900684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:11.900961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:11.901618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:11.903613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:11.926515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:11.940345 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1776/3537] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:11 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:11.944858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:11.972918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466d733-cbc3-4e66-b3dc-b41701f7ca8d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:11.981073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:11.981975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:11.982816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:11.982816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:11.985010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:11.986873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:12.007789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1762/3538] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:12.012935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:12.037018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77507b-e462-429b-beca-15c898d0844d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:12.057354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6da086-5fff-4f39-98d0-a11a51614fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:12.061541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da086-5fff-4f39-98d0-a11a51614fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:12.062235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da086-5fff-4f39-98d0-a11a51614fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:12.062720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6da086-5fff-4f39-98d0-a11a51614fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:12.063882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6da086-5fff-4f39-98d0-a11a51614fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:12.072072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6da086-5fff-4f39-98d0-a11a51614fd6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:12.101641 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1777/3539] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:12.106469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:12.122312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866a156-33a2-469e-92e3-a086d07ef697 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:12.136467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:12.137683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:12.139681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:12.140424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] PATH_INFO: `/v3/projects/2b777025c44a4535bea435d55f8cb1e3/users/fe663342df9f495d8203d8bb817868ec/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:12.201153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b777025c44a4535bea435d55f8cb1e3, user_id=fe663342df9f495d8203d8bb817868ec, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:12.205689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:12.243103 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1763/3540] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:12 2022] PUT /identity/v3/projects/2b777025c44a4535bea435d55f8cb1e3/users/fe663342df9f495d8203d8bb817868ec/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:12.256038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:12.256331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:12.256605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:12.308723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] MFA Rules not processed for user `fe663342df9f495d8203d8bb817868ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:12.310104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:12.442500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1778/3541] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:12.455045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:12.457095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:12.644801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522c4b1-73e5-497e-9cad-bd058448321c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:12.655115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:12.655782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:12.656143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:12.656417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:12.675602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:12.677311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:12.702868 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1764/3542] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:13.856874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:13.858480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:13.985995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0db85-509c-4359-9b38-a6eeb4e60f0f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:13.996739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a777788-5991-454f-9a80-87b97f69a4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:13.997437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a777788-5991-454f-9a80-87b97f69a4aa service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:13.997756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a777788-5991-454f-9a80-87b97f69a4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:13.998134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a777788-5991-454f-9a80-87b97f69a4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:14.007386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2a777788-5991-454f-9a80-87b97f69a4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:14.143228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a777788-5991-454f-9a80-87b97f69a4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:14.144726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a777788-5991-454f-9a80-87b97f69a4aa service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:14.170491 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1779/3543] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:02:16.354355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:16.370866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86d08-a793-44bd-8c9e-d03e708378be service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:16.382991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:16.384100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.384443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.385149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] PATH_INFO: `/v3/users/fe663342df9f495d8203d8bb817868ec` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.387080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe663342df9f495d8203d8bb817868ec)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:16.389167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:16.484628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe663342df9f495d8203d8bb817868ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:16.485305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe663342df9f495d8203d8bb817868ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:16.491391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:16.502297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe663342df9f495d8203d8bb817868ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:16.504184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe663342df9f495d8203d8bb817868ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:16.526923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe663342df9f495d8203d8bb817868ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:16.566904 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1765/3544] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:16 2022] DELETE /identity/v3/users/fe663342df9f495d8203d8bb817868ec => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:16.571555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633961e-b9bc-472a-9947-8469f69a94c6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.572400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633961e-b9bc-472a-9947-8469f69a94c6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.572972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633961e-b9bc-472a-9947-8469f69a94c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.610482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d633961e-b9bc-472a-9947-8469f69a94c6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:16.612854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d633961e-b9bc-472a-9947-8469f69a94c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:16.653391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:16.729945 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1780/3545] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:16.754041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:16.756318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:16.953074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.954748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.955282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.976581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb0e07-f09c-41d8-949a-e750adcaf8a7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:16.991500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:16.992271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.992714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.992844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:17.003851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:17.006882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:17.047494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:17.054508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:17.067050 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1766/3546] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:17.207501 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1781/3547] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:17.232408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:17.234096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:17.335376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9226a2d-48a9-4e79-bc96-7ee1190ede0e service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:17.344810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:17.344810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:17.344810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:17.344810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:17.360150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:17.362011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:17.373295 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1767/3548] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:02:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:02:18.246429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:18.248819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:18.276834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777cf49-e05b-4526-8845-fe54d14e677f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:18.291696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:18.293268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:18.293579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:18.293855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] PATH_INFO: `/v3/users/22306a93c9404db393b92a267d6258d4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:18.298691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22306a93c9404db393b92a267d6258d4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:18.300997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:18.421192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22306a93c9404db393b92a267d6258d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:18.456064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22306a93c9404db393b92a267d6258d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:18.471526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22306a93c9404db393b92a267d6258d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:18.472153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22306a93c9404db393b92a267d6258d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:18.472781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4aad87da-ff9f-41cd-9ad0-72700be6a0f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22306a93c9404db393b92a267d6258d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:18.499546 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1782/3549] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:18 2022] DELETE /identity/v3/users/22306a93c9404db393b92a267d6258d4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:18.664300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:18.666066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:18.694006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4346dfe-b15e-4c94-be33-472f877e6530 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:18.886788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:18.887463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:18.887857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:18.889818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] PATH_INFO: `/v3/projects/2b777025c44a4535bea435d55f8cb1e3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:18.891177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b777025c44a4535bea435d55f8cb1e3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:18.895173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:18.907018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:19.028528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b777025c44a4535bea435d55f8cb1e3', 'request_id': 'req-6445173a-b60b-4ec0-9d3a-e5916338f4fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.038260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b777025c44a4535bea435d55f8cb1e3', 'request_id': 'req-6445173a-b60b-4ec0-9d3a-e5916338f4fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.054115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b777025c44a4535bea435d55f8cb1e3', 'request_id': 'req-6445173a-b60b-4ec0-9d3a-e5916338f4fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.080338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] The token cache is being invalidate because project 2b777025c44a4535bea435d55f8cb1e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:19.080714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.081933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6445173a-b60b-4ec0-9d3a-e5916338f4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.085908 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1768/3550] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:18 2022] DELETE /identity/v3/projects/2b777025c44a4535bea435d55f8cb1e3 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:19.092153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.092435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.092712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.160866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:19.166148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:19.283690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:19.360732 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1783/3551] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:19.368651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91242a-40ca-4b51-a5ed-1a61b2eee298 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.369048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91242a-40ca-4b51-a5ed-1a61b2eee298 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.369437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91242a-40ca-4b51-a5ed-1a61b2eee298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.402290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.403942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:19.405954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1e91242a-40ca-4b51-a5ed-1a61b2eee298 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:19.408118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1e91242a-40ca-4b51-a5ed-1a61b2eee298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:19.417635 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1769/3552] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:19.421916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91242a-40ca-4b51-a5ed-1a61b2eee298 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.429912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb65a9-8e82-4f20-87ad-a0a0f9adc614 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:19.435363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91242a-40ca-4b51-a5ed-1a61b2eee298 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:19.441844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:19.442670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.443073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.443450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.443893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:19.444293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:19.444681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:19.444796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.445059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.445335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] PATH_INFO: `/v3/projects/004f77fba4b14080bca4ae36c857add3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.448822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:19.450434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004f77fba4b14080bca4ae36c857add3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:19.452842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:19.480979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1770/3553] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:02:19.485847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.498174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f56ddf-08e6-49ab-9bb0-b622d521f38f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:19.506231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:19.507328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.507881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.508505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.510232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:19.513423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:19.570012 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1771/3554] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:19 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:19.575503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.584044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004f77fba4b14080bca4ae36c857add3', 'request_id': 'req-a40ef689-6730-4871-afa8-16f8e1ecbd18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.621517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004f77fba4b14080bca4ae36c857add3', 'request_id': 'req-a40ef689-6730-4871-afa8-16f8e1ecbd18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.632439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004f77fba4b14080bca4ae36c857add3', 'request_id': 'req-a40ef689-6730-4871-afa8-16f8e1ecbd18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.692151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] The token cache is being invalidate because project 004f77fba4b14080bca4ae36c857add3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:19.694034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.697602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a40ef689-6730-4871-afa8-16f8e1ecbd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:19.703106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1784/3555] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:19 2022] DELETE /identity/v3/projects/004f77fba4b14080bca4ae36c857add3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:19.714815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d1ea2-ca88-4b99-bb23-b49c6b9d7f5b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.715934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d1ea2-ca88-4b99-bb23-b49c6b9d7f5b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.716401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d1ea2-ca88-4b99-bb23-b49c6b9d7f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.736848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aae086-8085-44d2-8884-be84d08af058 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:19.742949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:19.743598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.743941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.744298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.745687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:19.748436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:19.793899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f17d1ea2-ca88-4b99-bb23-b49c6b9d7f5b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:19.803766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f17d1ea2-ca88-4b99-bb23-b49c6b9d7f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:19.809905 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1772/3556] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:19 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:19.813731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.817193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:19.839055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7c6a-403c-4075-aed8-e776f70ba687 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:19.842741 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1785/3557] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:19.844163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e4699-8aa2-4acd-832a-f901906bcaa1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.844449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e4699-8aa2-4acd-832a-f901906bcaa1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.844722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e4699-8aa2-4acd-832a-f901906bcaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.855474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:19.855474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.855474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.855474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.855963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:19.867104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:19.898203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-055e4699-8aa2-4acd-832a-f901906bcaa1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:19.901665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-055e4699-8aa2-4acd-832a-f901906bcaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:19.901838 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1773/3558] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:19.903253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.921974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f469db-54c0-4879-b171-5f1c00a6bb33 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:19.927589 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1786/3559] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:19.933557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e4699-8aa2-4acd-832a-f901906bcaa1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.934011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:19.935395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.936374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.937411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.940417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:19.944737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:19.952784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e4699-8aa2-4acd-832a-f901906bcaa1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:19.967292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:19.967922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:19.968197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:19.968459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:19.969114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:19.969411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:19.971053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:19.980526 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1774/3560] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:19.981985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:19.993768 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1787/3561] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:19.999874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.006972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396735e-3c6b-4b66-be5d-ab7444e693e3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.014254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f4485-f8df-4bfd-af46-e21090af376d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.019521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.020149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.020424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.020685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.021319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.022442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc584f4e-74fa-4b6a-9e69-e23a3fd0f3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.023559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.023678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584f4e-74fa-4b6a-9e69-e23a3fd0f3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.023916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584f4e-74fa-4b6a-9e69-e23a3fd0f3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.024195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc584f4e-74fa-4b6a-9e69-e23a3fd0f3ee admin admin] PATH_INFO: `/v3/projects/cca911041d3c4273b41e24522d8af674/users/4a4f71b79d8e45a28ef16b9e9b3b8795/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.058051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc584f4e-74fa-4b6a-9e69-e23a3fd0f3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca911041d3c4273b41e24522d8af674, user_id=4a4f71b79d8e45a28ef16b9e9b3b8795, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.060237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc584f4e-74fa-4b6a-9e69-e23a3fd0f3ee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.064752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:20.073648 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1788/3562] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:19 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:20.079877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.093080 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1775/3563] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:19 2022] PUT /identity/v3/projects/cca911041d3c4273b41e24522d8af674/users/4a4f71b79d8e45a28ef16b9e9b3b8795/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:20.113895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490b829-dd66-4d41-9126-9673d6c26f68 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.116801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490b829-dd66-4d41-9126-9673d6c26f68 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.118115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490b829-dd66-4d41-9126-9673d6c26f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.240180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1490b829-dd66-4d41-9126-9673d6c26f68 None None] MFA Rules not processed for user `4a4f71b79d8e45a28ef16b9e9b3b8795`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:20.241922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1490b829-dd66-4d41-9126-9673d6c26f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:20.256577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc638f-4015-4a28-bcbb-a7cd3f006107 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.345908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:20.407130 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1776/3564] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:20.412028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490b829-dd66-4d41-9126-9673d6c26f68 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.440875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.441332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.442423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.442706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.443597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.447958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.512299 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1789/3565] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:20 2022] POST /identity/v3/users => generated 286 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:20.516635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.540135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ce08a-67aa-4bc4-b748-f2fad58272e0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.550626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.551246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.551520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.552037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.552688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.555287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.581176 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1790/3566] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:20.584154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490b829-dd66-4d41-9126-9673d6c26f68 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.585999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.596534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.597424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.597730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.598001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.598812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.603559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.608804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0c9ac-a1b9-4517-99da-f5e23bafeff8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.618226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.618870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.619137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.619403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.623712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.625358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.642241 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1791/3567] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:20.646311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.660236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3d54d-38be-4dec-ade0-8b94fb43bb8d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.676234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7adae1-3df8-4194-9773-19d4981fc07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.676234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7adae1-3df8-4194-9773-19d4981fc07c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.676234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7adae1-3df8-4194-9773-19d4981fc07c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.676234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7adae1-3df8-4194-9773-19d4981fc07c admin admin] PATH_INFO: `/v3/projects/36f50dceeb7e46f0a1b3482f0c82d7c3/users/6ad16040f3e64b46beca7fc88d724173/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.688165 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1777/3568] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:20 2022] POST /identity/v3/projects => generated 375 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:20.689846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.706966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7adae1-3df8-4194-9773-19d4981fc07c admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f50dceeb7e46f0a1b3482f0c82d7c3, user_id=6ad16040f3e64b46beca7fc88d724173, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.709841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7adae1-3df8-4194-9773-19d4981fc07c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.737012 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1792/3569] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:20 2022] PUT /identity/v3/projects/36f50dceeb7e46f0a1b3482f0c82d7c3/users/6ad16040f3e64b46beca7fc88d724173/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:20.750364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.750848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.751217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.863553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] MFA Rules not processed for user `6ad16040f3e64b46beca7fc88d724173`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:20.865296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:20.873705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017aa84-0495-4fe2-b8a5-5cf55461dbb8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.904008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:21.032451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:21.033224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:21.033855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:21.034190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:21.035431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:21.038326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:21.084451 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1778/3570] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:20 2022] POST /identity/v3/users => generated 294 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:21.085860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:21.088292 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1793/3571] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:21.103060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:21.104866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:21.127659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d454a-5ed5-45c4-b521-dfd3f596cc0b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:21.141738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:21.142380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:21.143029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:21.143774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:21.144090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:21.149521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:21.184399 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1779/3572] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:02:21.185333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:21.234609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff197f-c0b4-44c0-8376-2b76f0e194e2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:21.271157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:21.272068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:21.272402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:21.272732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] PATH_INFO: `/v3/projects/9f39f15cbd2c4363a0446587cef825b0/users/e86a17c3956b407bbd7166fd1db1c010/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:21.289939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:21.296039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f39f15cbd2c4363a0446587cef825b0, user_id=e86a17c3956b407bbd7166fd1db1c010, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:21.300192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:21.353996 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1780/3573] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:21 2022] PUT /identity/v3/projects/9f39f15cbd2c4363a0446587cef825b0/users/e86a17c3956b407bbd7166fd1db1c010/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:21.357272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:21.405552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba989c-ea64-4e77-8a44-83431cf7cae9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:21.481016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595ec7-d9e6-4ee0-9346-860aebf6db2e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:21.488148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:21.489418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:21.489717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:21.490539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:21.492122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:21.495994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:21.513065 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1781/3574] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:21.516666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:21.549571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90906b0-f7ea-42fb-b2ed-acdf54141e2a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:21.557821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5888301-b7de-4a52-b7cf-4f77d8af0c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:21.558586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888301-b7de-4a52-b7cf-4f77d8af0c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:21.558924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888301-b7de-4a52-b7cf-4f77d8af0c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:21.559617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888301-b7de-4a52-b7cf-4f77d8af0c75 admin admin] PATH_INFO: `/v3/projects/9f39f15cbd2c4363a0446587cef825b0/users/e86a17c3956b407bbd7166fd1db1c010/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:21.561777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5888301-b7de-4a52-b7cf-4f77d8af0c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f39f15cbd2c4363a0446587cef825b0, user_id=e86a17c3956b407bbd7166fd1db1c010, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:21.564720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5888301-b7de-4a52-b7cf-4f77d8af0c75 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:21.581813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:21.583193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:21.583420 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1782/3575] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:21 2022] PUT /identity/v3/projects/9f39f15cbd2c4363a0446587cef825b0/users/e86a17c3956b407bbd7166fd1db1c010/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:21.583931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:21.584443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:21.678900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6d2bd-7a9e-4924-a7a1-1ea02793fbfa None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:21.679545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6d2bd-7a9e-4924-a7a1-1ea02793fbfa None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:21.680263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6d2bd-7a9e-4924-a7a1-1ea02793fbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:21.728670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:21.734785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:21.757747 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1794/3576] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:02:21.761418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6ed6d2bd-7a9e-4924-a7a1-1ea02793fbfa None None] MFA Rules not processed for user `e86a17c3956b407bbd7166fd1db1c010`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:21.762607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6ed6d2bd-7a9e-4924-a7a1-1ea02793fbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:21.885618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:21.976456 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1783/3577] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:21.981150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:22.096044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3aaf7-7876-4730-98dc-afa22f032e4d service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:22.108889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:22.108889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:22.108889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:22.109182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:22.109182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:22.109342 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:22.109342 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:02:22.109342 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:02:22.114657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:22.168881 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1795/3578] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:02:21 2022] POST /identity/v3/domains => generated 245 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:22.170325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6d2bd-7a9e-4924-a7a1-1ea02793fbfa None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:22.195297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6d2bd-7a9e-4924-a7a1-1ea02793fbfa None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:22.340401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:22.341101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:22.341397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:22.341711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:22.342366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:22.355436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:22.510169 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1784/3579] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:22 2022] POST /identity/v3/projects => generated 408 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:22.514456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:22.542057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83351143-3edd-4292-8267-2d395780a1dd tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:22.633068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:22.786371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:22.786934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:22.787232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:22.787537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:22.788407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:22.790476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:22.843449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:22.853975 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1796/3580] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:22 2022] POST /identity/v3/users => generated 348 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:22.856799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28006118-999c-4467-8fc8-52d8b506cb30 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:22.872856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:22.873546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:22.873889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:22.874216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:22.875862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:22.877528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:22.886923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:22.889284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:22.917789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1785/3581] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:22.925740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:22.958817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5356850-c61e-4a07-991b-0f1fd2f0cd5c tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:22.976093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46120d5-e7f1-4e37-bc21-5c2a4bb3e5e3 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:22.977231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46120d5-e7f1-4e37-bc21-5c2a4bb3e5e3 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:22.977231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46120d5-e7f1-4e37-bc21-5c2a4bb3e5e3 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:22.977397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46120d5-e7f1-4e37-bc21-5c2a4bb3e5e3 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/projects/48b61561cbcf49c98a1dbf694e637a15/users/532db23d94fc49f38283b158fbbb7322/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.045972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46120d5-e7f1-4e37-bc21-5c2a4bb3e5e3 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:create_grant(project_id=48b61561cbcf49c98a1dbf694e637a15, user_id=532db23d94fc49f38283b158fbbb7322, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:23.049621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46120d5-e7f1-4e37-bc21-5c2a4bb3e5e3 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:23.095519 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1786/3582] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:22 2022] PUT /identity/v3/projects/48b61561cbcf49c98a1dbf694e637a15/users/532db23d94fc49f38283b158fbbb7322/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:23.100012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11d9ff-97ac-4ed6-b66e-30a8006089c2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:23.100299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11d9ff-97ac-4ed6-b66e-30a8006089c2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:23.100577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11d9ff-97ac-4ed6-b66e-30a8006089c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.102059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66188abc-0eef-4564-9d46-3409e5262bef tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:23.278187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:23.279827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:23.279827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:23.279827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] PATH_INFO: `/v3/users/3009c5feb1ef43a08b07996d8f63bf11` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.290370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3009c5feb1ef43a08b07996d8f63bf11)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:23.292501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:23.346364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:23.362513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0d11d9ff-97ac-4ed6-b66e-30a8006089c2 None None] MFA Rules not processed for user `532db23d94fc49f38283b158fbbb7322`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:23.366021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0d11d9ff-97ac-4ed6-b66e-30a8006089c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:23.406481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3009c5feb1ef43a08b07996d8f63bf11'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.414690 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1787/3583] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:23.424752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:23.425011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:23.425285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.449615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3009c5feb1ef43a08b07996d8f63bf11'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.454677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:23.481159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] MFA Rules not processed for user `532db23d94fc49f38283b158fbbb7322`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:23.481893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3009c5feb1ef43a08b07996d8f63bf11'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.482567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:23.482721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3009c5feb1ef43a08b07996d8f63bf11'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.483504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3009c5feb1ef43a08b07996d8f63bf11'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.501935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1788/3584] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:23.503085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:23.512203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:23.523103 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1797/3585] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:22 2022] DELETE /identity/v3/users/3009c5feb1ef43a08b07996d8f63bf11 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:23.523763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:23.553512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d722e88-6e20-4c55-aa7c-87ea2e9d8607 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:23.634367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:23.635462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:23.635785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:23.636061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] PATH_INFO: `/v3/users/6ad16040f3e64b46beca7fc88d724173` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.638217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad16040f3e64b46beca7fc88d724173)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:23.642126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:23.662722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b968b2e-7483-482e-a9de-dd2800f79eed admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:23.682339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '532db23d94fc49f38283b158fbbb7322', 'user_domain_id': '038526e9a74d47eda567b6029d166eba', 'system_scope': None, 'project_id': '48b61561cbcf49c98a1dbf694e637a15', 'project_domain_id': '038526e9a74d47eda567b6029d166eba', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:23.683124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:23.683497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:23.683860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.705737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:23.712987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:23.741941 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1798/3586] 10.222.0.26 () {56 vars in 1284 bytes} [Sat Sep 17 21:02:23 2022] GET /identity/v3/auth/tokens => generated 3360 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:23.742840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:23.744502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad16040f3e64b46beca7fc88d724173'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.745582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:23.746336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad16040f3e64b46beca7fc88d724173'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.764548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad16040f3e64b46beca7fc88d724173'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.765407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad16040f3e64b46beca7fc88d724173'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.783328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad16040f3e64b46beca7fc88d724173'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.796450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b15bb9-5a5f-475a-94bc-3f946f351483 tempest-test_project-686300386 tempest-user-1365851842] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:23.815013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:23.815891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:23.816178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:23.817610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] PATH_INFO: `/v3/users/da5d41f7e95d44d3898f6306bb093ed7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.820304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5d41f7e95d44d3898f6306bb093ed7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:23.821401 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1789/3587] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:23 2022] DELETE /identity/v3/users/6ad16040f3e64b46beca7fc88d724173 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:23.822644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:23.825116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:23.828568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:23.916118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5d41f7e95d44d3898f6306bb093ed7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.966432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5d41f7e95d44d3898f6306bb093ed7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.980975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5d41f7e95d44d3898f6306bb093ed7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.982128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5d41f7e95d44d3898f6306bb093ed7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.982923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1d2169c8-3df6-4088-a2f9-161bf4de6f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5d41f7e95d44d3898f6306bb093ed7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:23.983196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaabfd-7216-45ee-bae9-e0f7a8c2a3c0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:23.989833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:23.990502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:23.990781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:23.991309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/users/532db23d94fc49f38283b158fbbb7322` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:23.992695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:delete_user(user_id=532db23d94fc49f38283b158fbbb7322)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:23.994703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:23.997344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:24.008245 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1799/3588] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:23 2022] DELETE /identity/v3/users/da5d41f7e95d44d3898f6306bb093ed7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:24.010050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:24.010340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:24.010616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:24.046094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:24.047355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:24.081885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532db23d94fc49f38283b158fbbb7322'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.082673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532db23d94fc49f38283b158fbbb7322'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.102002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532db23d94fc49f38283b158fbbb7322'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.103623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532db23d94fc49f38283b158fbbb7322'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.138314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532db23d94fc49f38283b158fbbb7322'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.166651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:24.173636 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1790/3589] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:23 2022] DELETE /identity/v3/users/532db23d94fc49f38283b158fbbb7322 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:24.174769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:24.177798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:24.208884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1800/3590] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:24.212653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:24.216797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:24.421890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295298d-77b3-4b76-80b2-4c6404dbc3d1 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:24.429212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:24.430011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:24.430891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:24.430891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] PATH_INFO: `/v3/projects/89b46ae7347d4f2daf02f627e41a4847` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:24.432048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b46ae7347d4f2daf02f627e41a4847)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:24.435121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:24.438670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f837b6-0b6a-4127-91bc-83e8faea4e78 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:24.620345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b46ae7347d4f2daf02f627e41a4847', 'request_id': 'req-307d89c2-7874-48f0-b5ad-12a6464b514f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.637590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b46ae7347d4f2daf02f627e41a4847', 'request_id': 'req-307d89c2-7874-48f0-b5ad-12a6464b514f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.645099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:24.646245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:24.646480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:24.646803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/projects/48b61561cbcf49c98a1dbf694e637a15` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:24.648117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=48b61561cbcf49c98a1dbf694e637a15)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:24.650237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:24.653145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b46ae7347d4f2daf02f627e41a4847', 'request_id': 'req-307d89c2-7874-48f0-b5ad-12a6464b514f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.675372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] The token cache is being invalidate because project 89b46ae7347d4f2daf02f627e41a4847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:24.675886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.676685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.678926 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1791/3591] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:24 2022] DELETE /identity/v3/projects/89b46ae7347d4f2daf02f627e41a4847 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:24.680516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:24.681867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:24.714473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:24.722627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b61561cbcf49c98a1dbf694e637a15', 'request_id': 'req-d80a8901-62a7-4205-aaa4-ba074c30296e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.733265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b61561cbcf49c98a1dbf694e637a15', 'request_id': 'req-d80a8901-62a7-4205-aaa4-ba074c30296e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.740559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b61561cbcf49c98a1dbf694e637a15', 'request_id': 'req-d80a8901-62a7-4205-aaa4-ba074c30296e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.755673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] The token cache is being invalidate because project 48b61561cbcf49c98a1dbf694e637a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:24.756194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.757208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.760997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1801/3592] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:24 2022] DELETE /identity/v3/projects/48b61561cbcf49c98a1dbf694e637a15 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:24.761724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:24.765045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:24.789501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d89c2-7874-48f0-b5ad-12a6464b514f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:24.815028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:24.860332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8901-62a7-4205-aaa4-ba074c30296e tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:24.866089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:24.866780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:24.867062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:24.867334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] PATH_INFO: `/v3/projects/f8813aaee4604882b4cd7a2a8df58077` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:24.869090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8813aaee4604882b4cd7a2a8df58077)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:24.871336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:24.887559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7722bc3-fe11-4bf0-b565-be9a649b6ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:24.888959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7722bc3-fe11-4bf0-b565-be9a649b6ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:24.889415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7722bc3-fe11-4bf0-b565-be9a649b6ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:24.890072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7722bc3-fe11-4bf0-b565-be9a649b6ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:24.892614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7722bc3-fe11-4bf0-b565-be9a649b6ae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:24.912709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7722bc3-fe11-4bf0-b565-be9a649b6ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:24.914269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7722bc3-fe11-4bf0-b565-be9a649b6ae8 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:24.929830 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1792/3593] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:24.931886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:24.932173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:24.932695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:24.952888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8813aaee4604882b4cd7a2a8df58077', 'request_id': 'req-051763ad-8d6e-420c-8591-d27c71223844'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.964926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8813aaee4604882b4cd7a2a8df58077', 'request_id': 'req-051763ad-8d6e-420c-8591-d27c71223844'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:24.968759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] MFA Rules not processed for user `33fa2d463884464fbe90666c98da0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:24.970096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:24.993522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8813aaee4604882b4cd7a2a8df58077', 'request_id': 'req-051763ad-8d6e-420c-8591-d27c71223844'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.013115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] The token cache is being invalidate because project f8813aaee4604882b4cd7a2a8df58077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:25.013584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.014360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-051763ad-8d6e-420c-8591-d27c71223844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.018735 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1802/3594] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:24 2022] DELETE /identity/v3/projects/f8813aaee4604882b4cd7a2a8df58077 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:25.018735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:25.018735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:25.018735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:25.059184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:25.061904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:25.157111 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1803/3595] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:25.158111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:25.159560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:25.213450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1793/3596] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:25.217351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:25.220346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:25.271322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667e870-9e1d-4b8a-8617-1657798d6c74 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:25.283233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:25.284189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:25.284886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:25.285919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] PATH_INFO: `/v3/users/fdbe8a5b76e54af0b3cfc40a7ae5dcd2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:25.290944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdbe8a5b76e54af0b3cfc40a7ae5dcd2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:25.297336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:25.405137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:25.418970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd747d-7c23-46d6-8591-4ee1f67dff9a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:25.425606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:25.432597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:25.433181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:25.433714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/domains/038526e9a74d47eda567b6029d166eba` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:25.436789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=038526e9a74d47eda567b6029d166eba)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:25.439179 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:25.439179 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:02:25.439179 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:02:25.440602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:25.483901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdbe8a5b76e54af0b3cfc40a7ae5dcd2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.484893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdbe8a5b76e54af0b3cfc40a7ae5dcd2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.505271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdbe8a5b76e54af0b3cfc40a7ae5dcd2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.506283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdbe8a5b76e54af0b3cfc40a7ae5dcd2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.507506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '038526e9a74d47eda567b6029d166eba'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.508635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '038526e9a74d47eda567b6029d166eba'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.525233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:25.527396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b216d71-46bb-4b69-9499-c2910c5f9082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdbe8a5b76e54af0b3cfc40a7ae5dcd2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.546821 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1794/3597] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:25 2022] DELETE /identity/v3/users/fdbe8a5b76e54af0b3cfc40a7ae5dcd2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:25.548254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:25.548914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:25.549962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:25.583654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:25.584480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:25.598666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '038526e9a74d47eda567b6029d166eba'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.599770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '038526e9a74d47eda567b6029d166eba'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.616232 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1804/3598] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:02:25 2022] PATCH /identity/v3/domains/038526e9a74d47eda567b6029d166eba => generated 246 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:25.618808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:25.621076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:25.742971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35a08c-028b-4311-b20c-8b7bf4596996 tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:25.774573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1795/3599] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:25.775345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:25.778674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:25.847873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:25.849321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:25.849816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:25.850088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] PATH_INFO: `/v3/projects/36f50dceeb7e46f0a1b3482f0c82d7c3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:25.852528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f50dceeb7e46f0a1b3482f0c82d7c3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:25.854945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:25.886140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a7ffb-d2cc-4ba4-a17e-71f586fdf7ff None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:25.892611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e86a17c3956b407bbd7166fd1db1c010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f39f15cbd2c4363a0446587cef825b0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:25.893412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:25.893926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:25.894301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] PATH_INFO: `/v3/domains/038526e9a74d47eda567b6029d166eba` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:25.922654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=038526e9a74d47eda567b6029d166eba)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:25.926632 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:25.926632 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:02:25.926632 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:02:25.927839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:25.933448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f50dceeb7e46f0a1b3482f0c82d7c3', 'request_id': 'req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.955773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f50dceeb7e46f0a1b3482f0c82d7c3', 'request_id': 'req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.976259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f50dceeb7e46f0a1b3482f0c82d7c3', 'request_id': 'req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.993479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '038526e9a74d47eda567b6029d166eba'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:25.994602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:26.001895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '038526e9a74d47eda567b6029d166eba'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.015865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] The token cache is being invalidate because project 36f50dceeb7e46f0a1b3482f0c82d7c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:26.016231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.020475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.028915 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1805/3600] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:25 2022] DELETE /identity/v3/projects/36f50dceeb7e46f0a1b3482f0c82d7c3 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:26.028915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:26.028915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.112062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038526e9a74d47eda567b6029d166eba', 'request_id': 'req-1d6190c7-0183-4cfd-9722-49ff36760ebc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.121745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f333b75-20d1-4e9c-bcbd-71ad6dc62338 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:26.128674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038526e9a74d47eda567b6029d166eba', 'request_id': 'req-1d6190c7-0183-4cfd-9722-49ff36760ebc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.132730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:26.142503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038526e9a74d47eda567b6029d166eba', 'request_id': 'req-1d6190c7-0183-4cfd-9722-49ff36760ebc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.160207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] The token cache is being invalidate because project 038526e9a74d47eda567b6029d166eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:26.160684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.163784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.187323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '038526e9a74d47eda567b6029d166eba', 'request_id': 'req-1d6190c7-0183-4cfd-9722-49ff36760ebc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.192810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '038526e9a74d47eda567b6029d166eba', 'request_id': 'req-1d6190c7-0183-4cfd-9722-49ff36760ebc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.193795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '038526e9a74d47eda567b6029d166eba', 'request_id': 'req-1d6190c7-0183-4cfd-9722-49ff36760ebc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.200480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1796/3601] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:02:25 2022] DELETE /identity/v3/domains/038526e9a74d47eda567b6029d166eba => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:26.201207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:26.205495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.228142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a7ba1-98e3-45da-a3da-11cd4a4f1302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:26.229198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a7ba1-98e3-45da-a3da-11cd4a4f1302 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.229656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a7ba1-98e3-45da-a3da-11cd4a4f1302 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.230075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a7ba1-98e3-45da-a3da-11cd4a4f1302 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.231089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a7ba1-98e3-45da-a3da-11cd4a4f1302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:26.232099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a7ba1-98e3-45da-a3da-11cd4a4f1302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:26.234184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a7ba1-98e3-45da-a3da-11cd4a4f1302 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:26.258686 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1806/3602] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:26.261512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.262085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.262770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.287377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:26.289136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.299096 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1807/3603] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:26.301430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:26.303627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.318842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25eb96-70df-457f-a29f-609c390bfd8e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:26.325517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:26.326254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.326591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.326927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] PATH_INFO: `/v3/users/4a4f71b79d8e45a28ef16b9e9b3b8795` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.328567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4f71b79d8e45a28ef16b9e9b3b8795)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:26.331579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:26.346228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6190c7-0183-4cfd-9722-49ff36760ebc tempest-TestDefaultProjectId-465708194 tempest-TestDefaultProjectId-465708194-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:26.351122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:26.351774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.352057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.352331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] PATH_INFO: `/v3/projects/21cb941f09fb43ddb7d5e921bab41181` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.353525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] RBAC: Authorizing `identity:delete_project(project_id=21cb941f09fb43ddb7d5e921bab41181)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:26.355455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:26.419961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21cb941f09fb43ddb7d5e921bab41181', 'request_id': 'req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.420220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4f71b79d8e45a28ef16b9e9b3b8795'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.436776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21cb941f09fb43ddb7d5e921bab41181', 'request_id': 'req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.462646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4f71b79d8e45a28ef16b9e9b3b8795'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.475119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21cb941f09fb43ddb7d5e921bab41181', 'request_id': 'req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.489837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] The token cache is being invalidate because project 21cb941f09fb43ddb7d5e921bab41181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:26.490267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.491434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.493052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4f71b79d8e45a28ef16b9e9b3b8795'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.497726 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1797/3604] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:26 2022] DELETE /identity/v3/projects/21cb941f09fb43ddb7d5e921bab41181 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:26.498298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4f71b79d8e45a28ef16b9e9b3b8795'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.498776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:26.501320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.503740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4537224-c794-4bc9-9c66-c2ef33a32e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4f71b79d8e45a28ef16b9e9b3b8795'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:26.553105 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1808/3605] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:26 2022] DELETE /identity/v3/users/4a4f71b79d8e45a28ef16b9e9b3b8795 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:26.557394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888500-6e05-4d3e-9938-bf1609d44b27 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.557717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888500-6e05-4d3e-9938-bf1609d44b27 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.557997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce888500-6e05-4d3e-9938-bf1609d44b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.575409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:26.584280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ce888500-6e05-4d3e-9938-bf1609d44b27 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:26.585763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce888500-6e05-4d3e-9938-bf1609d44b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.615032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:26.628203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70a6b7-aa91-4027-a15c-e2b23ce1f71c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:26.712006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b1588-a204-4397-8458-44dda95d267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:26.714759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b1588-a204-4397-8458-44dda95d267d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.715115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b1588-a204-4397-8458-44dda95d267d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.715669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b1588-a204-4397-8458-44dda95d267d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.716468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b1588-a204-4397-8458-44dda95d267d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:26.718369 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1809/3606] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:26.721476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.722089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b1588-a204-4397-8458-44dda95d267d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:26.723446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.724540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.775083 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1798/3607] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:26 2022] POST /identity/v3/projects => generated 395 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:26.775083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9b66-d93d-4d28-946c-2edbc786e285 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.775083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9b66-d93d-4d28-946c-2edbc786e285 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.775083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9b66-d93d-4d28-946c-2edbc786e285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.780201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:26.783704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.802714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0ca9b66-d93d-4d28-946c-2edbc786e285 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:26.804547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0ca9b66-d93d-4d28-946c-2edbc786e285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.901861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1799/3608] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:26.903030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca9b66-d93d-4d28-946c-2edbc786e285 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:26.922700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca9b66-d93d-4d28-946c-2edbc786e285 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:26.927300 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1810/3609] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:26.928094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:26.928716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.928991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.929381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.930050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:26.930391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:26.932008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:26.937250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:26.939131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.951284 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1800/3610] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:26.951284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:26.953796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:26.955003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c84bbd-810f-4e4d-b21f-025ec2013de2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:26.960484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab21dd7-15ca-488b-9b8f-2e8c7066be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:26.966717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab21dd7-15ca-488b-9b8f-2e8c7066be96 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:26.966717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab21dd7-15ca-488b-9b8f-2e8c7066be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:26.966717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab21dd7-15ca-488b-9b8f-2e8c7066be96 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:26.966717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab21dd7-15ca-488b-9b8f-2e8c7066be96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:26.969034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab21dd7-15ca-488b-9b8f-2e8c7066be96 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.021555 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1811/3611] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:26 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:27.023251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c533be-1ec4-4243-957f-d379eea88324 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.023634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c533be-1ec4-4243-957f-d379eea88324 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.024011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c533be-1ec4-4243-957f-d379eea88324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.040305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9fac2-1837-44d1-93db-edceab1aa1a0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.044506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.046947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.047282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.047673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.056453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-49c533be-1ec4-4243-957f-d379eea88324 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:27.058019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49c533be-1ec4-4243-957f-d379eea88324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:27.060248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.062220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.073309 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1812/3612] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:27.074514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c533be-1ec4-4243-957f-d379eea88324 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.085113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c533be-1ec4-4243-957f-d379eea88324 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.090457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.091140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.091290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1801/3613] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:27.091559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.091900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.092600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.093929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.094814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.120523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505fd54-fdfc-499a-a04f-068d032c892e service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.128963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.130070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.130486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.131228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.131653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.133832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.143364 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1813/3614] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:27 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:27.144428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.151086 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1802/3615] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:27.152426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.198878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72896d9-d88c-4049-8dcd-dfbac05328ea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.208349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:27.291405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02984f3d-8f1b-4049-8232-33af2a5f80e6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.298039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.298869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.299809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.300197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.301582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:27.302200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.304128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.307619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:27.339513 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1814/3616] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:27.339728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.339728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.339728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.339728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.340142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.340236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.341034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.358850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ad34-003d-4be9-908b-804470c48d9b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.367100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.367866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.368284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.368900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.369746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.371960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.376299 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1803/3617] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:27 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:27.376299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.394524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517dbac-1cb5-41d1-afda-c10d51463ba3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.401380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.401380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.401380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.401380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.402237 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1815/3618] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:27.402410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.403162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.404406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.432134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0302a-734d-40e0-8b88-16438dc07d2f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.436951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.437590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.437870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.438143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.438790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.440481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.442895 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1804/3619] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:27 2022] POST /identity/v3/projects => generated 389 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:27.443709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.460583 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1816/3620] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:27.461632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.475917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e65c3-a03b-48be-9b38-6c453db03b58 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.625927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c263bc-71ee-4153-bab9-d38b16487256 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.638243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.640400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.641316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.642057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.642867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.643249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] PATH_INFO: `/v3/projects/985f1a6b63a1461ebf46b655825d90a3/users/742e9cf3a45c4c2c9a5a315122c8a6d5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.643618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.644617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.646741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.650787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.699200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985f1a6b63a1461ebf46b655825d90a3, user_id=742e9cf3a45c4c2c9a5a315122c8a6d5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.702525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.721065 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1817/3621] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:27 2022] POST /identity/v3/users => generated 295 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:27.723581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.741228 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1805/3622] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:27 2022] PUT /identity/v3/projects/985f1a6b63a1461ebf46b655825d90a3/users/742e9cf3a45c4c2c9a5a315122c8a6d5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:27.742270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.757829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b9134-0bde-44b5-a363-f333845bc838 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.827179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.828132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.828485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.828828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.829637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.831630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.834129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:27.839488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a868a9-dd71-468d-b3c4-008b130ec325 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.845502 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1818/3623] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:27.846482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:27.849641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:27.851876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63be1e1-9d62-42b3-b2ae-0eca160320d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.851876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63be1e1-9d62-42b3-b2ae-0eca160320d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.851876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63be1e1-9d62-42b3-b2ae-0eca160320d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.852310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63be1e1-9d62-42b3-b2ae-0eca160320d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.857219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63be1e1-9d62-42b3-b2ae-0eca160320d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.860491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63be1e1-9d62-42b3-b2ae-0eca160320d5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.867274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54585b97-e096-4f25-843d-ae4b1fcbf8fc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:27.872343 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1806/3624] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:27.872504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:27.872985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.873338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.873661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] PATH_INFO: `/v3/projects/cca911041d3c4273b41e24522d8af674` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.874846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca911041d3c4273b41e24522d8af674)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:27.876715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dc15d-a1c7-4744-b0fe-c55f6d40bede None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.877002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dc15d-a1c7-4744-b0fe-c55f6d40bede None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.877117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:27.877408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dc15d-a1c7-4744-b0fe-c55f6d40bede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.911697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e09dc15d-a1c7-4744-b0fe-c55f6d40bede None None] MFA Rules not processed for user `742e9cf3a45c4c2c9a5a315122c8a6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:27.913976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e09dc15d-a1c7-4744-b0fe-c55f6d40bede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:27.943198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca911041d3c4273b41e24522d8af674', 'request_id': 'req-7270ae15-df3f-4e3b-9abe-8970d3135e17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:27.948743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:27.960965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca911041d3c4273b41e24522d8af674', 'request_id': 'req-7270ae15-df3f-4e3b-9abe-8970d3135e17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:27.983112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca911041d3c4273b41e24522d8af674', 'request_id': 'req-7270ae15-df3f-4e3b-9abe-8970d3135e17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.003735 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1807/3625] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:28.004772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dc15d-a1c7-4744-b0fe-c55f6d40bede None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.015702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] The token cache is being invalidate because project cca911041d3c4273b41e24522d8af674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:28.015996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.016705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.021950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1819/3626] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:27 2022] DELETE /identity/v3/projects/cca911041d3c4273b41e24522d8af674 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:28.021950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.021950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.031836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dc15d-a1c7-4744-b0fe-c55f6d40bede None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:28.102937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270ae15-df3f-4e3b-9abe-8970d3135e17 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:28.107988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a5218-de3f-4f31-b75e-75da49f9d501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:28.108752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a5218-de3f-4f31-b75e-75da49f9d501 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.109019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:28.109213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a5218-de3f-4f31-b75e-75da49f9d501 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.109615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a5218-de3f-4f31-b75e-75da49f9d501 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.109733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.110009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.110279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] PATH_INFO: `/v3/projects/251f64d1f7564cb1888a35e1a2b54bd7/users/6b6cf77443ea4fd199e3b3f9a48ee7d5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.110602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a5218-de3f-4f31-b75e-75da49f9d501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:28.112314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a5218-de3f-4f31-b75e-75da49f9d501 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:28.130300 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1820/3627] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:28.130300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f653f-36cf-4c1e-a0e3-e30da11d8e00 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.130300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f653f-36cf-4c1e-a0e3-e30da11d8e00 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.130300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f653f-36cf-4c1e-a0e3-e30da11d8e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.139478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251f64d1f7564cb1888a35e1a2b54bd7, user_id=6b6cf77443ea4fd199e3b3f9a48ee7d5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:28.142632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:28.153653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1d8f653f-36cf-4c1e-a0e3-e30da11d8e00 None None] MFA Rules not processed for user `742e9cf3a45c4c2c9a5a315122c8a6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:28.155243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1d8f653f-36cf-4c1e-a0e3-e30da11d8e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.160234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1808/3628] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:28 2022] PUT /identity/v3/projects/251f64d1f7564cb1888a35e1a2b54bd7/users/6b6cf77443ea4fd199e3b3f9a48ee7d5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:28.161246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.162919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.246239 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1821/3629] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:28.246239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f653f-36cf-4c1e-a0e3-e30da11d8e00 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.247578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7e167-6ddb-41e3-9884-7e7598a691f6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:28.255296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:28.255968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.256557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.256557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] PATH_INFO: `/v3/users/e86a17c3956b407bbd7166fd1db1c010` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.258542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e86a17c3956b407bbd7166fd1db1c010)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:28.260872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:28.262186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f653f-36cf-4c1e-a0e3-e30da11d8e00 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:28.267589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:28.267724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c639f9f-7fb4-461c-b088-03f8d00936e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:28.268596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c639f9f-7fb4-461c-b088-03f8d00936e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.268738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c639f9f-7fb4-461c-b088-03f8d00936e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.268928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c639f9f-7fb4-461c-b088-03f8d00936e6 admin admin] PATH_INFO: `/v3/projects/a46b32cda9764580a17254358cda6f27/users/d91dcfd497a94654ad913f11d302ae49/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.295244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c639f9f-7fb4-461c-b088-03f8d00936e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a46b32cda9764580a17254358cda6f27, user_id=d91dcfd497a94654ad913f11d302ae49, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:28.299661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c639f9f-7fb4-461c-b088-03f8d00936e6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:28.319622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1822/3630] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:28 2022] PUT /identity/v3/projects/a46b32cda9764580a17254358cda6f27/users/d91dcfd497a94654ad913f11d302ae49/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:28.330767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09772ef-5876-4ec6-af04-fe2863a6e774 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.331045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09772ef-5876-4ec6-af04-fe2863a6e774 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.331329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09772ef-5876-4ec6-af04-fe2863a6e774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.344164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e86a17c3956b407bbd7166fd1db1c010'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.345197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e86a17c3956b407bbd7166fd1db1c010'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.363166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e86a17c3956b407bbd7166fd1db1c010'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.364183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e86a17c3956b407bbd7166fd1db1c010'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.369593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e09772ef-5876-4ec6-af04-fe2863a6e774 None None] MFA Rules not processed for user `6b6cf77443ea4fd199e3b3f9a48ee7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:28.371075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e09772ef-5876-4ec6-af04-fe2863a6e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.384728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e86a17c3956b407bbd7166fd1db1c010'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:28.410894 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1809/3631] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:28 2022] DELETE /identity/v3/users/e86a17c3956b407bbd7166fd1db1c010 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:28.411778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.413320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.442858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:28.492621 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1823/3632] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:28.495536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.495990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.496327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.535419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73366c9b-88fc-446b-a5b8-046157f68632 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:28.546700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] MFA Rules not processed for user `d91dcfd497a94654ad913f11d302ae49`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:28.547606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6703d-b846-4754-8651-d1958e5bceef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:28.548082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.549004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6703d-b846-4754-8651-d1958e5bceef service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.549384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6703d-b846-4754-8651-d1958e5bceef service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.549918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6703d-b846-4754-8651-d1958e5bceef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.552006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f9c6703d-b846-4754-8651-d1958e5bceef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.679746 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1824/3633] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:28.695817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.699201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.733397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6703d-b846-4754-8651-d1958e5bceef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:28.735862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6703d-b846-4754-8651-d1958e5bceef service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:28.737239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c208143-3613-468b-a6a9-d36f24829a36 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:28.751297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:28.751604 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1810/3634] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:28 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:28.752793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.753231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.753714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.783616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.783903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.784184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.808834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] MFA Rules not processed for user `742e9cf3a45c4c2c9a5a315122c8a6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:28.810601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.833110 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1811/3635] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:28.858465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:28.860722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:28.879637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.881446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:28.883698 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1825/3636] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:28.919917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:28.940713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.942196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.039662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c5306-fc27-4a93-9598-67bad015fbe4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.051746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:29.052069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.052345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.052617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] PATH_INFO: `/v3/projects/9f39f15cbd2c4363a0446587cef825b0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.056557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f39f15cbd2c4363a0446587cef825b0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:29.058531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:29.076866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b141d-31f6-4888-844d-facda0687b54 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.089421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:29.090791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.091146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.091492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] PATH_INFO: `/v3/users/6b6cf77443ea4fd199e3b3f9a48ee7d5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.094107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6cf77443ea4fd199e3b3f9a48ee7d5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:29.096804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:29.130678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:29.149008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f39f15cbd2c4363a0446587cef825b0', 'request_id': 'req-be3ced16-f950-4c42-b48e-9403f1b42e30'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.158078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f39f15cbd2c4363a0446587cef825b0', 'request_id': 'req-be3ced16-f950-4c42-b48e-9403f1b42e30'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.168054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f39f15cbd2c4363a0446587cef825b0', 'request_id': 'req-be3ced16-f950-4c42-b48e-9403f1b42e30'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.178494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6cf77443ea4fd199e3b3f9a48ee7d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.190927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] The token cache is being invalidate because project 9f39f15cbd2c4363a0446587cef825b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:29.191318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.192427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.196869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6cf77443ea4fd199e3b3f9a48ee7d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.197951 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1812/3637] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:28 2022] DELETE /identity/v3/projects/9f39f15cbd2c4363a0446587cef825b0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:29.198897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:29.201791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.211459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6cf77443ea4fd199e3b3f9a48ee7d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.212552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6cf77443ea4fd199e3b3f9a48ee7d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.213353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37e1e039-6142-40a9-adb0-2cd84359ce74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6cf77443ea4fd199e3b3f9a48ee7d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:29.236184 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1826/3638] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:28 2022] DELETE /identity/v3/users/6b6cf77443ea4fd199e3b3f9a48ee7d5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:29.237996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1982e4-7cb2-4772-a1e7-f9dbeaaafead None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.238426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1982e4-7cb2-4772-a1e7-f9dbeaaafead None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.238562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1982e4-7cb2-4772-a1e7-f9dbeaaafead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.282537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2a1982e4-7cb2-4772-a1e7-f9dbeaaafead None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:29.283771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2a1982e4-7cb2-4772-a1e7-f9dbeaaafead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.337918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ced16-f950-4c42-b48e-9403f1b42e30 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.386092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:29.393533 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1827/3639] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:29.393533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d34f4-10f4-4c79-a46a-8ef0feb458d1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.393533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d34f4-10f4-4c79-a46a-8ef0feb458d1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.393533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d34f4-10f4-4c79-a46a-8ef0feb458d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.418875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c92d34f4-10f4-4c79-a46a-8ef0feb458d1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:29.420093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c92d34f4-10f4-4c79-a46a-8ef0feb458d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.423203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c9e0a-3a15-4902-a1f5-4def57522ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:29.425534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c9e0a-3a15-4902-a1f5-4def57522ad6 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.425865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c9e0a-3a15-4902-a1f5-4def57522ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.426139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c9e0a-3a15-4902-a1f5-4def57522ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.429504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b37c9e0a-3a15-4902-a1f5-4def57522ad6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.433569 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1828/3640] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:29.435049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.435434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.435809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.461525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] MFA Rules not processed for user `742e9cf3a45c4c2c9a5a315122c8a6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:29.463129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.510593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c9e0a-3a15-4902-a1f5-4def57522ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:29.512246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c9e0a-3a15-4902-a1f5-4def57522ad6 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:29.525117 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1813/3641] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:29.526537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477fb2d-3556-4583-bca5-ff1fdbd3c82d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.526829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477fb2d-3556-4583-bca5-ff1fdbd3c82d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.527161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477fb2d-3556-4583-bca5-ff1fdbd3c82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.557069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2477fb2d-3556-4583-bca5-ff1fdbd3c82d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:29.560155 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1829/3642] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:29.568432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2477fb2d-3556-4583-bca5-ff1fdbd3c82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.569994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:29.572370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.604137 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1814/3643] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:29.613524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477fb2d-3556-4583-bca5-ff1fdbd3c82d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:29.633532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477fb2d-3556-4583-bca5-ff1fdbd3c82d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.639549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ce745-2d71-404a-b28f-92e479b4d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:29.640391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce745-2d71-404a-b28f-92e479b4d555 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.640766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce745-2d71-404a-b28f-92e479b4d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.641124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce745-2d71-404a-b28f-92e479b4d555 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.641972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ce745-2d71-404a-b28f-92e479b4d555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:29.642468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ce745-2d71-404a-b28f-92e479b4d555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:29.650612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ce745-2d71-404a-b28f-92e479b4d555 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:29.671237 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1815/3644] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:29.672721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a81b00-c00f-4a89-857b-a6ac8e23a87b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.673121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a81b00-c00f-4a89-857b-a6ac8e23a87b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.673994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a81b00-c00f-4a89-857b-a6ac8e23a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.689218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:29.702144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-40a81b00-c00f-4a89-857b-a6ac8e23a87b None None] MFA Rules not processed for user `742e9cf3a45c4c2c9a5a315122c8a6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:29.708933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-40a81b00-c00f-4a89-857b-a6ac8e23a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.716446 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1816/3645] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:29.721728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a81b00-c00f-4a89-857b-a6ac8e23a87b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:29.723559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570225a-3b9c-4ae8-9c32-78a4df57f325 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.731227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79c16f-1408-44c3-92ca-714586a2d389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:29.732179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79c16f-1408-44c3-92ca-714586a2d389 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.732643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79c16f-1408-44c3-92ca-714586a2d389 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.733124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79c16f-1408-44c3-92ca-714586a2d389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.734764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a81b00-c00f-4a89-857b-a6ac8e23a87b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.745429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:29.745429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.745429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.745429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.745766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:29.745766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:29.766665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79c16f-1408-44c3-92ca-714586a2d389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:29.775643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79c16f-1408-44c3-92ca-714586a2d389 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:29.792423 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1830/3646] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:29.794853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4277c-92dc-4515-bd51-f2ee7d8d520f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.799202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4277c-92dc-4515-bd51-f2ee7d8d520f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.799575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4277c-92dc-4515-bd51-f2ee7d8d520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.831067 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1817/3647] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:29.832054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:29.839526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.849946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-16c4277c-92dc-4515-bd51-f2ee7d8d520f None None] MFA Rules not processed for user `742e9cf3a45c4c2c9a5a315122c8a6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:29.851755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-16c4277c-92dc-4515-bd51-f2ee7d8d520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.972490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19819ee1-dec0-41a2-bc48-a646bcd7db9f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.982492 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1831/3648] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:29.983757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4277c-92dc-4515-bd51-f2ee7d8d520f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:29.999388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.000046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.000323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.000578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] PATH_INFO: `/v3/users/d91dcfd497a94654ad913f11d302ae49` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.005028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91dcfd497a94654ad913f11d302ae49)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.006267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.011785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4277c-92dc-4515-bd51-f2ee7d8d520f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.024779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.025641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.025956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.026327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.027001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.032976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.073878 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1832/3649] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/users => generated 289 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:30.073878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:30.076908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.094446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:30.111501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336a27-0e19-4205-bc6f-38e09c5a9218 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.120401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91dcfd497a94654ad913f11d302ae49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.121090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91dcfd497a94654ad913f11d302ae49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.127598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.128661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.129165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.129692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] PATH_INFO: `/v3/users/742e9cf3a45c4c2c9a5a315122c8a6d5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.133334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91dcfd497a94654ad913f11d302ae49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.135809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91dcfd497a94654ad913f11d302ae49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.137246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=742e9cf3a45c4c2c9a5a315122c8a6d5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.141849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.167617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91dcfd497a94654ad913f11d302ae49'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.193763 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1818/3650] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:29 2022] DELETE /identity/v3/users/d91dcfd497a94654ad913f11d302ae49 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:30.196302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:30.202496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.247452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742e9cf3a45c4c2c9a5a315122c8a6d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.278093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742e9cf3a45c4c2c9a5a315122c8a6d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.301270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742e9cf3a45c4c2c9a5a315122c8a6d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.304396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742e9cf3a45c4c2c9a5a315122c8a6d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.309903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742e9cf3a45c4c2c9a5a315122c8a6d5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.317930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de8c4d-4db2-48d0-bad8-1d626584635f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.324375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ae7aa-c005-48e8-bd39-07871ff13512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.325055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ae7aa-c005-48e8-bd39-07871ff13512 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.325333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ae7aa-c005-48e8-bd39-07871ff13512 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.325678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ae7aa-c005-48e8-bd39-07871ff13512 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.326385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ae7aa-c005-48e8-bd39-07871ff13512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.328462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ae7aa-c005-48e8-bd39-07871ff13512 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.345099 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1819/3651] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:30.346523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c10c50-513f-41d7-b0e9-541c3b386c69 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.346869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c10c50-513f-41d7-b0e9-541c3b386c69 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.347146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c10c50-513f-41d7-b0e9-541c3b386c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.353291 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1833/3652] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:30 2022] DELETE /identity/v3/users/742e9cf3a45c4c2c9a5a315122c8a6d5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:30.353291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:30.365592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.391554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-22c10c50-513f-41d7-b0e9-541c3b386c69 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:30.392995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-22c10c50-513f-41d7-b0e9-541c3b386c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.470132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:30.516954 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1820/3653] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:30.519031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.519319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.519589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.530031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf952ea-622b-4722-8234-98991b0030a3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.542191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.542969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.546059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.548387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.549110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.551017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.581502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:30.581502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.588377 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1834/3654] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:30.588377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:30.590966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.600483 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1821/3655] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:30.602758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:30.604348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.613652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1018a5-f825-45b7-bb99-d7b3ab61ebdb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.619252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.619889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.620181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.620438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] PATH_INFO: `/v3/projects/251f64d1f7564cb1888a35e1a2b54bd7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.621614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=251f64d1f7564cb1888a35e1a2b54bd7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.623623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.675889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:30.691118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251f64d1f7564cb1888a35e1a2b54bd7', 'request_id': 'req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.705481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251f64d1f7564cb1888a35e1a2b54bd7', 'request_id': 'req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.709711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b148b-643f-4aae-9fc9-299405c6145a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.712458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251f64d1f7564cb1888a35e1a2b54bd7', 'request_id': 'req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.726779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] The token cache is being invalidate because project 251f64d1f7564cb1888a35e1a2b54bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:30.727108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.727916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:30.729751 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1835/3656] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:30 2022] DELETE /identity/v3/projects/251f64d1f7564cb1888a35e1a2b54bd7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:30.730333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:30.731724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.775900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9c29d-1ea4-4cea-86a7-318ff0a6742f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.776982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9c29d-1ea4-4cea-86a7-318ff0a6742f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.777450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9c29d-1ea4-4cea-86a7-318ff0a6742f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.777973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9c29d-1ea4-4cea-86a7-318ff0a6742f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.780014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-00f9c29d-1ea4-4cea-86a7-318ff0a6742f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.816550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84ade2-27e5-4e84-a806-28dcbdcfb0a5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.822359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.823216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.823550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.823880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] PATH_INFO: `/v3/projects/fe6e6d8c6fb4445492abb5cb243a6600/users/2da51a5c76b24768a150a90e183a8caa/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.842128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6e6d8c6fb4445492abb5cb243a6600, user_id=2da51a5c76b24768a150a90e183a8caa, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.845126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.864084 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1836/3657] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:30 2022] PUT /identity/v3/projects/fe6e6d8c6fb4445492abb5cb243a6600/users/2da51a5c76b24768a150a90e183a8caa/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:30.865716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:30.865906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9c29d-1ea4-4cea-86a7-318ff0a6742f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:30.868028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.875135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9c29d-1ea4-4cea-86a7-318ff0a6742f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:30.888603 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1822/3658] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:30.894928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a952d-40eb-4cc4-87ea-a57c21bb7bc8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.895479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a952d-40eb-4cc4-87ea-a57c21bb7bc8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.895937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a952d-40eb-4cc4-87ea-a57c21bb7bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.933117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:30.947640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e00a952d-40eb-4cc4-87ea-a57c21bb7bc8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:30.951874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e00a952d-40eb-4cc4-87ea-a57c21bb7bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:30.954143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148ad09-4b2e-4561-9c54-444090cb7ff7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:30.959043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca13a5-bab6-4ca0-b2c2-5f9b8cb5eef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:30.960451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca13a5-bab6-4ca0-b2c2-5f9b8cb5eef6 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:30.961160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca13a5-bab6-4ca0-b2c2-5f9b8cb5eef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:30.961654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca13a5-bab6-4ca0-b2c2-5f9b8cb5eef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:30.965077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b2ca13a5-bab6-4ca0-b2c2-5f9b8cb5eef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:31.077499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca13a5-bab6-4ca0-b2c2-5f9b8cb5eef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.080355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca13a5-bab6-4ca0-b2c2-5f9b8cb5eef6 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.096373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1837/3660] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:31.098063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fcbc8-1c16-4e3b-9884-4525528aad34 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.098529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fcbc8-1c16-4e3b-9884-4525528aad34 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.098998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fcbc8-1c16-4e3b-9884-4525528aad34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.099668 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1823/3660] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:31.102982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6746-cd83-4913-a2cc-619ce664acc5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.103391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6746-cd83-4913-a2cc-619ce664acc5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.103763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6746-cd83-4913-a2cc-619ce664acc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.135591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dbdc6746-cd83-4913-a2cc-619ce664acc5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:31.137717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dbdc6746-cd83-4913-a2cc-619ce664acc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:31.147358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ab4fcbc8-1c16-4e3b-9884-4525528aad34 None None] MFA Rules not processed for user `2da51a5c76b24768a150a90e183a8caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:31.152650 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1824/3661] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:31.152808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ab4fcbc8-1c16-4e3b-9884-4525528aad34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:31.154266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc6746-cd83-4913-a2cc-619ce664acc5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.170569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc6746-cd83-4913-a2cc-619ce664acc5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.178624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.180084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.180656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.181007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.182333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:31.182802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.185252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.210493 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1825/3662] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:02:31.211678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.240805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d0bf8-13f1-4132-970e-ad09fbac2f01 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.283256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.287496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.289546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.302392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.304132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.307467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.391789 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1838/3663] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:31.396059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fcbc8-1c16-4e3b-9884-4525528aad34 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.401699 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1826/3664] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:31 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:31.402603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.414865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:31.416351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fcbc8-1c16-4e3b-9884-4525528aad34 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.550622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.551163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009059b-a0f6-4e85-bceb-f373137e5517 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.551292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.551520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.551788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.552431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.554493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.566113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.566915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.567536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.567536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.568357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.570675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.611806 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1839/3665] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:31 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:31.612971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.615053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:31.619907 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1827/3666] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:31 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:31.620881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.628997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:31.647407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0855f-01c6-4ad0-a20b-e8c3776d08c7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.756194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.756877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.757299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.757689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.758523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.761108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.798252 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1828/3667] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:31 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:31.799108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.816190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439fc63-f27e-4e85-8e10-ddb5693a73f4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.821911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.822777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.823065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.823343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.823906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cddf9-3244-4507-885a-05592c008132 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.824029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.826049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.833783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.836627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.837076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.837832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] PATH_INFO: `/v3/projects/a46b32cda9764580a17254358cda6f27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.839315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a46b32cda9764580a17254358cda6f27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.840301 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1829/3668] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:31.841708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.842126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.844668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:31.861187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48048a-9d5a-4d36-bef8-f84b81708a23 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:31.877482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:31.877482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:31.877482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:31.877482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] PATH_INFO: `/v3/projects/985f1a6b63a1461ebf46b655825d90a3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:31.877933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] RBAC: Authorizing `identity:delete_project(project_id=985f1a6b63a1461ebf46b655825d90a3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:31.877933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:31.934656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a46b32cda9764580a17254358cda6f27', 'request_id': 'req-11463b2c-9ba7-451e-bee8-4357530045a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:31.950845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a46b32cda9764580a17254358cda6f27', 'request_id': 'req-11463b2c-9ba7-451e-bee8-4357530045a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:31.963459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a46b32cda9764580a17254358cda6f27', 'request_id': 'req-11463b2c-9ba7-451e-bee8-4357530045a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:31.985395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] The token cache is being invalidate because project a46b32cda9764580a17254358cda6f27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:31.985746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:31.986800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:31.991975 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1840/3669] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:31 2022] DELETE /identity/v3/projects/a46b32cda9764580a17254358cda6f27 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:31.991975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:31.991975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:31.994561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985f1a6b63a1461ebf46b655825d90a3', 'request_id': 'req-4553fb27-ceff-4449-8f27-222e4935775f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:32.007297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985f1a6b63a1461ebf46b655825d90a3', 'request_id': 'req-4553fb27-ceff-4449-8f27-222e4935775f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:32.022446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985f1a6b63a1461ebf46b655825d90a3', 'request_id': 'req-4553fb27-ceff-4449-8f27-222e4935775f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:32.037969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:32.049163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] The token cache is being invalidate because project 985f1a6b63a1461ebf46b655825d90a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:32.049999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:32.050757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:32.052822 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1830/3670] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:31 2022] DELETE /identity/v3/projects/985f1a6b63a1461ebf46b655825d90a3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:32.054039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.056467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.092784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463b2c-9ba7-451e-bee8-4357530045a8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.151652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:32.156168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553fb27-ceff-4449-8f27-222e4935775f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.166454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b80999-69d1-4829-b100-f044d3be314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.167369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80999-69d1-4829-b100-f044d3be314b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.167812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80999-69d1-4829-b100-f044d3be314b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.168239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80999-69d1-4829-b100-f044d3be314b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.169056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b80999-69d1-4829-b100-f044d3be314b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.170878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b80999-69d1-4829-b100-f044d3be314b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.177450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a44e-a165-4e56-9806-bd633f4c041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.178219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a44e-a165-4e56-9806-bd633f4c041a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.178597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a44e-a165-4e56-9806-bd633f4c041a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.179087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a44e-a165-4e56-9806-bd633f4c041a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.179844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a44e-a165-4e56-9806-bd633f4c041a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.181591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a44e-a165-4e56-9806-bd633f4c041a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.185757 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1831/3671] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:32.185757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceadd98-94d8-4841-bc04-8ecb0f6a0e35 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.185757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceadd98-94d8-4841-bc04-8ecb0f6a0e35 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.185757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceadd98-94d8-4841-bc04-8ecb0f6a0e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.201924 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1841/3672] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:32.201924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.201924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.201924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.206626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dceadd98-94d8-4841-bc04-8ecb0f6a0e35 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:32.207960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dceadd98-94d8-4841-bc04-8ecb0f6a0e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.217391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1832/3673] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:32.217684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceadd98-94d8-4841-bc04-8ecb0f6a0e35 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.230454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:32.231799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.235611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceadd98-94d8-4841-bc04-8ecb0f6a0e35 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.240504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7bce-deaa-4974-979f-2513c8f6876a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.241303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7bce-deaa-4974-979f-2513c8f6876a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.241667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7bce-deaa-4974-979f-2513c8f6876a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.241985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7bce-deaa-4974-979f-2513c8f6876a admin admin] PATH_INFO: `/v3/projects/b33b687e3d75472bb38e672e90871a0b/users/00e4ba59294f4529ba188e07cfc6681a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.244187 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1842/3674] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:32.245296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.247695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.260966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd48fc-89b0-41e1-a0cc-0dc90c51c67a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.262170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7bce-deaa-4974-979f-2513c8f6876a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33b687e3d75472bb38e672e90871a0b, user_id=00e4ba59294f4529ba188e07cfc6681a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.264985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7bce-deaa-4974-979f-2513c8f6876a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.265759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7320f-5503-4d24-9192-e96844ab2caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.266430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7320f-5503-4d24-9192-e96844ab2caa admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.266707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7320f-5503-4d24-9192-e96844ab2caa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.266981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7320f-5503-4d24-9192-e96844ab2caa admin admin] PATH_INFO: `/v3/projects/530a35c0c37841afb82bf3a00ab9b8d6/users/84558779f0954f3db60615f78930dad5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.282386 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1833/3675] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:32 2022] PUT /identity/v3/projects/b33b687e3d75472bb38e672e90871a0b/users/00e4ba59294f4529ba188e07cfc6681a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:32.283709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9c87d-ac81-41ef-b981-471a9a97ac2c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.284061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9c87d-ac81-41ef-b981-471a9a97ac2c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.284660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9c87d-ac81-41ef-b981-471a9a97ac2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.288550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7320f-5503-4d24-9192-e96844ab2caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=530a35c0c37841afb82bf3a00ab9b8d6, user_id=84558779f0954f3db60615f78930dad5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.291908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7320f-5503-4d24-9192-e96844ab2caa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.308629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b6b9c87d-ac81-41ef-b981-471a9a97ac2c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:32.310285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b6b9c87d-ac81-41ef-b981-471a9a97ac2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.313274 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1843/3676] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:32 2022] PUT /identity/v3/projects/530a35c0c37841afb82bf3a00ab9b8d6/users/84558779f0954f3db60615f78930dad5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:32.314687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bfb2c-86f8-416c-b2aa-f32601a3708a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.315037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bfb2c-86f8-416c-b2aa-f32601a3708a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.315244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bfb2c-86f8-416c-b2aa-f32601a3708a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.343458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a76bfb2c-86f8-416c-b2aa-f32601a3708a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:32.344696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a76bfb2c-86f8-416c-b2aa-f32601a3708a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.390019 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1834/3677] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:32.395649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016788a-a89f-4ae5-a7c6-6e47f6271d8b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.396111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016788a-a89f-4ae5-a7c6-6e47f6271d8b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.396587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016788a-a89f-4ae5-a7c6-6e47f6271d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.426863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6016788a-a89f-4ae5-a7c6-6e47f6271d8b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:32.428584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6016788a-a89f-4ae5-a7c6-6e47f6271d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.443957 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1835/3678] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:32.445184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016788a-a89f-4ae5-a7c6-6e47f6271d8b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.458249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016788a-a89f-4ae5-a7c6-6e47f6271d8b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.468299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.470320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.470705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.471634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.472428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.489362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.500521 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1844/3679] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:32.501794 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1836/3680] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:32.502821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.504952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bfb2c-86f8-416c-b2aa-f32601a3708a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.507878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.521332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bfb2c-86f8-416c-b2aa-f32601a3708a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.530711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.531709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.532158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.532569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.533378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:32.533908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.535697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.549001 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1845/3681] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:32.550705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.563629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f9a14-bbcb-452e-9070-9858733ba403 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.568864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.569515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.569893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.570117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.570778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:32.571242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.572889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.593114 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1846/3682] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:32.594113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.607307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dc41d-23f0-4978-bec0-5e10593cc84d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.612383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.613039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.613314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.617093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] PATH_INFO: `/v3/projects/530a35c0c37841afb82bf3a00ab9b8d6/users/84558779f0954f3db60615f78930dad5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.617093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530a35c0c37841afb82bf3a00ab9b8d6, user_id=84558779f0954f3db60615f78930dad5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.618728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.621965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:32.634726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4c895-7af3-4252-a992-b1e54ad2ddda admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.634924 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1847/3683] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:32 2022] PUT /identity/v3/projects/530a35c0c37841afb82bf3a00ab9b8d6/users/84558779f0954f3db60615f78930dad5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:32.635697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.734452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.735335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.737756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.737756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.741612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9900c5-608c-4b9e-9a52-fbdb946f6cb7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.749977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e8ae5-63c9-4eac-aee3-958fca711412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.751304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e8ae5-63c9-4eac-aee3-958fca711412 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.751906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e8ae5-63c9-4eac-aee3-958fca711412 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.752796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e8ae5-63c9-4eac-aee3-958fca711412 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.753434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.753737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e8ae5-63c9-4eac-aee3-958fca711412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.755835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e8ae5-63c9-4eac-aee3-958fca711412 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.757201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.776877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:32.781925 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1837/3684] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:32.784428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:32.805437 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1848/3685] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:32.807031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bd21b-5073-4ef2-9005-b76cc25a5755 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.807329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bd21b-5073-4ef2-9005-b76cc25a5755 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.807607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bd21b-5073-4ef2-9005-b76cc25a5755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.808697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93b758-390e-40e9-b9e1-ca8e4d9e63e9 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:32.882222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-480bd21b-5073-4ef2-9005-b76cc25a5755 None None] MFA Rules not processed for user `84558779f0954f3db60615f78930dad5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:32.884735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-480bd21b-5073-4ef2-9005-b76cc25a5755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:32.914609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:32.915565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:32.915994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:32.916489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:32.917585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:32.920253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:32.993926 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1838/3686] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:32.995393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.040904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1849/3687] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:33.040904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89c917-70c0-42da-9289-5cfd8158ab61 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.040904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89c917-70c0-42da-9289-5cfd8158ab61 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.040904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89c917-70c0-42da-9289-5cfd8158ab61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.074542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ad89c917-70c0-42da-9289-5cfd8158ab61 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:33.076399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ad89c917-70c0-42da-9289-5cfd8158ab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:33.176819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f85e1-30aa-43d4-90d6-f51e6f45b385 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.183047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.184431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.184872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.185223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.186070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.188643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.235220 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1850/3688] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:02:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:33.241486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89c917-70c0-42da-9289-5cfd8158ab61 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.245593 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1839/3689] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:32 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:33.245593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.281964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89c917-70c0-42da-9289-5cfd8158ab61 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.291846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:33.294893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.295555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.295846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.296382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.297034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.299076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.356038 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1851/3690] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:33 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:33.357704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.364185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf322c0-c975-45db-b977-ee958bc78d79 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.369820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.371543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.372093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.372607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.373536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.375690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.392990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42458273-692e-42c0-a820-be671cbe5392 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.405755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.406789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.407404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.407895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.409429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.412505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.424634 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1840/3691] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:33 2022] POST /identity/v3/projects => generated 333 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:33.424634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.424925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:33.428570 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1852/3692] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:33.429853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.445233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a466e-5d7c-4992-b71c-0efe30d01e30 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.582980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:33.593618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.597849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.600166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.600581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.601430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.603221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.616946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d7974-b20e-4199-adb8-a4ba91a7b3f6 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.618966 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1853/3693] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:33.619810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.625871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.626492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.626760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.627027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.702687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f73da-f426-4acf-b094-f6b17ddfe265 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.707814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.708658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.709091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.709666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.710656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.712788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.750156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.752276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.765350 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1841/3694] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:02:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:02:33.766694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.780957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe44bfd-d194-4d0a-b3d4-15519d6229fd service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.790784 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1854/3695] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:02:33 2022] POST /identity/v3/projects => generated 332 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:33.792692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.800791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.802511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.803398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.803845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.804699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.809696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.825627 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1842/3696] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:33.825627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.841888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0fbb7-bfe1-4406-8a7c-ce21bd1208f6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.916712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:33.945996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41108257-5cae-4916-84ef-996838d630bd tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.954387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.955161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.955448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.955724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.956368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.958689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.968169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.969237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.973528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.973528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.973528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.975664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:34.004517 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1855/3697] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:34.005379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:34.043593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c8fc-31a6-4621-ac91-9344c70d510d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:34.065183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:34.070619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:34.071142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:34.071445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] PATH_INFO: `/v3/projects/9f9877108f734092be1197672d500334/users/128155ce2ea242bd9d59ec188c57c34b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:34.078546 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1843/3698] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:33 2022] POST /identity/v3/projects => generated 358 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:34.080881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:34.118214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9877108f734092be1197672d500334, user_id=128155ce2ea242bd9d59ec188c57c34b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:34.121490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:34.152844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1856/3699] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:34 2022] PUT /identity/v3/projects/9f9877108f734092be1197672d500334/users/128155ce2ea242bd9d59ec188c57c34b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:34.153869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:34.228855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d538338-42b6-4115-a4e1-9705f813b567 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:34.274985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:34.300239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9a50c-e273-4fbb-862b-88b71e83ef7a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:34.314235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6d49b-001b-4711-b46e-078e783e192d tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:34.314870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6d49b-001b-4711-b46e-078e783e192d tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:34.315166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6d49b-001b-4711-b46e-078e783e192d tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:34.315448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6d49b-001b-4711-b46e-078e783e192d tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:34.316106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6d49b-001b-4711-b46e-078e783e192d tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:34.318043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6d49b-001b-4711-b46e-078e783e192d tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:34.362808 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1857/3700] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:02:34 2022] GET /identity/v3/projects => generated 7244 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:34.364242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51ab0e-042a-41c0-ad8d-cd8cc73e9773 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:34.364529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51ab0e-042a-41c0-ad8d-cd8cc73e9773 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:34.364806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51ab0e-042a-41c0-ad8d-cd8cc73e9773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:34.428763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3f51ab0e-042a-41c0-ad8d-cd8cc73e9773 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:34.430525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3f51ab0e-042a-41c0-ad8d-cd8cc73e9773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:34.549491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:34.550528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:34.550969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:34.551433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] PATH_INFO: `/v3/projects/575b0ac677234f6ba9c5b9715cc02f63/users/11f65cdf47c1489f905d517a7a4fdea5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:34.590563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575b0ac677234f6ba9c5b9715cc02f63, user_id=11f65cdf47c1489f905d517a7a4fdea5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:34.596731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:34.654349 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1844/3701] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:34 2022] PUT /identity/v3/projects/575b0ac677234f6ba9c5b9715cc02f63/users/11f65cdf47c1489f905d517a7a4fdea5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:34.655997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:34.660799 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1858/3702] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:34.665936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:34.666359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:34.666734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:34.759814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] MFA Rules not processed for user `11f65cdf47c1489f905d517a7a4fdea5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:34.761411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:34.825861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:34.891886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e16b5-aea7-432e-98ab-5596c4a74c07 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:34.901869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:34.902559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:34.902843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:34.903120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:34.903835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:34.904136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:34.905177 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:34.905177 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:02:34.905177 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:02:34.907506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:34.934999 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1859/3703] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:34.938116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:34.940564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1845/3704] 10.222.0.26 () {58 vars in 1158 bytes} [Sat Sep 17 21:02:34 2022] GET /identity/v3/projects?enabled=False => generated 444 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:34.942056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:34.951365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:34.992328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa88d97-1201-46ae-9799-dac1f18fed3c tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:35.001403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cfc26-81d4-49bd-85a0-f12efd028a6b tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:35.002057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cfc26-81d4-49bd-85a0-f12efd028a6b tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:35.002342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cfc26-81d4-49bd-85a0-f12efd028a6b tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:35.002620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cfc26-81d4-49bd-85a0-f12efd028a6b tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:35.003979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cfc26-81d4-49bd-85a0-f12efd028a6b tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Adding query filter params (parent_id=ce0fcfff54b24daebe1cd2e89f1cdefc) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:35.004299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cfc26-81d4-49bd-85a0-f12efd028a6b tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:35.006248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cfc26-81d4-49bd-85a0-f12efd028a6b tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:35.051831 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1846/3705] 10.222.0.26 () {58 vars in 1216 bytes} [Sat Sep 17 21:02:34 2022] GET /identity/v3/projects?parent_id=ce0fcfff54b24daebe1cd2e89f1cdefc => generated 498 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:35.061797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf74e5-4f64-455e-9318-6372662634a6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:35.062149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf74e5-4f64-455e-9318-6372662634a6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:35.062342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf74e5-4f64-455e-9318-6372662634a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:35.099937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cdaf74e5-4f64-455e-9318-6372662634a6 None None] MFA Rules not processed for user `11f65cdf47c1489f905d517a7a4fdea5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:35.102535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cdaf74e5-4f64-455e-9318-6372662634a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:35.111102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1847/3706] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:35.112689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf74e5-4f64-455e-9318-6372662634a6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:35.128109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf74e5-4f64-455e-9318-6372662634a6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:35.134983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:35.135505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be1cc1-676a-4c6f-b81a-cc7283c19a0e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:35.135762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:35.136047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:35.136318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects/bd77e583bdda4e4f9b79026d11c78c83` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:35.137669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bd77e583bdda4e4f9b79026d11c78c83)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:35.139759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:35.146870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:35.147659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:35.148167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:35.148167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:35.208015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:35.265935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd77e583bdda4e4f9b79026d11c78c83', 'request_id': 'req-7b7869a7-f079-450c-be04-1b8776548200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.283822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:35.285296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:35.285747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd77e583bdda4e4f9b79026d11c78c83', 'request_id': 'req-7b7869a7-f079-450c-be04-1b8776548200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.306287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd77e583bdda4e4f9b79026d11c78c83', 'request_id': 'req-7b7869a7-f079-450c-be04-1b8776548200'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.311666 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1860/3707] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:35.321487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] The token cache is being invalidate because project bd77e583bdda4e4f9b79026d11c78c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:35.322109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.322392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:35.324260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.326326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:35.329360 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1848/3708] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:35 2022] DELETE /identity/v3/projects/bd77e583bdda4e4f9b79026d11c78c83 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:35.333196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:35.336932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:35.505051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7869a7-f079-450c-be04-1b8776548200 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:35.512508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:35.513061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:35.513437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:35.513832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects/ce0fcfff54b24daebe1cd2e89f1cdefc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:35.515128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ce0fcfff54b24daebe1cd2e89f1cdefc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:35.523706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:35.592108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797b8ac-d174-45c7-a252-ed2f0d0588f8 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:35.620649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:35.627021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce0fcfff54b24daebe1cd2e89f1cdefc', 'request_id': 'req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.644042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce0fcfff54b24daebe1cd2e89f1cdefc', 'request_id': 'req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.664712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce0fcfff54b24daebe1cd2e89f1cdefc', 'request_id': 'req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.681337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] The token cache is being invalidate because project ce0fcfff54b24daebe1cd2e89f1cdefc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:35.686803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.694034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:35.699645 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1849/3709] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:35 2022] DELETE /identity/v3/projects/ce0fcfff54b24daebe1cd2e89f1cdefc => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:35.705793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:35.709954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:35.812593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:35.814167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:35.814613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:35.815038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:35.821873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:35.917878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae9b34-3085-4a6e-96d5-64b8d6a187c3 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:35.943059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84558779f0954f3db60615f78930dad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '530a35c0c37841afb82bf3a00ab9b8d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:35.944061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:35.944440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:35.944840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] PATH_INFO: `/v3/projects/c459b88280d549b2b2efcef4a33f93ca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:35.946191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c459b88280d549b2b2efcef4a33f93ca)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:35.948604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:35.996814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:35.999103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:36.008522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:36.022096 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1861/3710] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:36.102051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c459b88280d549b2b2efcef4a33f93ca', 'request_id': 'req-a36ccbca-7b4e-4930-a666-d89e798134e7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.113801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c459b88280d549b2b2efcef4a33f93ca', 'request_id': 'req-a36ccbca-7b4e-4930-a666-d89e798134e7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.139529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c459b88280d549b2b2efcef4a33f93ca', 'request_id': 'req-a36ccbca-7b4e-4930-a666-d89e798134e7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.160377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] The token cache is being invalidate because project c459b88280d549b2b2efcef4a33f93ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:36.160832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.165132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a36ccbca-7b4e-4930-a666-d89e798134e7 tempest-ListProjectsTestJSON-1263802493 tempest-ListProjectsTestJSON-1263802493-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.172106 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1850/3711] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:35 2022] DELETE /identity/v3/projects/c459b88280d549b2b2efcef4a33f93ca => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:36.176651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:36.178525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:36.485078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af105292-83a2-42e0-af2a-60ff6da0e6de service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:36.691104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:36.692274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:36.692753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:36.693196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] PATH_INFO: `/v3/users/2da51a5c76b24768a150a90e183a8caa` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:36.696528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da51a5c76b24768a150a90e183a8caa)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:36.699364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:36.804791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da51a5c76b24768a150a90e183a8caa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.834331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da51a5c76b24768a150a90e183a8caa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.855937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da51a5c76b24768a150a90e183a8caa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.856910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da51a5c76b24768a150a90e183a8caa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.859680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da51a5c76b24768a150a90e183a8caa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:36.901250 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1862/3712] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:36 2022] DELETE /identity/v3/users/2da51a5c76b24768a150a90e183a8caa => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:36.911633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:36.911633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:36.911633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:36.972205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:36.973434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:37.214282 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1851/3713] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:37.262872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:37.264962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:37.336840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:37.453182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915611d-9ab6-4d49-9249-8d638cd84874 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:37.464794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:37.467799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:37.467985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:37.471996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:37.515380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:37.516967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:37.550656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1863/3714] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:39.451261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:39.454099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:39.463700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:39.491596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff73cf-a351-4675-8fc6-24a3ef4d41d3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:39.514090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:39.515169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:39.515864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:39.516364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] PATH_INFO: `/v3/projects/fe6e6d8c6fb4445492abb5cb243a6600` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:39.523489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6e6d8c6fb4445492abb5cb243a6600)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:39.530048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:39.742204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6e6d8c6fb4445492abb5cb243a6600', 'request_id': 'req-279886e6-206f-4da0-bfe8-e7dc9666e5e1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:39.766166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6e6d8c6fb4445492abb5cb243a6600', 'request_id': 'req-279886e6-206f-4da0-bfe8-e7dc9666e5e1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:39.798983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6e6d8c6fb4445492abb5cb243a6600', 'request_id': 'req-279886e6-206f-4da0-bfe8-e7dc9666e5e1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:39.826928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] The token cache is being invalidate because project fe6e6d8c6fb4445492abb5cb243a6600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:39.827294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:39.828859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:39.833333 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1852/3715] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:39 2022] DELETE /identity/v3/projects/fe6e6d8c6fb4445492abb5cb243a6600 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:39.837603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:39.860514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a829574-3caf-4b0f-b0cb-60ca4441b928 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:40.176007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:40.176682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:40.176959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:40.177267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] PATH_INFO: `/v3/users/84558779f0954f3db60615f78930dad5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:40.192610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84558779f0954f3db60615f78930dad5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:40.200614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:40.399124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84558779f0954f3db60615f78930dad5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:40.436417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84558779f0954f3db60615f78930dad5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:40.463856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84558779f0954f3db60615f78930dad5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:40.464977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84558779f0954f3db60615f78930dad5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:40.466435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-90aa4144-349f-4282-999c-898113d83724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84558779f0954f3db60615f78930dad5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:40.505303 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1864/3716] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:39 2022] DELETE /identity/v3/users/84558779f0954f3db60615f78930dad5 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:42.341860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:42.345508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:42.381031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279886e6-206f-4da0-bfe8-e7dc9666e5e1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:42.572953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ee51-4436-4970-bad9-aa65c6a47c4a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:42.573453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ee51-4436-4970-bad9-aa65c6a47c4a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:42.573453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57ee51-4436-4970-bad9-aa65c6a47c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:42.573759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:42.574388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:42.576651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:42.579784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] PATH_INFO: `/v3/projects/530a35c0c37841afb82bf3a00ab9b8d6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:42.583108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=530a35c0c37841afb82bf3a00ab9b8d6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:42.588153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:42.648489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3f57ee51-4436-4970-bad9-aa65c6a47c4a None None] MFA Rules not processed for user `00e4ba59294f4529ba188e07cfc6681a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:42.649805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3f57ee51-4436-4970-bad9-aa65c6a47c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:42.715491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530a35c0c37841afb82bf3a00ab9b8d6', 'request_id': 'req-c4d3a9b4-f631-4f23-8826-bf758338e14e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:42.726311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530a35c0c37841afb82bf3a00ab9b8d6', 'request_id': 'req-c4d3a9b4-f631-4f23-8826-bf758338e14e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:42.728522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:42.751851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530a35c0c37841afb82bf3a00ab9b8d6', 'request_id': 'req-c4d3a9b4-f631-4f23-8826-bf758338e14e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:42.800560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] The token cache is being invalidate because project 530a35c0c37841afb82bf3a00ab9b8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:42.800784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:42.813320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:42.820829 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1865/3717] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:42.828388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1853/3718] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:42 2022] DELETE /identity/v3/projects/530a35c0c37841afb82bf3a00ab9b8d6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:42.830307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb1041-eae6-46d9-a86f-ffa67a3ee84a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:42.830597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb1041-eae6-46d9-a86f-ffa67a3ee84a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:42.830892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb1041-eae6-46d9-a86f-ffa67a3ee84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:42.848005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:42.852731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:42.892863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a7fb1041-eae6-46d9-a86f-ffa67a3ee84a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:42.902022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7fb1041-eae6-46d9-a86f-ffa67a3ee84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:42.928265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:43.157829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a9b4-f631-4f23-8826-bf758338e14e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.211164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.212253 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1866/3719] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:43.212843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.213229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.213639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.218602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:43.220289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe00fcf-24dc-4988-839b-88b3fd54456d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.223162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe00fcf-24dc-4988-839b-88b3fd54456d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.223660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe00fcf-24dc-4988-839b-88b3fd54456d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.286177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ffe00fcf-24dc-4988-839b-88b3fd54456d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:43.287666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ffe00fcf-24dc-4988-839b-88b3fd54456d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:43.311850 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1867/3720] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:43.328255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe00fcf-24dc-4988-839b-88b3fd54456d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:43.348427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe00fcf-24dc-4988-839b-88b3fd54456d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.359827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.362938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.363939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.364396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.365714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:43.366208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.369595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.389484 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1868/3721] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:43.390690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:43.403297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6de30-351a-4c8c-b765-8af6cf07b181 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.403601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.409250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.418338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.419040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.419236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.419513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.420151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.422136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.438602 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1854/3722] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:43.482936 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1869/3723] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:43 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:43.484276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:43.503762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01bc39-541d-4b00-b44a-8739eec36117 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.631548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.632636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.633185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.633600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.634705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.636881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.678946 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1855/3724] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:43 2022] POST /identity/v3/users => generated 283 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:43.680256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:43.696457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea321ed-82c7-42f3-a78b-947f90c71568 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.713658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.715242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.715242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.715242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.715982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.717487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.751668 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1870/3725] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:43.756967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:43.798888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78454010-2014-4d8b-80d0-db3e3ce28338 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.828020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83988282-86f2-4060-bfb5-bfc0bd724c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.829147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988282-86f2-4060-bfb5-bfc0bd724c95 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.830862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988282-86f2-4060-bfb5-bfc0bd724c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.831280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988282-86f2-4060-bfb5-bfc0bd724c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.832052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988282-86f2-4060-bfb5-bfc0bd724c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.837499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988282-86f2-4060-bfb5-bfc0bd724c95 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.855887 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1856/3726] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:43.859906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:43.874154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f87fb-be6a-44fc-8d7c-b29a3da53f73 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.887678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.888811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.889328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.889858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] PATH_INFO: `/v3/projects/9255d49a290f490dad4e2324875785a5/users/898ddc0cccec4bcbbb908e3d86f31198/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.923620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9255d49a290f490dad4e2324875785a5, user_id=898ddc0cccec4bcbbb908e3d86f31198, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.926719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.969548 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1871/3727] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:43 2022] PUT /identity/v3/projects/9255d49a290f490dad4e2324875785a5/users/898ddc0cccec4bcbbb908e3d86f31198/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:43.972334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6361c-e847-4585-9d38-3cd535253610 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.972718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6361c-e847-4585-9d38-3cd535253610 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.973031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6361c-e847-4585-9d38-3cd535253610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:44.039608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-31a6361c-e847-4585-9d38-3cd535253610 None None] MFA Rules not processed for user `898ddc0cccec4bcbbb908e3d86f31198`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:44.041025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-31a6361c-e847-4585-9d38-3cd535253610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:44.087512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:44.240446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:44.242872 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1857/3728] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:44.372009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f88d4-34f8-444b-931b-351b1939c40c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:44.396451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:44.397770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:44.400206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:44.400761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:44.403805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:44.408021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:44.431988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:44.494058 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1872/3729] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:44 2022] POST /identity/v3/projects => generated 367 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:44.498087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6361c-e847-4585-9d38-3cd535253610 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:44.518136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6361c-e847-4585-9d38-3cd535253610 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:44.782405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:44.783037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:44.783654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:44.783654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:44.784196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:44.786170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:44.834716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1858/3730] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:44 2022] POST /identity/v3/users => generated 290 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:44.838960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:44.855855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d2a50-3460-4c36-994d-11924fa90a68 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:44.870021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:44.870656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:44.870946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:44.871199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:44.871849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:44.873512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:44.901236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1873/3731] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:44.906529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:44.924434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd77d69-bd00-4f03-b7c6-c08854379862 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:44.935483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97d1ac-6257-4a15-9970-d5151acf81fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:44.936564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d1ac-6257-4a15-9970-d5151acf81fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:44.937001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d1ac-6257-4a15-9970-d5151acf81fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:44.937435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d1ac-6257-4a15-9970-d5151acf81fb admin admin] PATH_INFO: `/v3/projects/40ed016297474bdcaef582af10e42200/users/36dfb78cdeb74d8687e891beaa2435d7/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.011322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97d1ac-6257-4a15-9970-d5151acf81fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ed016297474bdcaef582af10e42200, user_id=36dfb78cdeb74d8687e891beaa2435d7, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:45.031446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97d1ac-6257-4a15-9970-d5151acf81fb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:45.078539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1859/3732] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:44 2022] PUT /identity/v3/projects/40ed016297474bdcaef582af10e42200/users/36dfb78cdeb74d8687e891beaa2435d7/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:45.083161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:45.099370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4827d0-f26b-4ec5-b177-cb0ba2a66a38 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:45.196343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:45.196791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:45.198155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.223661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:45.224625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:45.225073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:45.225778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.227337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:45.229768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:45.233957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:45.238479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:45.251412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1874/3733] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:45.252654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:45.266200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b279-80eb-4896-9924-ca87a027f1f5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:45.271448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024da70-8e46-4a60-ae00-a0f831db05b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:45.272123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024da70-8e46-4a60-ae00-a0f831db05b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:45.272424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024da70-8e46-4a60-ae00-a0f831db05b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:45.272729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024da70-8e46-4a60-ae00-a0f831db05b6 admin admin] PATH_INFO: `/v3/projects/40ed016297474bdcaef582af10e42200/users/36dfb78cdeb74d8687e891beaa2435d7/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.276896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024da70-8e46-4a60-ae00-a0f831db05b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ed016297474bdcaef582af10e42200, user_id=36dfb78cdeb74d8687e891beaa2435d7, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:45.286376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024da70-8e46-4a60-ae00-a0f831db05b6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:45.310968 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1875/3734] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:45 2022] PUT /identity/v3/projects/40ed016297474bdcaef582af10e42200/users/36dfb78cdeb74d8687e891beaa2435d7/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:45.324101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0db99-1c1f-43ab-90b8-bd40573a0497 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:45.324589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0db99-1c1f-43ab-90b8-bd40573a0497 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:45.325226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0db99-1c1f-43ab-90b8-bd40573a0497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.377491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1860/3735] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:45.392250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-98c0db99-1c1f-43ab-90b8-bd40573a0497 None None] MFA Rules not processed for user `36dfb78cdeb74d8687e891beaa2435d7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:45.393720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-98c0db99-1c1f-43ab-90b8-bd40573a0497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:45.398865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:45.415388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:45.528718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:45.608878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c03d3-6459-49bb-af61-b13f6c97874e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:45.615308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1876/3736] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:45.620798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0db99-1c1f-43ab-90b8-bd40573a0497 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:45.636334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0db99-1c1f-43ab-90b8-bd40573a0497 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:45.652115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:45.653076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:45.653542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:45.653974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.654812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:45.656718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:45.735622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1877/3737] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:45 2022] POST /identity/v3/projects => generated 358 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:45.740515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:45.789021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:45.822844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:45.823778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:45.824165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:45.824542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.876601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac2eac-7d70-4909-aabf-e99257ba3a6f tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:45.892416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:45.893753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:45.895439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:45.895816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:45.896399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:45.905353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:45.965752 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1878/3738] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:45 2022] POST /identity/v3/users => generated 373 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:45.969107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:45.991060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2eb675-fa13-459f-8759-7d1e54a6f42d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:45.997582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:45.999494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:46.005123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:46.005880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:46.006315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:46.006590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/713c8d28bedd49b29af2101caee3599b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:46.021047 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1861/3739] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:02:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:02:46.045528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:get_user(user_id=713c8d28bedd49b29af2101caee3599b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:46.052631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:46.069795 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1879/3740] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:02:45 2022] GET /identity/v3/users/713c8d28bedd49b29af2101caee3599b => generated 373 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:46.072119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:46.094252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769297b-2c97-4d84-81cd-1e930fd53477 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:46.118202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:46.119513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:46.119513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:46.119513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/713c8d28bedd49b29af2101caee3599b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:46.126576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_user(user_id=713c8d28bedd49b29af2101caee3599b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:46.128609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:46.260407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '713c8d28bedd49b29af2101caee3599b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.261180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '713c8d28bedd49b29af2101caee3599b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.279247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '713c8d28bedd49b29af2101caee3599b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.279840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '713c8d28bedd49b29af2101caee3599b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.300860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:46.302372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '713c8d28bedd49b29af2101caee3599b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.328527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1862/3741] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:46 2022] DELETE /identity/v3/users/713c8d28bedd49b29af2101caee3599b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:46.333086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:46.335780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:46.364169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4f057-e122-400b-8445-468dad091ef2 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:46.620177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:46.620713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:46.620999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:46.621270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/32997d5b45914709bdc28ffa7e75e317` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:46.624740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_project(project_id=32997d5b45914709bdc28ffa7e75e317)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:46.627109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:46.750342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32997d5b45914709bdc28ffa7e75e317', 'request_id': 'req-2ab68511-2759-45bb-a029-8a411c840e82'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.774074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32997d5b45914709bdc28ffa7e75e317', 'request_id': 'req-2ab68511-2759-45bb-a029-8a411c840e82'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.786433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32997d5b45914709bdc28ffa7e75e317', 'request_id': 'req-2ab68511-2759-45bb-a029-8a411c840e82'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.824353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] The token cache is being invalidate because project 32997d5b45914709bdc28ffa7e75e317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:46.824664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.828243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:46.833415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1880/3742] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:46 2022] DELETE /identity/v3/projects/32997d5b45914709bdc28ffa7e75e317 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:46.837919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:46.860953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42488121-e908-4950-817d-f8863a65fb2c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:47.067599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:47.068312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:47.068594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:47.068866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:47.069595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:47.073304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:47.170434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1863/3743] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:46 2022] POST /identity/v3/projects => generated 360 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:47.173598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:47.176465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:47.204376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68511-2759-45bb-a029-8a411c840e82 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:47.285991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:47.471755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:47.473094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:47.474477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:47.474931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:47.476114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:47.481219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:47.567253 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1881/3744] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:47 2022] POST /identity/v3/users => generated 411 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:47.571762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:47.588854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c97002-7ad7-4e32-8d59-b81683039a46 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:47.603156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:47.603858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:47.604145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:47.604422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:47.605109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:47.610735 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:47.610735 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:02:47.610735 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:02:47.615912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:47.662261 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1864/3745] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:02:47 2022] POST /identity/v3/roles => generated 239 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:47.664363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:47.678184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26add089-e9af-4d3c-8856-c2c44e04abc4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:47.699949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:47.701031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:47.705924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:47.706442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/1aeb1357e45b408da83658bb2c7e978e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:47.770980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:get_user(user_id=1aeb1357e45b408da83658bb2c7e978e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:47.780877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:47.790018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:47.798122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1882/3746] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:02:47 2022] GET /identity/v3/users/1aeb1357e45b408da83658bb2c7e978e => generated 411 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:47.805840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adf3bd-3b52-4230-b5d5-c7e44a5c0b86 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:47.819488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:47.821050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:47.821488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:47.821757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/roles/471ab75be0274f59be1a54b13fb0007d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:47.824346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:get_role(role_id=471ab75be0274f59be1a54b13fb0007d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:47.827805 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:47.827805 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:02:47.827805 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:02:47.829741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:47.838889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:47.848429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1865/3747] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:02:47 2022] GET /identity/v3/roles/471ab75be0274f59be1a54b13fb0007d => generated 239 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:47.854354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a426f-7452-4c92-a716-01ddde29ee6d tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:47.862990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:47.863741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:47.891639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:47.893521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:47.897612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:47.903945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:47.990623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:47.990867 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1883/3748] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:47 2022] POST /identity/v3/projects => generated 360 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:48.002078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0354c1e-f706-47f3-91cd-dcc05ea54b08 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:48.141617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:48.142639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:48.142924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:48.143195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/96b700224df545b49381b70a77a357c8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:48.145871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:get_project(project_id=96b700224df545b49381b70a77a357c8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:48.148527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:48.154697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:48.168671 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1866/3749] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:02:47 2022] GET /identity/v3/projects/96b700224df545b49381b70a77a357c8 => generated 360 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:48.171250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd1d09-526b-4136-aeec-a9bd7f7d3110 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:48.181490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:48.183716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:48.183716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:48.183716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/96b700224df545b49381b70a77a357c8/users/1aeb1357e45b408da83658bb2c7e978e/roles/471ab75be0274f59be1a54b13fb0007d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:48.188898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_grant(project_id=96b700224df545b49381b70a77a357c8, user_id=1aeb1357e45b408da83658bb2c7e978e, role_id=471ab75be0274f59be1a54b13fb0007d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:48.194144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:48.223828 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1884/3750] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:48 2022] PUT /identity/v3/projects/96b700224df545b49381b70a77a357c8/users/1aeb1357e45b408da83658bb2c7e978e/roles/471ab75be0274f59be1a54b13fb0007d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:48.228159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:48.244367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b7729-b8f8-4ae6-9d2c-7ec5baf97b38 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:48.626296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:48.662679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:48.663370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:48.663653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:48.663924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:48.664566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:48.673540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:48.804608 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1867/3751] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:48 2022] POST /identity/v3/projects => generated 358 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:48.807485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:48.829453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927745-7b53-4340-b417-1ca2cfeb182e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:49.098993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:49.102855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:49.104501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:49.104786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/1c31260cd4c240258759fdf946263d64` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:49.113220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:get_project(project_id=1c31260cd4c240258759fdf946263d64)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:49.123586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:49.142609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:49.143608 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1885/3752] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:02:48 2022] GET /identity/v3/projects/1c31260cd4c240258759fdf946263d64 => generated 358 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:49.161982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045103d-340d-4646-9b03-35f1a1fde36c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:49.187428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:49.196881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:49.197202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:49.197498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/1c31260cd4c240258759fdf946263d64/users/1aeb1357e45b408da83658bb2c7e978e/roles/471ab75be0274f59be1a54b13fb0007d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:49.214688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1c31260cd4c240258759fdf946263d64, user_id=1aeb1357e45b408da83658bb2c7e978e, role_id=471ab75be0274f59be1a54b13fb0007d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:49.231593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:49.306727 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1868/3753] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:49 2022] PUT /identity/v3/projects/1c31260cd4c240258759fdf946263d64/users/1aeb1357e45b408da83658bb2c7e978e/roles/471ab75be0274f59be1a54b13fb0007d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:49.307887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:49.322339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6711cb-f09e-4fe0-bbbf-0e2c3aa915b6 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:49.541690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:49.545583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:49.545801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:49.546059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/1aeb1357e45b408da83658bb2c7e978e/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:49.548562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=1aeb1357e45b408da83658bb2c7e978e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:49.552116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:49.577397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:49.683635 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1886/3754] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:02:49 2022] GET /identity/v3/users/1aeb1357e45b408da83658bb2c7e978e/projects => generated 842 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:49.689673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:49.715435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63b4dc-2458-4322-96c7-3d3c1b306e01 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:49.734519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:49.736909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:49.738804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:49.739741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/1c31260cd4c240258759fdf946263d64` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:49.744815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1c31260cd4c240258759fdf946263d64)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:49.750045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:49.955525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c31260cd4c240258759fdf946263d64', 'request_id': 'req-7e655851-73c3-4cf9-a203-2b433750f85e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:49.980402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c31260cd4c240258759fdf946263d64', 'request_id': 'req-7e655851-73c3-4cf9-a203-2b433750f85e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.016379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c31260cd4c240258759fdf946263d64', 'request_id': 'req-7e655851-73c3-4cf9-a203-2b433750f85e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.047566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] The token cache is being invalidate because project 1c31260cd4c240258759fdf946263d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:50.049518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.057789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e655851-73c3-4cf9-a203-2b433750f85e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.067726 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1869/3755] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:49 2022] DELETE /identity/v3/projects/1c31260cd4c240258759fdf946263d64 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:50.072012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:50.075056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:50.106629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eda81-44bf-4e0b-8e1e-d78e6e033126 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:50.407160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:50.407756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:50.408019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:50.408289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/96b700224df545b49381b70a77a357c8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:50.409957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_project(project_id=96b700224df545b49381b70a77a357c8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:50.415322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:50.539431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b700224df545b49381b70a77a357c8', 'request_id': 'req-a0aec126-5cca-4008-8460-d6d36be5ef24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.554964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b067ac-4a08-41f0-8ab4-aa8eef030431 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:50.555524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b067ac-4a08-41f0-8ab4-aa8eef030431 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:50.556495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b067ac-4a08-41f0-8ab4-aa8eef030431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:50.564214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b700224df545b49381b70a77a357c8', 'request_id': 'req-a0aec126-5cca-4008-8460-d6d36be5ef24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.575417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b700224df545b49381b70a77a357c8', 'request_id': 'req-a0aec126-5cca-4008-8460-d6d36be5ef24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.591901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] The token cache is being invalidate because project 96b700224df545b49381b70a77a357c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:50.592232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.593240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.598719 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1887/3756] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:50 2022] DELETE /identity/v3/projects/96b700224df545b49381b70a77a357c8 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:50.599988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:50.601584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:50.662487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-04b067ac-4a08-41f0-8ab4-aa8eef030431 None None] MFA Rules not processed for user `128155ce2ea242bd9d59ec188c57c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:50.672347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-04b067ac-4a08-41f0-8ab4-aa8eef030431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:50.770143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aec126-5cca-4008-8460-d6d36be5ef24 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:50.796426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:50.799686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:50.799973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:50.804378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/roles/471ab75be0274f59be1a54b13fb0007d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:50.810472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_role(role_id=471ab75be0274f59be1a54b13fb0007d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:50.818219 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:50.818219 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:02:50.818219 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:02:50.827346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:50.844307 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1870/3757] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:50.856552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f9525-e578-4343-a03e-70bc12704256 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:50.858207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f9525-e578-4343-a03e-70bc12704256 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:50.858207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f9525-e578-4343-a03e-70bc12704256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:50.868002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:50.903870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b03f9525-e578-4343-a03e-70bc12704256 None None] MFA Rules not processed for user `128155ce2ea242bd9d59ec188c57c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:50.907941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b03f9525-e578-4343-a03e-70bc12704256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:50.935827 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1871/3758] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:50.963951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invalidating the token cache because role 471ab75be0274f59be1a54b13fb0007d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:50.964401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.965557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d81225d8-5076-4098-9913-17a85a0e4800 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:50.968452 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1888/3759] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:50 2022] DELETE /identity/v3/roles/471ab75be0274f59be1a54b13fb0007d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:50.972500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f9525-e578-4343-a03e-70bc12704256 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:50.990895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f9525-e578-4343-a03e-70bc12704256 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:51.073887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:51.135965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:51.136795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:51.137185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:51.137618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/1aeb1357e45b408da83658bb2c7e978e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:51.141676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1aeb1357e45b408da83658bb2c7e978e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:51.144469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:51.184334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d00f8e-6aea-482c-91c4-094a7780a8ad None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:51.184334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d00f8e-6aea-482c-91c4-094a7780a8ad None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:51.185140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d00f8e-6aea-482c-91c4-094a7780a8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:51.244831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-25d00f8e-6aea-482c-91c4-094a7780a8ad None None] MFA Rules not processed for user `00e4ba59294f4529ba188e07cfc6681a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:51.246035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25d00f8e-6aea-482c-91c4-094a7780a8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:51.354505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aeb1357e45b408da83658bb2c7e978e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.355561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aeb1357e45b408da83658bb2c7e978e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.377027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aeb1357e45b408da83658bb2c7e978e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.378139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aeb1357e45b408da83658bb2c7e978e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.407982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aeb1357e45b408da83658bb2c7e978e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.447018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1872/3760] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:50 2022] DELETE /identity/v3/users/1aeb1357e45b408da83658bb2c7e978e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:51.454660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:51.461512 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1889/3761] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:51.462085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:51.605552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785bded-115a-4980-aa6d-ae306a6ef14e tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:51.615803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:51.616539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:51.616812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:51.617088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/bd46727e420547de8eba5c1bd1a6c8a8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:51.621445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bd46727e420547de8eba5c1bd1a6c8a8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:51.623453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:51.733198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd46727e420547de8eba5c1bd1a6c8a8', 'request_id': 'req-1a138293-84d0-4159-8935-b6f55ca27063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.748654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd46727e420547de8eba5c1bd1a6c8a8', 'request_id': 'req-1a138293-84d0-4159-8935-b6f55ca27063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.752005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:51.767870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd46727e420547de8eba5c1bd1a6c8a8', 'request_id': 'req-1a138293-84d0-4159-8935-b6f55ca27063'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.793347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] The token cache is being invalidate because project bd46727e420547de8eba5c1bd1a6c8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:51.793991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.795342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a138293-84d0-4159-8935-b6f55ca27063 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:51.803153 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1873/3762] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:51 2022] DELETE /identity/v3/projects/bd46727e420547de8eba5c1bd1a6c8a8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:51.804515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d00f8e-6aea-482c-91c4-094a7780a8ad None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:51.820698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d00f8e-6aea-482c-91c4-094a7780a8ad None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:51.931218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:51.932230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:51.932665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:51.934563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/898ddc0cccec4bcbbb908e3d86f31198` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:51.941696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:update_user(user_id=898ddc0cccec4bcbbb908e3d86f31198)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:51.943042 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:51.943042 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:02:51.943042 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:02:51.947382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:52.029989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.060216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invalidating the token cache because user 898ddc0cccec4bcbbb908e3d86f31198 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:52.060876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.062268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.067701 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1890/3763] 10.222.0.26 () {60 vars in 1243 bytes} [Sat Sep 17 21:02:51 2022] PATCH /identity/v3/users/898ddc0cccec4bcbbb908e3d86f31198 => generated 296 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:52.069818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48baba8-c956-4e12-a032-35a7dccf60ea None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:52.070146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48baba8-c956-4e12-a032-35a7dccf60ea None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:52.070436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48baba8-c956-4e12-a032-35a7dccf60ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:52.201363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f48baba8-c956-4e12-a032-35a7dccf60ea None None] MFA Rules not processed for user `898ddc0cccec4bcbbb908e3d86f31198`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:52.202648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f48baba8-c956-4e12-a032-35a7dccf60ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:52.214928 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1874/3764] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:52 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:02:52.216072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:52.217606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:52.239384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ee472-0aca-468f-8639-375b7847a4d0 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:52.255938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:52.256646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:52.256948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:52.257209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/898ddc0cccec4bcbbb908e3d86f31198` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:52.259827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:update_user(user_id=898ddc0cccec4bcbbb908e3d86f31198)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:52.267090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:52.353309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.376519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invalidating the token cache because user 898ddc0cccec4bcbbb908e3d86f31198 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:52.376908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.378019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b98989c5-07b7-43c1-85f8-0722e65e7f14 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.385216 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1891/3765] 10.222.0.26 () {60 vars in 1243 bytes} [Sat Sep 17 21:02:52 2022] PATCH /identity/v3/users/898ddc0cccec4bcbbb908e3d86f31198 => generated 296 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:52.386844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48baba8-c956-4e12-a032-35a7dccf60ea None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:52.411295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48baba8-c956-4e12-a032-35a7dccf60ea None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:52.426722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:52.440784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:52.441024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:52.441339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/898ddc0cccec4bcbbb908e3d86f31198` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:52.479667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:update_user(user_id=898ddc0cccec4bcbbb908e3d86f31198)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:52.480881 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:02:52.480881 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:02:52.480881 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:02:52.482232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:52.589351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.619294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invalidating the token cache because user 898ddc0cccec4bcbbb908e3d86f31198 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:52.619998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.620238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.625043 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1875/3766] 10.222.0.26 () {60 vars in 1243 bytes} [Sat Sep 17 21:02:52 2022] PATCH /identity/v3/users/898ddc0cccec4bcbbb908e3d86f31198 => generated 296 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:52.626896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fa830-1f7c-4c95-a7cd-6088287fb4ec None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:52.627326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fa830-1f7c-4c95-a7cd-6088287fb4ec None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:52.627576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fa830-1f7c-4c95-a7cd-6088287fb4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:52.700375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-181fa830-1f7c-4c95-a7cd-6088287fb4ec None None] MFA Rules not processed for user `898ddc0cccec4bcbbb908e3d86f31198`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:52.702722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-181fa830-1f7c-4c95-a7cd-6088287fb4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:52.718875 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1892/3767] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:52 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:02:52.720201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:52.721664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:52.739128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb007117-9daa-4e32-9ff5-0294a0190ce1 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:52.749200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:52.750415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:52.751857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:52.751857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/898ddc0cccec4bcbbb908e3d86f31198` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:52.754262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:update_user(user_id=898ddc0cccec4bcbbb908e3d86f31198)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:52.756506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:52.825343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.839567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invalidating the token cache because user 898ddc0cccec4bcbbb908e3d86f31198 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:52.840002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.840626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:52.849490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1876/3768] 10.222.0.26 () {60 vars in 1243 bytes} [Sat Sep 17 21:02:52 2022] PATCH /identity/v3/users/898ddc0cccec4bcbbb908e3d86f31198 => generated 296 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:02:52.861527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fa830-1f7c-4c95-a7cd-6088287fb4ec None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:52.873066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:UPDATE=2,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:52.882327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fa830-1f7c-4c95-a7cd-6088287fb4ec None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:52.897217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc26b4-ad6e-451d-9de7-c077cff05ab4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:52.902420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc26b4-ad6e-451d-9de7-c077cff05ab4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:52.903390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc26b4-ad6e-451d-9de7-c077cff05ab4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:52.903390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc26b4-ad6e-451d-9de7-c077cff05ab4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:52.904494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc26b4-ad6e-451d-9de7-c077cff05ab4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:52.907981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc26b4-ad6e-451d-9de7-c077cff05ab4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:52.993364 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1893/3769] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:02:52 2022] POST /identity/v3/users => generated 265 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:53.008535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:53.011253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:53.053226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c544c74-e699-4b94-b530-e0e1c7755365 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:53.064951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47b7b2-78b9-4c52-a5a7-dfa4fe56aa02 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:53.068757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47b7b2-78b9-4c52-a5a7-dfa4fe56aa02 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:53.069008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47b7b2-78b9-4c52-a5a7-dfa4fe56aa02 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:53.069372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47b7b2-78b9-4c52-a5a7-dfa4fe56aa02 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/ded803c9c1cf49fbab8fb252a15dbe28/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:53.215716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bb47b7b2-78b9-4c52-a5a7-dfa4fe56aa02 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ded803c9c1cf49fbab8fb252a15dbe28'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:53.230464 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1877/3770] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:02:53 2022] POST /identity/v3/users/ded803c9c1cf49fbab8fb252a15dbe28/password => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:54.235736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfbaf1-0b91-436c-aef3-962faf748371 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:54.236376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfbaf1-0b91-436c-aef3-962faf748371 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:54.236640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfbaf1-0b91-436c-aef3-962faf748371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:54.267911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2cbfbaf1-0b91-436c-aef3-962faf748371 None None] MFA Rules not processed for user `ded803c9c1cf49fbab8fb252a15dbe28`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:54.271643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2cbfbaf1-0b91-436c-aef3-962faf748371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:54.282974 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1894/3771] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:54 2022] POST /identity/v3/auth/tokens => generated 330 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:02:54.287155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47b7b2-78b9-4c52-a5a7-dfa4fe56aa02 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:54.307167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47b7b2-78b9-4c52-a5a7-dfa4fe56aa02 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:54.322942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:54.324349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:54.324810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:54.325250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:54.338304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:54.346975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:54.370890 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1878/3772] 10.222.0.26 () {56 vars in 1263 bytes} [Sat Sep 17 21:02:54 2022] GET /identity/v3/auth/tokens => generated 330 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 21:02:54.376000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfbaf1-0b91-436c-aef3-962faf748371 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:54.402587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfbaf1-0b91-436c-aef3-962faf748371 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:54.423666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:54.428101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:54.429032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:54.429586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/ded803c9c1cf49fbab8fb252a15dbe28` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:54.436650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ded803c9c1cf49fbab8fb252a15dbe28)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:54.453331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:54.573497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded803c9c1cf49fbab8fb252a15dbe28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:54.607325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded803c9c1cf49fbab8fb252a15dbe28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:54.620831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded803c9c1cf49fbab8fb252a15dbe28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:54.622325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded803c9c1cf49fbab8fb252a15dbe28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:54.623393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded803c9c1cf49fbab8fb252a15dbe28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:54.650631 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1895/3773] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:54 2022] DELETE /identity/v3/users/ded803c9c1cf49fbab8fb252a15dbe28 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:54.658105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:54.676222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98376c8a-c38e-495b-a77e-7b7a430318e9 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:54.797115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=8,keystone:SELECT=83,keystone:UPDATE=3,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:54.825800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:54.826501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:54.826712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:54.826995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:54.828203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:54.830632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:54.870285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:54.871287 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1879/3774] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:54 2022] POST /identity/v3/users => generated 364 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:54.872077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:54.887068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c99d9-c89b-4b2f-a53f-8576b2ae116b tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:54.892710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:54.893606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:54.893968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:54.894308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:54.895031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:54.896991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:54.945630 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1896/3775] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:54 2022] POST /identity/v3/projects => generated 359 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:54.949897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:54.969330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53577416-ff77-4923-b336-ea630d87233c tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:55.342747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:55.344232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:55.344800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:55.345223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/625ed348ee7d4a9b8379d314fd1cb7f2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:55.369915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:update_user(user_id=625ed348ee7d4a9b8379d314fd1cb7f2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:55.372333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:55.439887 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1880/3776] 10.222.0.26 () {60 vars in 1244 bytes} [Sat Sep 17 21:02:54 2022] PATCH /identity/v3/users/625ed348ee7d4a9b8379d314fd1cb7f2 => generated 524 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:55.444640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:55.463828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9498bf-28b3-473c-b293-546d249c3d23 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:55.482790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:55.483608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:55.484294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:55.484672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/625ed348ee7d4a9b8379d314fd1cb7f2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:55.587236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:get_user(user_id=625ed348ee7d4a9b8379d314fd1cb7f2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:55.592178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:55.594613 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1897/3777] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:02:55 2022] GET /identity/v3/users/625ed348ee7d4a9b8379d314fd1cb7f2 => generated 390 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:55.599780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:55.612483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eea67b-20e3-44b3-98f5-50dc83cc0587 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:55.618088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:55.618888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:55.619236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:55.619571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/projects/7ce9832adb6b451d9d2d3fdb9b0642ac` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:55.620808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7ce9832adb6b451d9d2d3fdb9b0642ac)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:55.622929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:55.722886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce9832adb6b451d9d2d3fdb9b0642ac', 'request_id': 'req-bc462530-8ca5-4d04-a77a-cf690c0515af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:55.730701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce9832adb6b451d9d2d3fdb9b0642ac', 'request_id': 'req-bc462530-8ca5-4d04-a77a-cf690c0515af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:55.753741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce9832adb6b451d9d2d3fdb9b0642ac', 'request_id': 'req-bc462530-8ca5-4d04-a77a-cf690c0515af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:55.770923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] The token cache is being invalidate because project 7ce9832adb6b451d9d2d3fdb9b0642ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:55.771208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:55.772008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:55.782319 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1881/3778] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:55 2022] DELETE /identity/v3/projects/7ce9832adb6b451d9d2d3fdb9b0642ac => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:55.783832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:55.786214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:55.807879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae202f40-7692-4326-8f1a-abee443fb7c4 tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:55.901235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dfb78cdeb74d8687e891beaa2435d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '40ed016297474bdcaef582af10e42200', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:55.902013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:55.902293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:55.902563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] PATH_INFO: `/v3/users/625ed348ee7d4a9b8379d314fd1cb7f2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:55.903724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorizing `identity:delete_user(user_id=625ed348ee7d4a9b8379d314fd1cb7f2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:55.905825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:55.929224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:INSERT=5,keystone:DELETE=11 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:55.994042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '625ed348ee7d4a9b8379d314fd1cb7f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.024185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '625ed348ee7d4a9b8379d314fd1cb7f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.044156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '625ed348ee7d4a9b8379d314fd1cb7f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.049874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '625ed348ee7d4a9b8379d314fd1cb7f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.050729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-593dd18c-ea92-4cf2-b6c7-3f13d349477a tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '625ed348ee7d4a9b8379d314fd1cb7f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.075680 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1898/3779] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:55 2022] DELETE /identity/v3/users/625ed348ee7d4a9b8379d314fd1cb7f2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:56.083090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:56.084926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:56.155347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:UPDATE=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:56.208399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462530-8ca5-4d04-a77a-cf690c0515af tempest-UsersV3TestJSON-1888410479 tempest-UsersV3TestJSON-1888410479-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:56.317369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:56.320202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:56.320202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:56.320202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] PATH_INFO: `/v3/users/898ddc0cccec4bcbbb908e3d86f31198` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:56.342154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=898ddc0cccec4bcbbb908e3d86f31198)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:56.345451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:56.448742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.449435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.478315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.479145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.511772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '898ddc0cccec4bcbbb908e3d86f31198'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.534941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1882/3780] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:56 2022] DELETE /identity/v3/users/898ddc0cccec4bcbbb908e3d86f31198 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:56.539998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:56.540928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:56.541427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:56.577825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:56.579138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:56.695329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1899/3781] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:56.705695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:56.707367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:56.724211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:56.726632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:56.752732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57af6da-4568-48a2-8df7-7dd769ad7604 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:56.760916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:56.761856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:56.762137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:56.762405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] PATH_INFO: `/v3/users/11f65cdf47c1489f905d517a7a4fdea5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:56.763653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f65cdf47c1489f905d517a7a4fdea5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:56.767405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:56.843469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:56.852142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e2ab-c8ed-4d16-8d56-6c2af5a846f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:56.857038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded07b29-24a8-4354-a42e-2191c9be320b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:56.858186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded07b29-24a8-4354-a42e-2191c9be320b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:56.858808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded07b29-24a8-4354-a42e-2191c9be320b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:56.859429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded07b29-24a8-4354-a42e-2191c9be320b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:56.874128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded07b29-24a8-4354-a42e-2191c9be320b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:56.876266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded07b29-24a8-4354-a42e-2191c9be320b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:56.911761 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1883/3782] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:56.911897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f65cdf47c1489f905d517a7a4fdea5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.929203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f65cdf47c1489f905d517a7a4fdea5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.940514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f65cdf47c1489f905d517a7a4fdea5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.942006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f65cdf47c1489f905d517a7a4fdea5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.943698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f65cdf47c1489f905d517a7a4fdea5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:56.963831 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1900/3783] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:56 2022] DELETE /identity/v3/users/11f65cdf47c1489f905d517a7a4fdea5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:56.969080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634915-3380-402e-876d-03708c52e955 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:56.969535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634915-3380-402e-876d-03708c52e955 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:56.970284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634915-3380-402e-876d-03708c52e955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:57.010415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b4634915-3380-402e-876d-03708c52e955 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:57.012992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4634915-3380-402e-876d-03708c52e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:57.142992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:57.144664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:57.176370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:57.178885 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1884/3784] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:57.242613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4634915-3380-402e-876d-03708c52e955 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:57.244789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4634915-3380-402e-876d-03708c52e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:57.368679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c5ce5-5426-4f4e-8c64-040737fc1ff7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:57.374455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4634915-3380-402e-876d-03708c52e955 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:57.385351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:57.386018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:57.386297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:57.386567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:57.387713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09825d9f-0779-4318-90ac-16086f829a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:57.388334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825d9f-0779-4318-90ac-16086f829a6e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:57.388608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825d9f-0779-4318-90ac-16086f829a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:57.388875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825d9f-0779-4318-90ac-16086f829a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:57.391891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:57.414084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09825d9f-0779-4318-90ac-16086f829a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:57.416839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09825d9f-0779-4318-90ac-16086f829a6e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:57.429862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1885/3785] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:02:57.533695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:57.535464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:57.547099 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1901/3786] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:02:57.719300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:57.719597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:57.719858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:57.762293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:57.763546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:57.887583 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1886/3787] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:57.905223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:57.906905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:58.065027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:58.066463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:58.087168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1494dab-1ab8-40a7-9628-7ad98d53a61e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:58.093608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:58.095687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:58.095823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:58.096286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] PATH_INFO: `/v3/projects/575b0ac677234f6ba9c5b9715cc02f63` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:58.097687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575b0ac677234f6ba9c5b9715cc02f63)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:58.100799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:58.141990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b28c1-2990-49b2-bce4-d639a4fd567f service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:58.153622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ba597-7079-4306-9f6a-2dc1cb450e51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:58.155938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba597-7079-4306-9f6a-2dc1cb450e51 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:58.157980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba597-7079-4306-9f6a-2dc1cb450e51 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:58.158570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ba597-7079-4306-9f6a-2dc1cb450e51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:58.177591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ba597-7079-4306-9f6a-2dc1cb450e51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:58.179847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ba597-7079-4306-9f6a-2dc1cb450e51 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:58.210147 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1902/3788] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:02:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:02:58.278511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:58.367614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575b0ac677234f6ba9c5b9715cc02f63', 'request_id': 'req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.394681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575b0ac677234f6ba9c5b9715cc02f63', 'request_id': 'req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.450630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575b0ac677234f6ba9c5b9715cc02f63', 'request_id': 'req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.491038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] The token cache is being invalidate because project 575b0ac677234f6ba9c5b9715cc02f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:58.491306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.493183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.501409 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1887/3789] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:58 2022] DELETE /identity/v3/projects/575b0ac677234f6ba9c5b9715cc02f63 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:58.503058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f465b8-fc27-4d8b-9e47-ce3d6946e63a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:58.503352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f465b8-fc27-4d8b-9e47-ce3d6946e63a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:58.503630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f465b8-fc27-4d8b-9e47-ce3d6946e63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:58.570544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:58.572093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:58.585021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-53f465b8-fc27-4d8b-9e47-ce3d6946e63a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:58.586896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-53f465b8-fc27-4d8b-9e47-ce3d6946e63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:58.652794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:58.740694 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1903/3790] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:02:58.745018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048da79b-ed99-45fb-a232-b2562d6ac327 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:58.745018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048da79b-ed99-45fb-a232-b2562d6ac327 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:58.745018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048da79b-ed99-45fb-a232-b2562d6ac327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:58.774674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6d4b-ef18-463a-8666-fa8dfaa8d347 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:58.789362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-048da79b-ed99-45fb-a232-b2562d6ac327 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:58.789617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-048da79b-ed99-45fb-a232-b2562d6ac327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:58.792184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:58.792853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:58.793128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:58.793396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] PATH_INFO: `/v3/projects/9255d49a290f490dad4e2324875785a5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:58.794571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9255d49a290f490dad4e2324875785a5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:58.796718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:58.812889 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1904/3791] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:58.823838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048da79b-ed99-45fb-a232-b2562d6ac327 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:58.838105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048da79b-ed99-45fb-a232-b2562d6ac327 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:58.854213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:58.855032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:58.855309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:58.855627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:58.856300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:02:58.856642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:58.859141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:58.881627 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1905/3792] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:58.890803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:58.896582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9255d49a290f490dad4e2324875785a5', 'request_id': 'req-bdf73653-bb9a-4e4f-a776-c6a5fe812675'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.912943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9255d49a290f490dad4e2324875785a5', 'request_id': 'req-bdf73653-bb9a-4e4f-a776-c6a5fe812675'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.928204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbddfc-57a9-4bb9-9dbb-4d1c23ac46b5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:58.933647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9255d49a290f490dad4e2324875785a5', 'request_id': 'req-bdf73653-bb9a-4e4f-a776-c6a5fe812675'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.956055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] The token cache is being invalidate because project 9255d49a290f490dad4e2324875785a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:02:58.956055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.956631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:58.962939 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1888/3793] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:02:58 2022] DELETE /identity/v3/projects/9255d49a290f490dad4e2324875785a5 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:58.966614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:58.969420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:59.075886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:59.110855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf73653-bb9a-4e4f-a776-c6a5fe812675 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:59.131993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:59.134479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:59.134773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:59.135057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] PATH_INFO: `/v3/users/36dfb78cdeb74d8687e891beaa2435d7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:59.140303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36dfb78cdeb74d8687e891beaa2435d7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:59.142622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:59.163792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:59.164001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:59.164249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:59.164516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:59.165156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:59.167124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:59.262024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1906/3794] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:02:58 2022] POST /identity/v3/projects => generated 373 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:02:59.265871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:59.267731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:59.275158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36dfb78cdeb74d8687e891beaa2435d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:59.275857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36dfb78cdeb74d8687e891beaa2435d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:59.298151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36dfb78cdeb74d8687e891beaa2435d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:59.298965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36dfb78cdeb74d8687e891beaa2435d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:59.353421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36dfb78cdeb74d8687e891beaa2435d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:02:59.390623 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1889/3795] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:02:58 2022] DELETE /identity/v3/users/36dfb78cdeb74d8687e891beaa2435d7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:02:59.420810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c68a-572f-4978-af50-9a2109d35c5d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:59.550964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:59.581078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:59.582743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:59.583258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:59.583739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:59.584711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:59.588169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:59.655789 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1907/3796] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:02:59 2022] POST /identity/v3/users => generated 287 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:02:59.660268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:59.661863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:59.703034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28994f-e1e4-4386-9bbd-771059385e91 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:59.715162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:59.715814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:59.716081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:59.716353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:59.717008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:59.718686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:59.742276 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1890/3797] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:59.743554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:59.756985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8d050-01f9-4c67-8ed7-e93ff7b239da admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:59.770420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9487e-96a3-4fca-8cc5-c3b8c201648f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:59.771052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9487e-96a3-4fca-8cc5-c3b8c201648f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:59.771332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9487e-96a3-4fca-8cc5-c3b8c201648f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:59.771594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9487e-96a3-4fca-8cc5-c3b8c201648f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:59.772356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9487e-96a3-4fca-8cc5-c3b8c201648f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:59.774023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9487e-96a3-4fca-8cc5-c3b8c201648f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:59.795469 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1908/3798] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:02:59.796726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:59.810729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10040460-fe23-4359-b1da-f83f2bbf175c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:59.819727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:59.820970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:59.821348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:59.822625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] PATH_INFO: `/v3/projects/d9024c27866542428191da960659ac2e/users/43e3adea586f4772b39b62323b1cc911/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:59.854804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9024c27866542428191da960659ac2e, user_id=43e3adea586f4772b39b62323b1cc911, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:59.857739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:59.894509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1891/3799] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:02:59 2022] PUT /identity/v3/projects/d9024c27866542428191da960659ac2e/users/43e3adea586f4772b39b62323b1cc911/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:02:59.896353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:59.896723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:59.897097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:59.954917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:59.956045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:00.111289 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1909/3800] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:00.118778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:00.120626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:00.357890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e51f6-ffa7-4843-8660-9b58377c2360 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:00.371024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c01b57-86f3-41c4-9d52-df278337e3cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:00.371840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01b57-86f3-41c4-9d52-df278337e3cf service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:00.372820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01b57-86f3-41c4-9d52-df278337e3cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:00.373504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01b57-86f3-41c4-9d52-df278337e3cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:00.404963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01b57-86f3-41c4-9d52-df278337e3cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:00.406803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01b57-86f3-41c4-9d52-df278337e3cf service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:00.416855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:00.418899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:00.457332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b2a9-3215-4e35-bc27-deec7c8e9712 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:00.464630 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1892/3801] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:03:00.467243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:00.468124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:00.468575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:00.469486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] PATH_INFO: `/v3/projects/40ed016297474bdcaef582af10e42200` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:00.471789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ed016297474bdcaef582af10e42200)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:00.474921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:00.698940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ed016297474bdcaef582af10e42200', 'request_id': 'req-50ca6966-914c-42ab-a956-fe19c3e5bd46'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:00.724767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ed016297474bdcaef582af10e42200', 'request_id': 'req-50ca6966-914c-42ab-a956-fe19c3e5bd46'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:00.743075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ed016297474bdcaef582af10e42200', 'request_id': 'req-50ca6966-914c-42ab-a956-fe19c3e5bd46'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:00.773155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] The token cache is being invalidate because project 40ed016297474bdcaef582af10e42200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:00.773737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:00.775226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-50ca6966-914c-42ab-a956-fe19c3e5bd46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:00.778348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1910/3802] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:00 2022] DELETE /identity/v3/projects/40ed016297474bdcaef582af10e42200 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:03:00.786314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c499da-f21e-4c05-98c4-66b52f77d9fc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:00.787427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c499da-f21e-4c05-98c4-66b52f77d9fc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:00.788498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c499da-f21e-4c05-98c4-66b52f77d9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:00.859277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0c499da-f21e-4c05-98c4-66b52f77d9fc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:00.865338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0c499da-f21e-4c05-98c4-66b52f77d9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:00.979723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:01.036983 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1893/3803] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:01.046417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a26d6-1269-4f99-8a92-b25433076b3e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.048851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a26d6-1269-4f99-8a92-b25433076b3e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.049151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a26d6-1269-4f99-8a92-b25433076b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.118080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c55a26d6-1269-4f99-8a92-b25433076b3e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:01.119360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c55a26d6-1269-4f99-8a92-b25433076b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:01.164325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c499da-f21e-4c05-98c4-66b52f77d9fc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:01.174078 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1911/3804] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:01.183570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c499da-f21e-4c05-98c4-66b52f77d9fc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:01.197349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:01.198012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.198337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.198676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.199478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:01.199720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:01.203457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:01.218079 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1894/3805] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:03:01.228893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a26d6-1269-4f99-8a92-b25433076b3e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:01.242664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a26d6-1269-4f99-8a92-b25433076b3e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:01.255355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:01.256438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.257001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.257279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.258134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:01.260673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:01.309916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:01.318736 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1912/3806] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:01 2022] POST /identity/v3/projects => generated 379 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:01.324331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baca96-738c-4a0a-be7f-ebe9f952e7f0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:01.477017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:01.478412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.478698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.478972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.479806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:01.482647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:01.520239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:01.531862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1895/3807] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:01 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:01.540540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e00f-34f6-4939-8418-999df3c06dec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:01.557564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:01.558400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.558690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.560209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.562560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:01.568007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:01.589803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:01.604813 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1913/3808] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:01.606988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644557-cc71-41b4-96e4-bd6989bd7997 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:01.613069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb777e-f49e-4b43-ad92-cfc879e06e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:01.613733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb777e-f49e-4b43-ad92-cfc879e06e81 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.614074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb777e-f49e-4b43-ad92-cfc879e06e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.614395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb777e-f49e-4b43-ad92-cfc879e06e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.615388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb777e-f49e-4b43-ad92-cfc879e06e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:01.618272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb777e-f49e-4b43-ad92-cfc879e06e81 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:01.645550 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1896/3809] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:01.647232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:01.665481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce45ca2-3023-4891-a647-492a33c89f82 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:01.698110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:01.699351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.702415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.703012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] PATH_INFO: `/v3/projects/5590cf39760f44b8a2c3bcf9fbcc7c2f/users/c2c8e009fbf8432c95319417d14cdba5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.716131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:01.777651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5590cf39760f44b8a2c3bcf9fbcc7c2f, user_id=c2c8e009fbf8432c95319417d14cdba5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:01.786697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:01.824221 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1914/3810] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:01 2022] PUT /identity/v3/projects/5590cf39760f44b8a2c3bcf9fbcc7c2f/users/c2c8e009fbf8432c95319417d14cdba5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:01.834713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96e50d-95ab-4063-99ac-be2cc115866f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:01.836195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96e50d-95ab-4063-99ac-be2cc115866f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:01.836664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96e50d-95ab-4063-99ac-be2cc115866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:01.931130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ee96e50d-95ab-4063-99ac-be2cc115866f None None] MFA Rules not processed for user `c2c8e009fbf8432c95319417d14cdba5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:01.937646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee96e50d-95ab-4063-99ac-be2cc115866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:02.253213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:02.253594 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1897/3811] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:02.273313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5d304-31ef-4441-9ea2-dd83e5fec0fe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:02.319448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b31dda-47cb-4d67-bd8e-be71165e5545 tempest-DefaultDomainTestJSON-1308242036 tempest-DefaultDomainTestJSON-1308242036-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2c8e009fbf8432c95319417d14cdba5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5590cf39760f44b8a2c3bcf9fbcc7c2f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:02.320004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b31dda-47cb-4d67-bd8e-be71165e5545 tempest-DefaultDomainTestJSON-1308242036 tempest-DefaultDomainTestJSON-1308242036-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:02.323010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b31dda-47cb-4d67-bd8e-be71165e5545 tempest-DefaultDomainTestJSON-1308242036 tempest-DefaultDomainTestJSON-1308242036-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:02.323010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b31dda-47cb-4d67-bd8e-be71165e5545 tempest-DefaultDomainTestJSON-1308242036 tempest-DefaultDomainTestJSON-1308242036-project] PATH_INFO: `/v3/domains/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:02.329069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b31dda-47cb-4d67-bd8e-be71165e5545 tempest-DefaultDomainTestJSON-1308242036 tempest-DefaultDomainTestJSON-1308242036-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:02.345512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b31dda-47cb-4d67-bd8e-be71165e5545 tempest-DefaultDomainTestJSON-1308242036 tempest-DefaultDomainTestJSON-1308242036-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:02.354810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96e50d-95ab-4063-99ac-be2cc115866f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:02.376388 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1915/3812] 10.222.0.26 () {58 vars in 1152 bytes} [Sat Sep 17 21:03:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:02.442409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:02.526291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96e50d-95ab-4063-99ac-be2cc115866f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:02.531220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:02.531897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:02.532176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:02.532460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] PATH_INFO: `/v3/users/c2c8e009fbf8432c95319417d14cdba5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:02.536644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c8e009fbf8432c95319417d14cdba5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:02.539305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:02.657971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c8e009fbf8432c95319417d14cdba5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:02.660428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c8e009fbf8432c95319417d14cdba5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:02.690299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c8e009fbf8432c95319417d14cdba5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:02.691800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c8e009fbf8432c95319417d14cdba5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:02.738156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c8e009fbf8432c95319417d14cdba5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:02.778867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1898/3813] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:02 2022] DELETE /identity/v3/users/c2c8e009fbf8432c95319417d14cdba5 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:02.783296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:02.783910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:02.783910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:02.841915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:02.843987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:03.027973 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1916/3814] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:03.039303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:03.041961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:03.208993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae9d0-28dc-4840-8c2d-256b69b9f74b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:03.214286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26621a6-4da1-4288-a290-bc416296a54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:03.215059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26621a6-4da1-4288-a290-bc416296a54e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:03.215465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26621a6-4da1-4288-a290-bc416296a54e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:03.215876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26621a6-4da1-4288-a290-bc416296a54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:03.240722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26621a6-4da1-4288-a290-bc416296a54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:03.247850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26621a6-4da1-4288-a290-bc416296a54e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:03.274923 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1899/3815] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:03.927424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:03.929653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:03.968005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880f412-42a6-4042-8ba4-0a62d6b9be30 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:03.975576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:03.977757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:03.978255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:03.978763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] PATH_INFO: `/v3/projects/5590cf39760f44b8a2c3bcf9fbcc7c2f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:03.982152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5590cf39760f44b8a2c3bcf9fbcc7c2f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:03.987482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:04.071951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5590cf39760f44b8a2c3bcf9fbcc7c2f', 'request_id': 'req-4486f018-b527-46b6-bf07-4a40c0314540'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:04.106279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5590cf39760f44b8a2c3bcf9fbcc7c2f', 'request_id': 'req-4486f018-b527-46b6-bf07-4a40c0314540'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:04.131948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5590cf39760f44b8a2c3bcf9fbcc7c2f', 'request_id': 'req-4486f018-b527-46b6-bf07-4a40c0314540'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:04.182138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] The token cache is being invalidate because project 5590cf39760f44b8a2c3bcf9fbcc7c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:04.182532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:04.183194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4486f018-b527-46b6-bf07-4a40c0314540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:04.185299 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1917/3816] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:03 2022] DELETE /identity/v3/projects/5590cf39760f44b8a2c3bcf9fbcc7c2f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:04.194308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ad404-28d4-4e6e-a161-37bb7eaa8695 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:04.194923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ad404-28d4-4e6e-a161-37bb7eaa8695 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:04.195603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ad404-28d4-4e6e-a161-37bb7eaa8695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:04.231914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3d8ad404-28d4-4e6e-a161-37bb7eaa8695 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:04.250967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3d8ad404-28d4-4e6e-a161-37bb7eaa8695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:04.484118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:04.507269 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1900/3817] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:04.515824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e536-21d4-4d1a-b5a3-f0e0dfeb7286 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:04.516462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e536-21d4-4d1a-b5a3-f0e0dfeb7286 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:04.516896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e536-21d4-4d1a-b5a3-f0e0dfeb7286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:04.561279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bdf7e536-21d4-4d1a-b5a3-f0e0dfeb7286 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:04.563519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bdf7e536-21d4-4d1a-b5a3-f0e0dfeb7286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:04.580035 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1918/3818] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:04.584365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ad404-28d4-4e6e-a161-37bb7eaa8695 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:04.605444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ad404-28d4-4e6e-a161-37bb7eaa8695 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:04.622172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:04.626834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:04.627270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:04.630370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:04.632141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:04.632455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:04.636706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:04.689659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7e536-21d4-4d1a-b5a3-f0e0dfeb7286 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:04.700745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7e536-21d4-4d1a-b5a3-f0e0dfeb7286 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:04.705039 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1901/3819] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:04.734549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:04.735217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:04.735500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:04.735775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:04.736799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:04.743017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:04.869451 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1919/3820] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:04 2022] POST /identity/v3/projects => generated 369 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:04.873703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:04.893518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4375247-f37c-4e51-a5f2-8b0bb3ab0368 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:05.289783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:05.291087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:05.291087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:05.291087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:05.291947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:05.294128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:05.348225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1902/3821] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:04 2022] POST /identity/v3/users => generated 285 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:05.362977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:05.403489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a15cc-3488-4f01-bc83-d53e17cff2d1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:05.416091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:05.416795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:05.417699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:05.417699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:05.419064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:05.420243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:05.440547 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1920/3822] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:05.441909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:05.455339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e42e3b-feba-4d07-8153-efb097bea8d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:05.463582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0aa3d3-2cf4-4e6f-87b1-e48ed3818c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:05.464742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0aa3d3-2cf4-4e6f-87b1-e48ed3818c73 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:05.465260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0aa3d3-2cf4-4e6f-87b1-e48ed3818c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:05.465775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0aa3d3-2cf4-4e6f-87b1-e48ed3818c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:05.466702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0aa3d3-2cf4-4e6f-87b1-e48ed3818c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:05.468565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0aa3d3-2cf4-4e6f-87b1-e48ed3818c73 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:05.480656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1903/3823] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:03:05.489297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:05.502205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc830-4f67-478c-b18b-d7c3cbd8fb19 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:05.507825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:05.509483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:05.509848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:05.510185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] PATH_INFO: `/v3/projects/c8f37c8e25ef41d8851dea039951e7f3/users/67a106c386164adc9fd407d4deaf3efe/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:05.555174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f37c8e25ef41d8851dea039951e7f3, user_id=67a106c386164adc9fd407d4deaf3efe, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:05.565568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:05.596136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:05.605257 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1921/3824] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:05 2022] PUT /identity/v3/projects/c8f37c8e25ef41d8851dea039951e7f3/users/67a106c386164adc9fd407d4deaf3efe/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:05.619200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb8b88-f4f2-4a75-8834-e6138372e839 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:05.621441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb8b88-f4f2-4a75-8834-e6138372e839 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:05.622452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb8b88-f4f2-4a75-8834-e6138372e839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:05.719907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f6bb8b88-f4f2-4a75-8834-e6138372e839 None None] MFA Rules not processed for user `67a106c386164adc9fd407d4deaf3efe`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:05.728369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f6bb8b88-f4f2-4a75-8834-e6138372e839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:05.856382 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1904/3825] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:05.861369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:06.026101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151b211-013e-4929-afc8-ad86e91ccee4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:06.031750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:06.032617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.032967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.033360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.034092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:06.036623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:06.083491 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1922/3826] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:05 2022] POST /identity/v3/projects => generated 367 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:06.087405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb8b88-f4f2-4a75-8834-e6138372e839 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:06.102992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb8b88-f4f2-4a75-8834-e6138372e839 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:06.182774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:06.198175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:06.199288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.199771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.200247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.201089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:06.203809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:06.252081 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1905/3827] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:06 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:06.256324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:06.270080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c6876-5c15-4e9f-905b-4be3cd0349bc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:06.282981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:06.283954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.284413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.284843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.285711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:06.289177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:06.303799 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1923/3828] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:03:06.309221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:06.334562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10c59-4691-4fc0-84ba-7e965413d739 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:06.376271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cf0fd-6e29-438e-ba5b-e1abca15080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:06.377087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cf0fd-6e29-438e-ba5b-e1abca15080c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.377484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cf0fd-6e29-438e-ba5b-e1abca15080c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.377800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cf0fd-6e29-438e-ba5b-e1abca15080c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.378465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cf0fd-6e29-438e-ba5b-e1abca15080c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:06.380479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cf0fd-6e29-438e-ba5b-e1abca15080c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:06.399396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1906/3829] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:03:06.402592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:06.416218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e78637-53b5-4c1b-8b02-d594c2f1b715 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:06.446756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e22b1-d59f-425d-8afa-888b00197262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:06.447459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e22b1-d59f-425d-8afa-888b00197262 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.447739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e22b1-d59f-425d-8afa-888b00197262 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.454727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e22b1-d59f-425d-8afa-888b00197262 admin admin] PATH_INFO: `/v3/projects/272afbc445bd4cdda59d997008659e06/users/17c35eae758943979b47b3105bc8f98f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.509906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.510171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.510873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.522530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e22b1-d59f-425d-8afa-888b00197262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272afbc445bd4cdda59d997008659e06, user_id=17c35eae758943979b47b3105bc8f98f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:06.530173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e22b1-d59f-425d-8afa-888b00197262 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:06.550177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:06.554027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:06.590073 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1924/3830] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:06 2022] PUT /identity/v3/projects/272afbc445bd4cdda59d997008659e06/users/17c35eae758943979b47b3105bc8f98f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:06.592309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.592600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.592876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.654669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] MFA Rules not processed for user `17c35eae758943979b47b3105bc8f98f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:06.655841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:06.705268 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1907/3831] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:06.717904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:06.721307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:06.853708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564d7e9-bcd3-43e9-b6de-8c55ac4fb39f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:06.888776 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1925/3832] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:06.889767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:06.896035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:06.965131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:06.968723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:06.969015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:06.969296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:06.978496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:06.994593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:07.011260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296cf4b-544a-4378-a122-cc8c8e3af567 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:07.017364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37bcc7-36d7-4a4d-8f17-441cad273b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:07.018803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37bcc7-36d7-4a4d-8f17-441cad273b56 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:07.019614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37bcc7-36d7-4a4d-8f17-441cad273b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:07.020112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37bcc7-36d7-4a4d-8f17-441cad273b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:07.022449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d37bcc7-36d7-4a4d-8f17-441cad273b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:07.040813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37bcc7-36d7-4a4d-8f17-441cad273b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:07.042739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37bcc7-36d7-4a4d-8f17-441cad273b56 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:07.068497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:07.070342 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1926/3833] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:07.070829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:07.080178 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1908/3834] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:14.085404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:14.085955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:14.085955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:14.164647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] MFA Rules not processed for user `43e3adea586f4772b39b62323b1cc911`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:14.165905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:14.435533 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1927/3835] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:14.580062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:14.582496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:14.805544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3f44a-6f99-429b-8ade-d45d564ed282 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:14.811966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:14.813158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:14.813540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:14.813824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:14.824240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:14.825736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:14.841393 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1909/3836] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:19.891480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:19.893035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:20.047622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270200c-5386-46f8-ac36-41b2ac8ec3d8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:20.054170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:20.054743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:20.055024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:20.056980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:20.202050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:20.205490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:20.227301 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1928/3837] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:20.516400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:20.526423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6739dc-cfc2-4a78-96a1-17f9d31bb233 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:20.531866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:20.532621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:20.532894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:20.533161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:20.535806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:20.559542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:20.561959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:20.599773 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1910/3838] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:03:20.888169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:20.911439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9ecee-4128-4aea-8b59-e7df9c06c7b3 service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:20.916482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:20.917689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:20.918202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:20.918704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:20.937944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:20.940186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:20.940570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:20.955571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd8d04-df66-4eda-a532-f9cc2838bf4f service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:20.971853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212a7b7-1673-4513-a1a5-a3fff38c1aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:20.972642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a7b7-1673-4513-a1a5-a3fff38c1aea service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:20.972875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a7b7-1673-4513-a1a5-a3fff38c1aea service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:20.973102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212a7b7-1673-4513-a1a5-a3fff38c1aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:20.975039 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1929/3839] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:20.980523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7212a7b7-1673-4513-a1a5-a3fff38c1aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:21.024506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212a7b7-1673-4513-a1a5-a3fff38c1aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:21.026015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212a7b7-1673-4513-a1a5-a3fff38c1aea service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:21.084574 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1911/3840] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:22.033864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:22.039725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:22.066735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:22.218841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca97da0-5df8-4e88-8d5a-b522ec80f70f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:22.219847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca97da0-5df8-4e88-8d5a-b522ec80f70f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:22.220188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca97da0-5df8-4e88-8d5a-b522ec80f70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:22.238129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455b6a0-c918-4c6d-a02d-30070a77416f service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:22.247105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2ca97da0-5df8-4e88-8d5a-b522ec80f70f None None] MFA Rules not processed for user `43e3adea586f4772b39b62323b1cc911`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:22.249453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2ca97da0-5df8-4e88-8d5a-b522ec80f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:22.255111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:22.255823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:22.256183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:22.256924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:22.259607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:22.260929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1912/3841] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:22.282321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:22.285209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:22.325530 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1930/3842] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:23.807031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:23.807833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:23.808252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:23.844616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:23.846361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:23.881943 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1913/3843] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:03:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:23.908753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:23.929593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed22d87-27b6-4e2f-917b-fb457fb299b7 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:23.947128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:23.947761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:23.948040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:23.948308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:23.972177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:23.973816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:23.987380 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1931/3844] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:25.374231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:25.376657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:25.414280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1283b94-e9fa-446c-9eff-6d8d7cd187a9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:25.469825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:25.473151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:25.475853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:25.477871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:25.484909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:25.581118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:25.585275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:25.629207 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1914/3845] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:28.987988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:29.185746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0aba63-6518-4551-b88b-a7dccdf09c34 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:29.190600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:29.191483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:29.191956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:29.192430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] PATH_INFO: `/v3/users/67a106c386164adc9fd407d4deaf3efe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:29.210578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=67a106c386164adc9fd407d4deaf3efe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:29.213395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:29.331609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67a106c386164adc9fd407d4deaf3efe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:29.359816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67a106c386164adc9fd407d4deaf3efe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:29.373681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:29.376277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:29.380241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67a106c386164adc9fd407d4deaf3efe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:29.381563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67a106c386164adc9fd407d4deaf3efe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:29.382385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ce98142-aaea-454b-8512-948e1e53b8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67a106c386164adc9fd407d4deaf3efe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:29.400968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156940c2-591e-4150-bd35-1794dace9687 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:29.409820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:29.410730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:29.411189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:29.411610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:29.415657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1932/3846] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:28 2022] DELETE /identity/v3/users/67a106c386164adc9fd407d4deaf3efe => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:29.417339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:29.420871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:29.421162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:29.421442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:29.446669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:29.448714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:29.468308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:29.469767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:29.474633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:29.484402 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1915/3847] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:03:29.585382 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1933/3848] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:29.595760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:29.597436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:29.621604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962193-03a6-47db-995f-0f5bf78bdf3f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:29.691244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:29.774319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:29.777101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:29.778773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:29.780680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:29.838152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:29.840211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:29.852875 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1916/3849] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:30.982018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:30.983617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:31.004665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8e535-2481-43b1-b40a-09c3d8a590fe None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:31.031837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:31.038055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:31.039051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:31.041306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] PATH_INFO: `/v3/projects/c8f37c8e25ef41d8851dea039951e7f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:31.043403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f37c8e25ef41d8851dea039951e7f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:31.048818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:31.217250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f37c8e25ef41d8851dea039951e7f3', 'request_id': 'req-b8abc18b-9e36-4b5d-9166-647eab00040d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.235398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f37c8e25ef41d8851dea039951e7f3', 'request_id': 'req-b8abc18b-9e36-4b5d-9166-647eab00040d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.253806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f37c8e25ef41d8851dea039951e7f3', 'request_id': 'req-b8abc18b-9e36-4b5d-9166-647eab00040d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.289405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] The token cache is being invalidate because project c8f37c8e25ef41d8851dea039951e7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:31.290080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.291647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.296051 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1934/3850] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:30 2022] DELETE /identity/v3/projects/c8f37c8e25ef41d8851dea039951e7f3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:31.300057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:31.317292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e8035-3164-4051-874c-78660122c9f0 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:31.492552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:31.493197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:31.494630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:31.495305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] PATH_INFO: `/v3/users/17c35eae758943979b47b3105bc8f98f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:31.514028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c35eae758943979b47b3105bc8f98f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:31.514028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:31.646641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c35eae758943979b47b3105bc8f98f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.647430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c35eae758943979b47b3105bc8f98f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.662362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c35eae758943979b47b3105bc8f98f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.664095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c35eae758943979b47b3105bc8f98f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.715085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c35eae758943979b47b3105bc8f98f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:31.756762 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1917/3851] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:31 2022] DELETE /identity/v3/users/17c35eae758943979b47b3105bc8f98f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:32.141693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:32.145121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:32.316490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abc18b-9e36-4b5d-9166-647eab00040d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:32.368029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:32.417433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c54641-e717-4590-89fb-8ba0ec2cb822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:32.418537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54641-e717-4590-89fb-8ba0ec2cb822 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:32.419055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54641-e717-4590-89fb-8ba0ec2cb822 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:32.419832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c54641-e717-4590-89fb-8ba0ec2cb822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:32.423686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78c54641-e717-4590-89fb-8ba0ec2cb822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:32.482129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54641-e717-4590-89fb-8ba0ec2cb822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:32.484095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c54641-e717-4590-89fb-8ba0ec2cb822 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:32.510372 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1935/3852] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:32.877345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:32.881703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:32.999133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709679d-6d6a-4447-99d1-34b5fefbb90b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:33.004503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:33.005448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.006117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.006549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] PATH_INFO: `/v3/projects/272afbc445bd4cdda59d997008659e06` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.008389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=272afbc445bd4cdda59d997008659e06)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:33.011704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:33.014069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:33.147921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272afbc445bd4cdda59d997008659e06', 'request_id': 'req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:33.150970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272afbc445bd4cdda59d997008659e06', 'request_id': 'req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:33.171247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272afbc445bd4cdda59d997008659e06', 'request_id': 'req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:33.195808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.196674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.196674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.210341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] The token cache is being invalidate because project 272afbc445bd4cdda59d997008659e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:33.215146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:33.229136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15d83e6f-2c17-4f49-ba2e-0e9495f5f357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:33.246131 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1918/3853] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:32 2022] DELETE /identity/v3/projects/272afbc445bd4cdda59d997008659e06 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:33.246577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:33.246883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:33.258343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0ae2-92e9-4325-afcf-f418c7affbad None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.258768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0ae2-92e9-4325-afcf-f418c7affbad None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.259695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f0ae2-92e9-4325-afcf-f418c7affbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.295113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fd5f0ae2-92e9-4325-afcf-f418c7affbad None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:33.301666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd5f0ae2-92e9-4325-afcf-f418c7affbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:33.340316 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1936/3854] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:03:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:33.359262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:33.360963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:33.414171 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1919/3855] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:33.417150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c866-4317-4eea-a427-4b42cc05361c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.418597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c866-4317-4eea-a427-4b42cc05361c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.418888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c866-4317-4eea-a427-4b42cc05361c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.470557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0004c866-4317-4eea-a427-4b42cc05361c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:33.471931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0004c866-4317-4eea-a427-4b42cc05361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:33.473969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b389a6d-df1c-48c7-b932-2882228173e8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:33.484317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1920/3856] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:33.498166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c866-4317-4eea-a427-4b42cc05361c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:33.499997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:33.500641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.500917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.501187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.513474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:33.516098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:33.519349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c866-4317-4eea-a427-4b42cc05361c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:33.529198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:33.529854 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1937/3857] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:33.530073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.530413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.530754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.531592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:33.531793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:33.533831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:33.555437 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1921/3858] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:33.560045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:33.588957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33434a-6ee3-4d8e-aced-a1ec2ae29741 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:33.606575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:33.609215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.609897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.612895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.614280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:33.617420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:33.696859 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1938/3859] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:33 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:33.700925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:33.726503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6a38-411d-4e14-9e7f-e6888a23e07b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:33.911325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:33.912120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:33.912482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:33.913137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:33.913894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:33.916201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:33.954721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:33.964162 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1922/3860] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:33 2022] POST /identity/v3/users => generated 285 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:33.967765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:33.982554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dbe9f-ca16-4ca2-805a-388b13bb98c4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:34.014631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:34.015262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:34.015542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:34.015826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:34.016481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:34.018174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:34.052277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:34.062595 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1939/3861] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:34.064062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494cb69-7475-4dec-ae50-aaddec1a29b5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:34.081618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8c171-efd8-4259-aacd-f2770873b75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:34.082507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8c171-efd8-4259-aacd-f2770873b75a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:34.082859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8c171-efd8-4259-aacd-f2770873b75a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:34.083256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8c171-efd8-4259-aacd-f2770873b75a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:34.085617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8c171-efd8-4259-aacd-f2770873b75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:34.091532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8c171-efd8-4259-aacd-f2770873b75a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:34.112697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:34.112898 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1923/3862] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:34.131948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c36670-b160-4025-abe7-0cd9e99c8a12 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:34.139760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:34.140881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:34.141377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:34.141900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] PATH_INFO: `/v3/projects/7ae50239f55846d58fd3db30c6aa58ca/users/b71af9a6d25d45a1a8b20627f9e74e28/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:34.179542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae50239f55846d58fd3db30c6aa58ca, user_id=b71af9a6d25d45a1a8b20627f9e74e28, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:34.179542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:34.181608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:34.236617 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1940/3863] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:34 2022] PUT /identity/v3/projects/7ae50239f55846d58fd3db30c6aa58ca/users/b71af9a6d25d45a1a8b20627f9e74e28/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:34.239969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabf001-e70f-4a13-a4b6-4af99b0d5cb6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:34.240553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabf001-e70f-4a13-a4b6-4af99b0d5cb6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:34.240905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabf001-e70f-4a13-a4b6-4af99b0d5cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:34.314624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2cabf001-e70f-4a13-a4b6-4af99b0d5cb6 None None] MFA Rules not processed for user `b71af9a6d25d45a1a8b20627f9e74e28`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:34.319325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2cabf001-e70f-4a13-a4b6-4af99b0d5cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:34.515875 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1924/3864] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:34.516777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:34.734296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9075e-e906-47de-af0f-c5ca8ad63957 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:34.760319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af2375-3ee3-40b0-ad98-fe0c6665fc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:34.763587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af2375-3ee3-40b0-ad98-fe0c6665fc34 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:34.764752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af2375-3ee3-40b0-ad98-fe0c6665fc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:34.765261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af2375-3ee3-40b0-ad98-fe0c6665fc34 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:34.766174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af2375-3ee3-40b0-ad98-fe0c6665fc34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:34.768345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af2375-3ee3-40b0-ad98-fe0c6665fc34 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:34.909816 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1941/3865] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:34 2022] POST /identity/v3/projects => generated 369 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:34.915625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabf001-e70f-4a13-a4b6-4af99b0d5cb6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:34.935491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabf001-e70f-4a13-a4b6-4af99b0d5cb6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:35.055142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4acca-83a0-41b0-b7bf-d3ffc525cbea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.055801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4acca-83a0-41b0-b7bf-d3ffc525cbea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.056229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4acca-83a0-41b0-b7bf-d3ffc525cbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.111549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:35.112223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.112682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.112958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.113249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-61a4acca-83a0-41b0-b7bf-d3ffc525cbea None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:35.113654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:35.114869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61a4acca-83a0-41b0-b7bf-d3ffc525cbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:35.116934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:35.161162 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1925/3866] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:34 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:35.166728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:35.185060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90984d-954f-4628-9641-a6888be51fe8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:35.193427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:35.194075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.194656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.195082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.202283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:35.204026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:35.217057 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1926/3867] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:35.231889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:35.243391 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1942/3868] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:03:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:35.253884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddcd7-9348-4d18-b766-dfabe4d430dc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:35.257370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4acca-83a0-41b0-b7bf-d3ffc525cbea None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:35.276658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:35.280931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.281448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.281956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] PATH_INFO: `/v3/projects/157e3530479f4c6f8d6f1c8426155b09/users/1bc26269c8c9458a9ab18f98006c580b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.304189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=157e3530479f4c6f8d6f1c8426155b09, user_id=1bc26269c8c9458a9ab18f98006c580b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:35.309322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:35.345196 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1927/3869] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:35 2022] PUT /identity/v3/projects/157e3530479f4c6f8d6f1c8426155b09/users/1bc26269c8c9458a9ab18f98006c580b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:35.350656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:35.490038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:35.497519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4acca-83a0-41b0-b7bf-d3ffc525cbea None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:35.566170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:35.588765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c97123-da1c-472f-8c94-ae5dd4066fde admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:35.615769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:35.619005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.619307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.621333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.621889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:35.629418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:35.664275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:35.665312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.665801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.666287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.673935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1928/3870] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:35.679094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:35.705858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac55271-5c66-47e6-b793-e47745c6c470 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:35.735033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a169eb-dfc3-4fc5-b7d1-be550efdd83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:35.735898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a169eb-dfc3-4fc5-b7d1-be550efdd83d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.736347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a169eb-dfc3-4fc5-b7d1-be550efdd83d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.736748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a169eb-dfc3-4fc5-b7d1-be550efdd83d admin admin] PATH_INFO: `/v3/projects/157e3530479f4c6f8d6f1c8426155b09/users/1bc26269c8c9458a9ab18f98006c580b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.740233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a169eb-dfc3-4fc5-b7d1-be550efdd83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=157e3530479f4c6f8d6f1c8426155b09, user_id=1bc26269c8c9458a9ab18f98006c580b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:35.745069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a169eb-dfc3-4fc5-b7d1-be550efdd83d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:35.764526 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1929/3871] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:35 2022] PUT /identity/v3/projects/157e3530479f4c6f8d6f1c8426155b09/users/1bc26269c8c9458a9ab18f98006c580b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:35.779207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bf3f-5059-41a6-9bee-6825fe81726f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:35.779711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bf3f-5059-41a6-9bee-6825fe81726f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:35.780197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bf3f-5059-41a6-9bee-6825fe81726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:35.802983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:35.805149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:35.816578 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1943/3872] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:35.829297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2087bf3f-5059-41a6-9bee-6825fe81726f None None] MFA Rules not processed for user `1bc26269c8c9458a9ab18f98006c580b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:35.831322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2087bf3f-5059-41a6-9bee-6825fe81726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:36.000440 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1930/3873] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:36.011323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:36.012821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:36.104091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934ef34-1af8-453e-9053-dcb1bac96fa7 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:36.179618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b409e-d781-44bb-b386-9e235ff0fba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:36.180164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b409e-d781-44bb-b386-9e235ff0fba1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:36.180446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b409e-d781-44bb-b386-9e235ff0fba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:36.180720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b409e-d781-44bb-b386-9e235ff0fba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:36.192887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b409e-d781-44bb-b386-9e235ff0fba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:36.194460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b409e-d781-44bb-b386-9e235ff0fba1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:36.207248 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1944/3874] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:36.468777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0585f-28dd-40c8-9a56-5dc8cbff9532 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:36.469069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0585f-28dd-40c8-9a56-5dc8cbff9532 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:36.469341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0585f-28dd-40c8-9a56-5dc8cbff9532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:36.473646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18abd15-f116-4ab7-a059-9b5ab073e873 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:36.474142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18abd15-f116-4ab7-a059-9b5ab073e873 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:36.474142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18abd15-f116-4ab7-a059-9b5ab073e873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:36.515568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e18abd15-f116-4ab7-a059-9b5ab073e873 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:36.516857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e18abd15-f116-4ab7-a059-9b5ab073e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:36.538073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bef0585f-28dd-40c8-9a56-5dc8cbff9532 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:36.540410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bef0585f-28dd-40c8-9a56-5dc8cbff9532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:36.541020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:36.620221 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1945/3875] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:36.621262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18abd15-f116-4ab7-a059-9b5ab073e873 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:36.675304 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1931/3876] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:36.682935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0585f-28dd-40c8-9a56-5dc8cbff9532 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:36.763115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18abd15-f116-4ab7-a059-9b5ab073e873 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:36.772647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995f93a-505c-4100-b8ef-e268e8983292 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:36.773674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f93a-505c-4100-b8ef-e268e8983292 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:36.774186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f93a-505c-4100-b8ef-e268e8983292 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:36.774716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f93a-505c-4100-b8ef-e268e8983292 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:36.783655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:36.801261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995f93a-505c-4100-b8ef-e268e8983292 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:36.803643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995f93a-505c-4100-b8ef-e268e8983292 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:36.825299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0585f-28dd-40c8-9a56-5dc8cbff9532 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:36.825641 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1946/3877] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:36.830541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:36.831167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:36.831444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:36.831764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:36.844490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:36.858331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:36.885477 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1932/3878] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:38.442865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2699766-d9fe-473c-b542-14bff01ecb01 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:38.443496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2699766-d9fe-473c-b542-14bff01ecb01 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:38.443496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2699766-d9fe-473c-b542-14bff01ecb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:38.480357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b2699766-d9fe-473c-b542-14bff01ecb01 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:38.484400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b2699766-d9fe-473c-b542-14bff01ecb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:38.504564 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1947/3879] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:38.518609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:38.533205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebd44d-9316-4d2e-a181-a9fa7a46508f service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:38.542577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e743b-f350-4fd3-a7c1-7ae030b18ef8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:38.543541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e743b-f350-4fd3-a7c1-7ae030b18ef8 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:38.543976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e743b-f350-4fd3-a7c1-7ae030b18ef8 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:38.544260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e743b-f350-4fd3-a7c1-7ae030b18ef8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:38.569536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e743b-f350-4fd3-a7c1-7ae030b18ef8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:38.572002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e743b-f350-4fd3-a7c1-7ae030b18ef8 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:38.589766 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1933/3880] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:38.958097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2699766-d9fe-473c-b542-14bff01ecb01 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:39.113413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2699766-d9fe-473c-b542-14bff01ecb01 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:39.134684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:39.136673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:39.142422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:39.142422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] PATH_INFO: `/v3/users/b71af9a6d25d45a1a8b20627f9e74e28` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:39.142422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71af9a6d25d45a1a8b20627f9e74e28)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:39.143609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:39.266614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71af9a6d25d45a1a8b20627f9e74e28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:39.288978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:39.306201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71af9a6d25d45a1a8b20627f9e74e28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:39.336576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71af9a6d25d45a1a8b20627f9e74e28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:39.340350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71af9a6d25d45a1a8b20627f9e74e28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:39.341176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71af9a6d25d45a1a8b20627f9e74e28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:39.376335 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1948/3881] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:38 2022] DELETE /identity/v3/users/b71af9a6d25d45a1a8b20627f9e74e28 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:39.380960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517406f-f270-46b6-be9d-c3c93e5f35d7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:39.381801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517406f-f270-46b6-be9d-c3c93e5f35d7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:39.382260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517406f-f270-46b6-be9d-c3c93e5f35d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:39.477455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b517406f-f270-46b6-be9d-c3c93e5f35d7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:39.490946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b517406f-f270-46b6-be9d-c3c93e5f35d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:39.605265 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1934/3882] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:39.637584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:39.639201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:39.730795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea51362-9f6c-40a1-9dac-6a11848bfb84 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:39.737146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:39.737813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:39.738091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:39.740546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:39.740826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:39.741096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:39.753675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:39.778250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:39.779732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:39.785495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:39.786939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:39.808085 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1949/3883] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:39.876763 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1935/3884] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:39.893644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:39.894984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:40.063821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd058b3-56e5-40f6-8c9c-f46ce7daebff service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:40.073487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:40.074821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:40.075682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:40.075979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:40.102442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:40.107308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:40.126966 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1950/3885] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:03:42.344143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:42.346935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:42.378952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cf078-27a3-40bb-8d1b-dd371151e72f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:42.394601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:42.397424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:42.398020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:42.400766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] PATH_INFO: `/v3/projects/7ae50239f55846d58fd3db30c6aa58ca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:42.415100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ae50239f55846d58fd3db30c6aa58ca)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:42.418781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:42.573503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ae50239f55846d58fd3db30c6aa58ca', 'request_id': 'req-4ef4b506-04b2-4442-aa3d-a82a8c43455e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:42.587608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ae50239f55846d58fd3db30c6aa58ca', 'request_id': 'req-4ef4b506-04b2-4442-aa3d-a82a8c43455e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:42.602275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ae50239f55846d58fd3db30c6aa58ca', 'request_id': 'req-4ef4b506-04b2-4442-aa3d-a82a8c43455e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:42.612003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:42.643567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] The token cache is being invalidate because project 7ae50239f55846d58fd3db30c6aa58ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:42.643878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:42.647007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:42.655243 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1936/3886] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:42 2022] DELETE /identity/v3/projects/7ae50239f55846d58fd3db30c6aa58ca => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:42.657612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:42.679499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee41331-e465-4e66-8dd5-1638a1a3fdea service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:42.889363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:42.890159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:42.890446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:42.890725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] PATH_INFO: `/v3/users/1bc26269c8c9458a9ab18f98006c580b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:42.892720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc26269c8c9458a9ab18f98006c580b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:42.895091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:42.918262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:43.035743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc26269c8c9458a9ab18f98006c580b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:43.066606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc26269c8c9458a9ab18f98006c580b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:43.090372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc26269c8c9458a9ab18f98006c580b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:43.091990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc26269c8c9458a9ab18f98006c580b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:43.092516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6be1d9c8-d1b6-4c96-9589-68f0f740ecdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc26269c8c9458a9ab18f98006c580b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:43.139142 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1951/3887] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:42 2022] DELETE /identity/v3/users/1bc26269c8c9458a9ab18f98006c580b => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:44.433301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:44.435605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:44.478424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b506-04b2-4442-aa3d-a82a8c43455e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:44.784210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:44.785274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:44.785749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:44.786321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] PATH_INFO: `/v3/projects/157e3530479f4c6f8d6f1c8426155b09` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:44.795731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=157e3530479f4c6f8d6f1c8426155b09)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:44.800258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:44.966935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157e3530479f4c6f8d6f1c8426155b09', 'request_id': 'req-f2faafa5-b2e9-402a-b374-a7ac4b26c812'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:44.976537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157e3530479f4c6f8d6f1c8426155b09', 'request_id': 'req-f2faafa5-b2e9-402a-b374-a7ac4b26c812'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:44.997832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157e3530479f4c6f8d6f1c8426155b09', 'request_id': 'req-f2faafa5-b2e9-402a-b374-a7ac4b26c812'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:45.022779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] The token cache is being invalidate because project 157e3530479f4c6f8d6f1c8426155b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:45.023066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:45.023721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f2faafa5-b2e9-402a-b374-a7ac4b26c812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:45.025510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1937/3888] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:44 2022] DELETE /identity/v3/projects/157e3530479f4c6f8d6f1c8426155b09 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:45.030804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ad754-b8eb-4176-aeb4-ba73855ad21f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.031282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ad754-b8eb-4176-aeb4-ba73855ad21f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.031368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ad754-b8eb-4176-aeb4-ba73855ad21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:45.093480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-291ad754-b8eb-4176-aeb4-ba73855ad21f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:45.097316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-291ad754-b8eb-4176-aeb4-ba73855ad21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:45.261787 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1952/3889] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:45.263550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a7d2-85e1-44f7-9c71-94c96917ca50 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.263825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a7d2-85e1-44f7-9c71-94c96917ca50 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.264090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a7d2-85e1-44f7-9c71-94c96917ca50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:45.294507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-57e5a7d2-85e1-44f7-9c71-94c96917ca50 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:45.295871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-57e5a7d2-85e1-44f7-9c71-94c96917ca50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:45.323631 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1938/3890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:45.328045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ad754-b8eb-4176-aeb4-ba73855ad21f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:45.340914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ad754-b8eb-4176-aeb4-ba73855ad21f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:45.349974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:45.353932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.354993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.355513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:45.356791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:45.356791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:45.360053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:45.383941 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1953/3891] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:45.388354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5a7d2-85e1-44f7-9c71-94c96917ca50 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:45.401551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5a7d2-85e1-44f7-9c71-94c96917ca50 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:45.444137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:45.444797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.445080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.445356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:45.446055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:45.457158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:45.551220 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1939/3892] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:45 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:45.551998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:45.573279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37606404-1d07-4d0a-b3c5-5526793592af admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:45.795497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:45.798118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.798723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.800108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:45.800970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:45.808165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:45.850382 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1954/3893] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:45 2022] POST /identity/v3/users => generated 287 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:03:45.854509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:45.882209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf0325-7594-4d82-bdde-07b36c421097 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:45.901774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:45.902409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.902682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.902945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:45.903590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:45.905187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:45.929549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:45.929947 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1940/3894] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:45.935196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:45.943008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855099d-4d15-4acd-ac07-096dcd1bda57 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:45.952204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:45.953983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.955094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.955604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:45.956579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:45.958528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:45.972751 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1955/3895] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:45.977429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:45.991668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a899ea-d393-47db-b33f-a1fe9bdbc193 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:45.998274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484d4ad-3cd4-4a70-bf2b-026177ae5c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:45.999081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d4ad-3cd4-4a70-bf2b-026177ae5c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:45.999455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d4ad-3cd4-4a70-bf2b-026177ae5c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:45.999818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d4ad-3cd4-4a70-bf2b-026177ae5c22 admin admin] PATH_INFO: `/v3/projects/97142ba057254f0f840eb8bda147beca/users/0fde0b6594d3484d9372268b2900f381/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.024762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484d4ad-3cd4-4a70-bf2b-026177ae5c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97142ba057254f0f840eb8bda147beca, user_id=0fde0b6594d3484d9372268b2900f381, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.028354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484d4ad-3cd4-4a70-bf2b-026177ae5c22 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.037094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.039057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:46.056040 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1941/3896] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:45 2022] PUT /identity/v3/projects/97142ba057254f0f840eb8bda147beca/users/0fde0b6594d3484d9372268b2900f381/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:46.073800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.073800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.073800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.101206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] MFA Rules not processed for user `0fde0b6594d3484d9372268b2900f381`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:46.102580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:46.108858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:46.169584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67297e-cb14-4cb4-8dc4-c561ef5eb5ba admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:46.210085 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1942/3897] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:46.214442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.216210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:46.265581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:46.266231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.266503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.266770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.268515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:46.280747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.283366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.295497 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1956/3898] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:03:46.296230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.307750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a207ce1-319b-4511-8e22-85cd391db092 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:46.311950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93f9cc-c823-4231-a65a-c68c2dc8d636 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:46.314198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:46.315080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.315420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.315764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] PATH_INFO: `/v3/users/128155ce2ea242bd9d59ec188c57c34b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.317288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=128155ce2ea242bd9d59ec188c57c34b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.319475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.320976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:46.321922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.322269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.322606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.323357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.326137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.370536 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1957/3899] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:46 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:46.375937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.403231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128155ce2ea242bd9d59ec188c57c34b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:46.403990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128155ce2ea242bd9d59ec188c57c34b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:46.438076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128155ce2ea242bd9d59ec188c57c34b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:46.439749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128155ce2ea242bd9d59ec188c57c34b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:46.461834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128155ce2ea242bd9d59ec188c57c34b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:46.482962 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1943/3900] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:46 2022] DELETE /identity/v3/users/128155ce2ea242bd9d59ec188c57c34b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:46.518180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2807a2d-b19d-4f3a-ab07-36ad2a11ee7c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:46.524059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:46.617095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:46.617943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.618221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.618742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.619601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.621938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.652206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1958/3901] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:46 2022] POST /identity/v3/users => generated 292 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:46.656124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.657821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:46.674087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5db86-9a75-43cd-92a1-a70a52c666fa admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:46.687328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:46.688489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.689094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.689769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.690662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.692774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.716380 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1944/3902] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:46.723530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.737206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19870b-968a-46d8-a053-0b1e850a144a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:46.763760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:46.764481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.764840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.765142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] PATH_INFO: `/v3/projects/06a66700fa6a483bbd8cb3b796b9ef9b/users/0fb1e20084894cf2b8587436fe42b554/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.783308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.793898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:46.804748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a66700fa6a483bbd8cb3b796b9ef9b, user_id=0fb1e20084894cf2b8587436fe42b554, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.810231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.832645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268156fc-a6be-4d59-b4b5-3db58a7a5d5d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:46.892981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:46.892981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:46.892981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:46.892981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] PATH_INFO: `/v3/projects/9f9877108f734092be1197672d500334` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:46.913143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9877108f734092be1197672d500334)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:46.913143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:46.913319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1959/3903] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:46 2022] PUT /identity/v3/projects/06a66700fa6a483bbd8cb3b796b9ef9b/users/0fb1e20084894cf2b8587436fe42b554/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:46.913319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:46.930999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:47.009452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9877108f734092be1197672d500334', 'request_id': 'req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:47.033994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9877108f734092be1197672d500334', 'request_id': 'req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:47.086070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9877108f734092be1197672d500334', 'request_id': 'req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:47.116035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] The token cache is being invalidate because project 9f9877108f734092be1197672d500334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:47.116445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:47.118942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cae8b50-6d88-4d8c-9459-d0625fd06a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:47.124055 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1945/3904] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:46 2022] DELETE /identity/v3/projects/9f9877108f734092be1197672d500334 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:47.129747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1d0b3-2404-4e32-8cad-d789be4629e7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.130094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1d0b3-2404-4e32-8cad-d789be4629e7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.130517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1d0b3-2404-4e32-8cad-d789be4629e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.150326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1a3a9-4a0c-456e-af53-f783911b0eac admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.162664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f7c1d0b3-2404-4e32-8cad-d789be4629e7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:47.164320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f7c1d0b3-2404-4e32-8cad-d789be4629e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:47.273424 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1946/3905] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:47.286896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1c47b-2601-4b82-9894-4c4863937963 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.287193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1c47b-2601-4b82-9894-4c4863937963 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.287483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1c47b-2601-4b82-9894-4c4863937963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.332540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a6a1c47b-2601-4b82-9894-4c4863937963 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:47.335517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a6a1c47b-2601-4b82-9894-4c4863937963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:47.340873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.341546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.341831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.342219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.342757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.346072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.357623 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1947/3906] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:47.363655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1c47b-2601-4b82-9894-4c4863937963 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.379345 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1960/3907] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:47.380419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.383428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1c47b-2601-4b82-9894-4c4863937963 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.386767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:47.388746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.389774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.390231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.390585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.391310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:47.391679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.394815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.410928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c867-44dc-446c-be6b-fe988fe5a332 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.427930 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1948/3908] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:47.431088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.445162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3007bfc-696d-41fb-9d82-bab4bdcb89fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.445908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007bfc-696d-41fb-9d82-bab4bdcb89fa admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.446192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007bfc-696d-41fb-9d82-bab4bdcb89fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.446467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007bfc-696d-41fb-9d82-bab4bdcb89fa admin admin] PATH_INFO: `/v3/projects/06a66700fa6a483bbd8cb3b796b9ef9b/users/0fb1e20084894cf2b8587436fe42b554/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.450530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37124b3-cb87-40bd-9022-b17897c34534 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.456078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.456292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3007bfc-696d-41fb-9d82-bab4bdcb89fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a66700fa6a483bbd8cb3b796b9ef9b, user_id=0fb1e20084894cf2b8587436fe42b554, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.457165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.457732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.458223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.459140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3007bfc-696d-41fb-9d82-bab4bdcb89fa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.459683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.462227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.490083 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1961/3909] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:47 2022] PUT /identity/v3/projects/06a66700fa6a483bbd8cb3b796b9ef9b/users/0fb1e20084894cf2b8587436fe42b554/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:47.495076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c59a-d060-451c-9cd7-1d76602d09e6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.495583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c59a-d060-451c-9cd7-1d76602d09e6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.496785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c59a-d060-451c-9cd7-1d76602d09e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.521640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:47.527410 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1949/3910] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:47 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:47.528787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.591198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2aa4c59a-d060-451c-9cd7-1d76602d09e6 None None] MFA Rules not processed for user `0fb1e20084894cf2b8587436fe42b554`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:47.592939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2aa4c59a-d060-451c-9cd7-1d76602d09e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:47.701442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:47.736278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34241113-e70e-46c2-bd36-97d9e0bd0df0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.753783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.756684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.757257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.758063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.761112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.768645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.804198 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1962/3911] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:03:47.809145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c59a-d060-451c-9cd7-1d76602d09e6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.823364 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1950/3912] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:47 2022] POST /identity/v3/users => generated 287 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:47.827952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.830213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c59a-d060-451c-9cd7-1d76602d09e6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.839333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fb1e20084894cf2b8587436fe42b554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a66700fa6a483bbd8cb3b796b9ef9b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.846427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.846945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.847510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.848294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:47.848719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.850930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.856798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e23d9f-255b-480b-93a2-a197eb694e62 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.880225 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1963/3913] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:47.886482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.888046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.889496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.889654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.889896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.890576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.892220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.907496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccd8e8-3667-4b7f-a2c4-28aa7151f98e tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.912495 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1951/3914] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:03:47.913618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.919361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fb1e20084894cf2b8587436fe42b554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a66700fa6a483bbd8cb3b796b9ef9b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.920115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.920510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.920881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.921653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.924096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.929589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b18cb3-cc7e-45b9-a233-532abda6265d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.937777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.938356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.938812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.938944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.939565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:47.941398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:47.962076 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1952/3915] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:47.963051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:47.979502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd020556-bcb7-484e-946d-b56410aeac7d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:47.989580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b852429-daf2-4a60-a88f-054a25667f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:47.990308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b852429-daf2-4a60-a88f-054a25667f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:47.990591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b852429-daf2-4a60-a88f-054a25667f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:47.991753 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1964/3916] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:47 2022] POST /identity/v3/projects => generated 349 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:47.993818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b852429-daf2-4a60-a88f-054a25667f84 admin admin] PATH_INFO: `/v3/projects/d2618a50d3bd4937aa66e6a1d3f785c8/users/5aea27a8d8004282b683490b9bfe525a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:47.995819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:48.025430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b852429-daf2-4a60-a88f-054a25667f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2618a50d3bd4937aa66e6a1d3f785c8, user_id=5aea27a8d8004282b683490b9bfe525a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:48.030287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b852429-daf2-4a60-a88f-054a25667f84 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:48.059602 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1953/3917] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:47 2022] PUT /identity/v3/projects/d2618a50d3bd4937aa66e6a1d3f785c8/users/5aea27a8d8004282b683490b9bfe525a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:48.071381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:48.071754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:48.072479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:48.123200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] MFA Rules not processed for user `5aea27a8d8004282b683490b9bfe525a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:48.125176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:48.154307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d8982-d44b-4f91-b20e-4560706800d1 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:48.259190 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1954/3918] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:48.267438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:48.269051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:48.272683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fb1e20084894cf2b8587436fe42b554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a66700fa6a483bbd8cb3b796b9ef9b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:48.273321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:48.273630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:48.273999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:48.274837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:48.275133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:48.276853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:48.290939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:48.309149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1965/3919] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:48.314839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:48.317011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:48.412205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3462-9d7d-4454-91cd-aad012e07595 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:48.421407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:48.423033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:48.423531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:48.424165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:48.437913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:48.440613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:48.442850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755f6d8-f6f5-40bf-8748-12f1efc707f8 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:48.448499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:48.449140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:48.452236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:48.452561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:48.476779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:48.478290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:48.497913 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1955/3920] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:03:48.501564 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1966/3921] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:51.190309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:51.191673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:51.234030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:51.316746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:51.327754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e12f6e-74f2-4561-80e6-86a3401cbb31 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:51.341391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fb1e20084894cf2b8587436fe42b554', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a66700fa6a483bbd8cb3b796b9ef9b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:51.344570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:51.344864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:51.345872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] PATH_INFO: `/v3/projects/cdbb8d7dcf10433d8ee101a0bbd2848d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:51.349685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cdbb8d7dcf10433d8ee101a0bbd2848d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:51.357494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:51.396409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2228b7a-f16d-4408-a3f0-3d4941b38d96 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:51.410871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:51.411577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:51.411859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:51.412131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:51.418250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:51.458893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:51.460822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:51.480674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbb8d7dcf10433d8ee101a0bbd2848d', 'request_id': 'req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.499247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbb8d7dcf10433d8ee101a0bbd2848d', 'request_id': 'req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.499517 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1956/3922] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:03:51.524890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbb8d7dcf10433d8ee101a0bbd2848d', 'request_id': 'req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.562789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] The token cache is being invalidate because project cdbb8d7dcf10433d8ee101a0bbd2848d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:51.563071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.563805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b03915-daa2-4eb3-98bc-266fcdb9bdc3 tempest-QuotasNegativeTest-453333034 tempest-QuotasNegativeTest-453333034-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.573629 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1967/3923] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:51 2022] DELETE /identity/v3/projects/cdbb8d7dcf10433d8ee101a0bbd2848d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:51.575414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:51.731296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32dc33-220c-4eca-9471-d624cfaba140 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:51.837540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:51.838776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:51.839210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:51.839635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] PATH_INFO: `/v3/users/0fde0b6594d3484d9372268b2900f381` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:51.841129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fde0b6594d3484d9372268b2900f381)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:51.843554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:51.911135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fde0b6594d3484d9372268b2900f381'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.911899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fde0b6594d3484d9372268b2900f381'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.935616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fde0b6594d3484d9372268b2900f381'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.937857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fde0b6594d3484d9372268b2900f381'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.943674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:51.971804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fde0b6594d3484d9372268b2900f381'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:51.992791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1957/3924] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:51 2022] DELETE /identity/v3/users/0fde0b6594d3484d9372268b2900f381 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:51.999013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056319e-de43-486e-9304-e61f1828213d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:51.999832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056319e-de43-486e-9304-e61f1828213d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:52.000381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056319e-de43-486e-9304-e61f1828213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:52.036942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c056319e-de43-486e-9304-e61f1828213d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:52.038882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c056319e-de43-486e-9304-e61f1828213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:52.122049 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1968/3925] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:52.138984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:52.141754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:52.232732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f57288-b9b1-4794-82b9-bcb4c6b39ff5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:52.236847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:52.237487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:52.237791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:52.238050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:52.254080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:52.255586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:52.266737 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1958/3926] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:52.676418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056319e-de43-486e-9304-e61f1828213d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:52.680598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c056319e-de43-486e-9304-e61f1828213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:52.704458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056319e-de43-486e-9304-e61f1828213d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:52.719533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:52.720188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:52.720463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:52.720733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] PATH_INFO: `/v3/projects/97142ba057254f0f840eb8bda147beca` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:52.721931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=97142ba057254f0f840eb8bda147beca)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:52.723859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:52.763066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:52.822545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97142ba057254f0f840eb8bda147beca', 'request_id': 'req-21ffef7b-846b-4208-8ce6-fc8000886f5c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:52.831750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97142ba057254f0f840eb8bda147beca', 'request_id': 'req-21ffef7b-846b-4208-8ce6-fc8000886f5c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:52.848640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97142ba057254f0f840eb8bda147beca', 'request_id': 'req-21ffef7b-846b-4208-8ce6-fc8000886f5c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:52.871820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] The token cache is being invalidate because project 97142ba057254f0f840eb8bda147beca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:52.872216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:52.872956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:52.874761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1969/3927] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:52 2022] DELETE /identity/v3/projects/97142ba057254f0f840eb8bda147beca => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:52.879810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:52.892028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651e42f-9755-43e1-9910-33e379f985ba service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:53.095685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:53.096594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:53.096933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:53.097262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] PATH_INFO: `/v3/users/0fb1e20084894cf2b8587436fe42b554` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:53.098920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb1e20084894cf2b8587436fe42b554)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:53.101658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:53.174712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb1e20084894cf2b8587436fe42b554'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.175458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb1e20084894cf2b8587436fe42b554'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.186585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb1e20084894cf2b8587436fe42b554'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.187269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb1e20084894cf2b8587436fe42b554'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.215965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-11deea03-b87b-4ac9-866b-18da23172dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb1e20084894cf2b8587436fe42b554'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.242890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:53.270287 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1959/3928] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:52 2022] DELETE /identity/v3/users/0fb1e20084894cf2b8587436fe42b554 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:53.428320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:53.431739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:53.456054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffef7b-846b-4208-8ce6-fc8000886f5c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:53.589641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:53.590685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:53.591055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:53.591432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] PATH_INFO: `/v3/users/5aea27a8d8004282b683490b9bfe525a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:53.593016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aea27a8d8004282b683490b9bfe525a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:53.595476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:53.685844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aea27a8d8004282b683490b9bfe525a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.714354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aea27a8d8004282b683490b9bfe525a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.729832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aea27a8d8004282b683490b9bfe525a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.730709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aea27a8d8004282b683490b9bfe525a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.731485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aea27a8d8004282b683490b9bfe525a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:53.752114 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1970/3929] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:53 2022] DELETE /identity/v3/users/5aea27a8d8004282b683490b9bfe525a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:53.759668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:53.759949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:53.760482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:53.787240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:53.789017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:53.868706 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1960/3930] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:53.880989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:53.882929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:53.935875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:53.941122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:53.949914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:53.969268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a80be-735b-4aaf-bcb8-2a75f0515b5c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:53.974729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:53.975986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:53.975986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:53.976370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] PATH_INFO: `/v3/projects/06a66700fa6a483bbd8cb3b796b9ef9b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:53.977695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a66700fa6a483bbd8cb3b796b9ef9b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:53.980823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.010072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5527fd7-0b37-46cb-af27-bb9e033df00e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:54.014756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652bd82-6b1e-47b6-837c-c30365766278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:54.015849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652bd82-6b1e-47b6-837c-c30365766278 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.016135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652bd82-6b1e-47b6-837c-c30365766278 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.016410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652bd82-6b1e-47b6-837c-c30365766278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.031678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652bd82-6b1e-47b6-837c-c30365766278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:54.034004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652bd82-6b1e-47b6-837c-c30365766278 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.067722 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1971/3931] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:54.119516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a66700fa6a483bbd8cb3b796b9ef9b', 'request_id': 'req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.138380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a66700fa6a483bbd8cb3b796b9ef9b', 'request_id': 'req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.156974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a66700fa6a483bbd8cb3b796b9ef9b', 'request_id': 'req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.201120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] The token cache is being invalidate because project 06a66700fa6a483bbd8cb3b796b9ef9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:54.201120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.204119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ec8ee877-bcac-4282-a3a8-a8efed93cdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.207149 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1961/3932] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:53 2022] DELETE /identity/v3/projects/06a66700fa6a483bbd8cb3b796b9ef9b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:54.212598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e4fb4-d20d-4a1d-b6c1-d43134dbe06a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.213016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e4fb4-d20d-4a1d-b6c1-d43134dbe06a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.213532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e4fb4-d20d-4a1d-b6c1-d43134dbe06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.271001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ce1e4fb4-d20d-4a1d-b6c1-d43134dbe06a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:54.272678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce1e4fb4-d20d-4a1d-b6c1-d43134dbe06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:54.465512 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1972/3933] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:54.470545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa2d9-8eb4-42c0-84ce-16010e7b5018 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.471195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa2d9-8eb4-42c0-84ce-16010e7b5018 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.471195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaa2d9-8eb4-42c0-84ce-16010e7b5018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.506843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cedaa2d9-8eb4-42c0-84ce-16010e7b5018 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:54.508460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cedaa2d9-8eb4-42c0-84ce-16010e7b5018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:54.520374 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1962/3934] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:54.522233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e4fb4-d20d-4a1d-b6c1-d43134dbe06a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:54.538813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e4fb4-d20d-4a1d-b6c1-d43134dbe06a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:54.544441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:54.545141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.545519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.545896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.546633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:54.547002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:54.549249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.587451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaa2d9-8eb4-42c0-84ce-16010e7b5018 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:54.587806 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1973/3935] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:54.601088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaa2d9-8eb4-42c0-84ce-16010e7b5018 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:54.609195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:54.610207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.610493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.610763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.611409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:54.613318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.693520 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1963/3936] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:54 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:54.695351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:54.707537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c133d7c-c936-4b40-8a33-6394ed1cd0cf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:54.772129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:54.772829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.773107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.773405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.774087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:54.776047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.786678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:54.788399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:54.814978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55a563-7f62-4597-bc70-f293f9faf5c4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:54.825504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1974/3937] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:54 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:54.827602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:54.834968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:54.835531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.835808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.836075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] PATH_INFO: `/v3/projects/d2618a50d3bd4937aa66e6a1d3f785c8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.837515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2618a50d3bd4937aa66e6a1d3f785c8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:54.839480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.850458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77852c6a-6293-4ebc-9aa0-4f6c93c81048 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:54.855892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:54.856848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.857320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.858190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.858913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:54.860810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.876029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1975/3938] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:54.879914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:54.891716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8345b4-5127-4fa4-a187-c84f05c49cfc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:54.893845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:54.898427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2618a50d3bd4937aa66e6a1d3f785c8', 'request_id': 'req-80084815-36b9-4759-9f6e-e96e3c8370b1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.906118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2618a50d3bd4937aa66e6a1d3f785c8', 'request_id': 'req-80084815-36b9-4759-9f6e-e96e3c8370b1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.920507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2618a50d3bd4937aa66e6a1d3f785c8', 'request_id': 'req-80084815-36b9-4759-9f6e-e96e3c8370b1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.930786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:54.934745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] The token cache is being invalidate because project d2618a50d3bd4937aa66e6a1d3f785c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:03:54.935162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.935987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-80084815-36b9-4759-9f6e-e96e3c8370b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:54.942166 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1964/3939] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:03:54 2022] DELETE /identity/v3/projects/d2618a50d3bd4937aa66e6a1d3f785c8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:03:54.943542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.943942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.944312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.966563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:54.967792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:54.968809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fa3e0-812e-4e02-acd0-9fc4404abdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:54.969427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fa3e0-812e-4e02-acd0-9fc4404abdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:54.969751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fa3e0-812e-4e02-acd0-9fc4404abdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:54.970019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fa3e0-812e-4e02-acd0-9fc4404abdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:54.970722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fa3e0-812e-4e02-acd0-9fc4404abdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:54.972301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fa3e0-812e-4e02-acd0-9fc4404abdf6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:54.976278 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1965/3940] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:54.982227 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1976/3941] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:54.991235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:54.992711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:55.008518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13527bed-3b19-4b3c-91c5-4b3e34b79bb3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.015148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85efc40a-b94d-4053-92d7-d3bb2993d337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.015846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc40a-b94d-4053-92d7-d3bb2993d337 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.016123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc40a-b94d-4053-92d7-d3bb2993d337 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.016396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85efc40a-b94d-4053-92d7-d3bb2993d337 admin admin] PATH_INFO: `/v3/projects/44687313f13747878e0369af9a9ef6f2/users/1032dca1542443a9a0ae30f1bbfb2d6e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.034658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efc40a-b94d-4053-92d7-d3bb2993d337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44687313f13747878e0369af9a9ef6f2, user_id=1032dca1542443a9a0ae30f1bbfb2d6e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.037606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85efc40a-b94d-4053-92d7-d3bb2993d337 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.054438 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1966/3942] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:54 2022] PUT /identity/v3/projects/44687313f13747878e0369af9a9ef6f2/users/1032dca1542443a9a0ae30f1bbfb2d6e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:55.065158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.065417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae7682-4f85-457e-a905-b92cae6c2609 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.065582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.065705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae7682-4f85-457e-a905-b92cae6c2609 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.065809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.065903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae7682-4f85-457e-a905-b92cae6c2609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.088331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-34ae7682-4f85-457e-a905-b92cae6c2609 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:55.089264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-34ae7682-4f85-457e-a905-b92cae6c2609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:55.115121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] MFA Rules not processed for user `1032dca1542443a9a0ae30f1bbfb2d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:55.117704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:55.182077 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1967/3943] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:55.192159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae7682-4f85-457e-a905-b92cae6c2609 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.203187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae7682-4f85-457e-a905-b92cae6c2609 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.208531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.209533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.209533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.209805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.210353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:03:55.210647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.212375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.223604 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1977/3944] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:55.224920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.228973 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1968/3945] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:03:55.229361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:55.230245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.240778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb039b-5b96-43df-a895-bd85026c5e2b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.245780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.246458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.246797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.247263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.247986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.251157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.288539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1969/3946] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:03:55 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:03:55.289672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.306862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28047000-dcc5-49a8-9476-8721a30f0674 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.363664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.364400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.364801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.365137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.366534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00629497-bf53-43de-abdb-a3601170c1fa admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.371352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.372042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.372297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.372560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.373184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.375499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.408867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:55.409636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:55.421504 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1970/3947] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:03:55 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:03:55.425560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.435269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c227c38-e033-4b3c-8ab9-06f69f529b42 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.435989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.437378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.440986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.441804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.442175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.442452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.443492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.445335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.447373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1978/3948] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:55.457505 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1971/3949] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:03:55.460929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.472419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c79dcf-3986-46fd-8e35-b0f166ed14a9 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.478408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee01f1-028e-4a01-846c-8660dedc2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.479102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee01f1-028e-4a01-846c-8660dedc2f85 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.479443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee01f1-028e-4a01-846c-8660dedc2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.479798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee01f1-028e-4a01-846c-8660dedc2f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.480549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee01f1-028e-4a01-846c-8660dedc2f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.482462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee01f1-028e-4a01-846c-8660dedc2f85 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.494705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1979/3950] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:03:55.498994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.509593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533f2a-da35-409a-abb1-ff86c33421af admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.514645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.515305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.515579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.515855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] PATH_INFO: `/v3/projects/6fd077de53b249ba95f26480a1786123/users/636ed74dc08c44dfaa1496a8016c3839/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.531969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd077de53b249ba95f26480a1786123, user_id=636ed74dc08c44dfaa1496a8016c3839, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.534662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.555433 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1972/3951] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:03:55 2022] PUT /identity/v3/projects/6fd077de53b249ba95f26480a1786123/users/636ed74dc08c44dfaa1496a8016c3839/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:55.566686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945a16f-f059-4691-befc-c859769cd3d8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.567132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945a16f-f059-4691-befc-c859769cd3d8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.567409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945a16f-f059-4691-befc-c859769cd3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.617211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2945a16f-f059-4691-befc-c859769cd3d8 None None] MFA Rules not processed for user `636ed74dc08c44dfaa1496a8016c3839`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:55.618589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2945a16f-f059-4691-befc-c859769cd3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:55.737673 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1980/3952] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:55.815235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:55.817703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:55.901606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbc47-c8d6-47aa-b8c2-c56c509eb78b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:55.907290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169fd48-d00a-4667-8570-f94e8a49e3dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:55.907921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169fd48-d00a-4667-8570-f94e8a49e3dc service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:55.908197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169fd48-d00a-4667-8570-f94e8a49e3dc service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:55.908469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169fd48-d00a-4667-8570-f94e8a49e3dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:55.917755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169fd48-d00a-4667-8570-f94e8a49e3dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:55.919742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169fd48-d00a-4667-8570-f94e8a49e3dc service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:55.931217 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1973/3953] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:03:57.610547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945a16f-f059-4691-befc-c859769cd3d8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:57.745226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945a16f-f059-4691-befc-c859769cd3d8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:57.764328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:57.765355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:57.765894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:57.766354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] PATH_INFO: `/v3/users/636ed74dc08c44dfaa1496a8016c3839` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:57.767781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=636ed74dc08c44dfaa1496a8016c3839)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:57.769992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:57.886238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '636ed74dc08c44dfaa1496a8016c3839'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:57.903915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:57.932461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '636ed74dc08c44dfaa1496a8016c3839'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:57.963449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '636ed74dc08c44dfaa1496a8016c3839'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:57.964525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '636ed74dc08c44dfaa1496a8016c3839'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:57.965251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2c2a004e-6409-4e75-b177-f5ae3cc22626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '636ed74dc08c44dfaa1496a8016c3839'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:03:58.008899 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1981/3954] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:03:57 2022] DELETE /identity/v3/users/636ed74dc08c44dfaa1496a8016c3839 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:03:58.013912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:58.017137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:58.017683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:58.070117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:58.071505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:58.159798 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1974/3955] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:58.194899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:58.195313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:58.195874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:58.230124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:03:58.231698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:58.256372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:58.258790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:58.335665 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1982/3956] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:03:58.357111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:03:58.358627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:03:58.404776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:58.421218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a2666-6b68-4f85-9327-2032a5b28838 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:58.429448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a4d7-4517-4336-9f6e-b2a9ff681a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:58.430500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a4d7-4517-4336-9f6e-b2a9ff681a6a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:58.430872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a4d7-4517-4336-9f6e-b2a9ff681a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:58.431244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a4d7-4517-4336-9f6e-b2a9ff681a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:58.447765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878a4d7-4517-4336-9f6e-b2a9ff681a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:58.449402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878a4d7-4517-4336-9f6e-b2a9ff681a6a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:58.472752 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1975/3957] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:03:58.530117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8289331-95d2-4e78-a8a2-b4ce4e4ed266 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:03:58.545486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:03:58.549380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:03:58.549380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:03:58.549662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:03:58.566184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:03:58.567985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:03:58.607011 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1983/3958] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:03:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:00.190345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:00.190905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:00.190905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:00.226841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:00.228460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:00.248147 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1976/3959] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:00.288233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:00.304911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26809499-99f0-4a65-9913-6e0cd99f621a service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:00.316789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf39242-7d98-46e3-8924-3b53b2f6bdbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:00.317920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf39242-7d98-46e3-8924-3b53b2f6bdbf service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:00.318424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf39242-7d98-46e3-8924-3b53b2f6bdbf service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:00.319161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf39242-7d98-46e3-8924-3b53b2f6bdbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:00.326151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:00.328728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:00.338089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf39242-7d98-46e3-8924-3b53b2f6bdbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:00.343677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf39242-7d98-46e3-8924-3b53b2f6bdbf service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:00.356394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3195adc-67ef-43d9-b9c3-96d6695ea988 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:00.363806 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1984/3960] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:00.364027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:00.364665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:00.365009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:00.365520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] PATH_INFO: `/v3/projects/6fd077de53b249ba95f26480a1786123` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:00.367051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd077de53b249ba95f26480a1786123)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:00.368831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:00.473433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd077de53b249ba95f26480a1786123', 'request_id': 'req-4c240bf8-1008-4112-a388-43b2fba6449f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:00.491903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd077de53b249ba95f26480a1786123', 'request_id': 'req-4c240bf8-1008-4112-a388-43b2fba6449f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:00.525187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd077de53b249ba95f26480a1786123', 'request_id': 'req-4c240bf8-1008-4112-a388-43b2fba6449f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:00.547752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] The token cache is being invalidate because project 6fd077de53b249ba95f26480a1786123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:00.548333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:00.549342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4c240bf8-1008-4112-a388-43b2fba6449f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:00.556263 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1977/3961] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:00 2022] DELETE /identity/v3/projects/6fd077de53b249ba95f26480a1786123 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:00.558040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03c8f-f43d-4b34-abcb-523069c703cf None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:00.558329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03c8f-f43d-4b34-abcb-523069c703cf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:00.558605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03c8f-f43d-4b34-abcb-523069c703cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:00.601268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-27e03c8f-f43d-4b34-abcb-523069c703cf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:00.604593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-27e03c8f-f43d-4b34-abcb-523069c703cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:00.696565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:00.717269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:00.717575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:00.717917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:00.754149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1985/3962] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:00.762506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21413b05-0529-46bd-b86f-d3871a1188c3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:00.762830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21413b05-0529-46bd-b86f-d3871a1188c3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:00.771265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21413b05-0529-46bd-b86f-d3871a1188c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:00.786595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:00.792571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:00.856119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-21413b05-0529-46bd-b86f-d3871a1188c3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:00.857359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-21413b05-0529-46bd-b86f-d3871a1188c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:00.904976 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1986/3963] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:00.906145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21413b05-0529-46bd-b86f-d3871a1188c3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:00.927893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21413b05-0529-46bd-b86f-d3871a1188c3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:00.944421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:00.945063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:00.945343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:00.947404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:00.948048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:00.948362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:00.950035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:00.995194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1987/3964] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:04:00.996302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:01.019173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e77e88-4045-47fd-ab13-879571271e91 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:01.037400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:01.038062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:01.038347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:01.038620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:01.039271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:01.041558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:01.062183 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1978/3965] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:01.101438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:01.107444 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1988/3966] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:00 2022] POST /identity/v3/projects => generated 395 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:01.108975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:01.110780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:01.349858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48a7b9-bfa4-486e-a240-700f762e2c82 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:01.366893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:01.431914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a78c-317d-4cda-9a9a-b88d07b1afcd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:01.460859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:01.461907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:01.462406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:01.462897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:01.464553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:01.466789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:01.534571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1989/3967] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:01 2022] POST /identity/v3/users => generated 298 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:01.540586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:01.565243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0eae71-6a6b-4069-bca1-98ac94f5de5e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:01.576010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be9901-dd48-4ffd-a0ef-2a5680ffbaa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:01.576620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be9901-dd48-4ffd-a0ef-2a5680ffbaa1 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:01.576902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be9901-dd48-4ffd-a0ef-2a5680ffbaa1 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:01.577177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be9901-dd48-4ffd-a0ef-2a5680ffbaa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:01.586052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:01.586795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:01.587142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:01.587484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:01.588156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:01.592899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:01.618346 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1990/3968] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:01.622428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:01.642608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746d35b-2225-4888-89f1-3cbc955a37a1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:01.655956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:01.656599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:01.656876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:01.657150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:01.657843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:01.659712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:01.708225 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1991/3969] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:04:01.718642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:01.744195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e28977-de44-4c43-8d34-715d1c277bdd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:01.756033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be9901-dd48-4ffd-a0ef-2a5680ffbaa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:01.756325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917e49c-67fb-4ed6-95c2-18c3b2e5c61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:01.756835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917e49c-67fb-4ed6-95c2-18c3b2e5c61f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:01.757116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917e49c-67fb-4ed6-95c2-18c3b2e5c61f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:01.757806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917e49c-67fb-4ed6-95c2-18c3b2e5c61f admin admin] PATH_INFO: `/v3/projects/651b69d82def44e9b63b45e9dccb9ce4/users/ff8b3ef6725c4e3b8947604fe1867889/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:01.758068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be9901-dd48-4ffd-a0ef-2a5680ffbaa1 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:01.776434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1979/3970] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:01.796506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917e49c-67fb-4ed6-95c2-18c3b2e5c61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=651b69d82def44e9b63b45e9dccb9ce4, user_id=ff8b3ef6725c4e3b8947604fe1867889, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:01.801133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917e49c-67fb-4ed6-95c2-18c3b2e5c61f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:01.845348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1992/3971] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:01 2022] PUT /identity/v3/projects/651b69d82def44e9b63b45e9dccb9ce4/users/ff8b3ef6725c4e3b8947604fe1867889/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:01.859186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:01.860722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:01.861213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:02.015831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] MFA Rules not processed for user `ff8b3ef6725c4e3b8947604fe1867889`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:02.017621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:02.186446 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1980/3972] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:02.679166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49f9ec-45c4-48b8-83fd-c02d9f161e17 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:02.680315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49f9ec-45c4-48b8-83fd-c02d9f161e17 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:02.680833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49f9ec-45c4-48b8-83fd-c02d9f161e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:02.727825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0d49f9ec-45c4-48b8-83fd-c02d9f161e17 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:02.729113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d49f9ec-45c4-48b8-83fd-c02d9f161e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:02.820516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:02.823104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:02.837839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:02.866841 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1993/3973] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:02.887610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49f9ec-45c4-48b8-83fd-c02d9f161e17 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:02.917085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:03.004154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117ca5-d15b-4443-91ad-b8e22af30ccc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:03.027457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7a7e0-97b7-4e10-8381-30f9f8d4fe6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:03.028361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a7e0-97b7-4e10-8381-30f9f8d4fe6e service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:03.029284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a7e0-97b7-4e10-8381-30f9f8d4fe6e service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:03.029813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a7e0-97b7-4e10-8381-30f9f8d4fe6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:03.055326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7a7e0-97b7-4e10-8381-30f9f8d4fe6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:03.057387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7a7e0-97b7-4e10-8381-30f9f8d4fe6e service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:03.063928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49f9ec-45c4-48b8-83fd-c02d9f161e17 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:03.078846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:03.079893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:03.080414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:03.080865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:03.093130 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1981/3974] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:02 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:04:03.100292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:03.104213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:03.138431 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1994/3975] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:04:03.227525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9b8d4-6b88-4b49-bf10-a11e50906c08 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:03.230454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9b8d4-6b88-4b49-bf10-a11e50906c08 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:03.230683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9b8d4-6b88-4b49-bf10-a11e50906c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:03.351333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-87c9b8d4-6b88-4b49-bf10-a11e50906c08 None None] MFA Rules not processed for user `ff8b3ef6725c4e3b8947604fe1867889`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:03.355270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-87c9b8d4-6b88-4b49-bf10-a11e50906c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:03.448480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1982/3976] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:03.552218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:03.553797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:03.743197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042b470-90aa-4870-8581-623c0e40fe7b service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:03.750755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:03.751828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:03.752327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:03.752907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] PATH_INFO: `/v3/users/7fc9559a966444d98e54edb4f948ae1a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:03.754442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc9559a966444d98e54edb4f948ae1a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:03.756818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:03.916207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc9559a966444d98e54edb4f948ae1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:03.960678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc9559a966444d98e54edb4f948ae1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:03.978736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc9559a966444d98e54edb4f948ae1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:03.979655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc9559a966444d98e54edb4f948ae1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:03.980507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc9559a966444d98e54edb4f948ae1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:04.013838 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1995/3977] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:03 2022] DELETE /identity/v3/users/7fc9559a966444d98e54edb4f948ae1a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:04.018802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:04.019378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:04.019514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:04.054688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:04.056197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:04.235302 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1983/3978] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:04.276832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:04.279248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:04.317971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:04.559136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9ce9d-021b-4dc3-93ef-32ab9c6c56bd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:04.573436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867af3e-a46e-4134-b293-fbb9a2a09c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:04.575323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867af3e-a46e-4134-b293-fbb9a2a09c7d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:04.575847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867af3e-a46e-4134-b293-fbb9a2a09c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:04.576131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867af3e-a46e-4134-b293-fbb9a2a09c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:04.608900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867af3e-a46e-4134-b293-fbb9a2a09c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:04.610434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867af3e-a46e-4134-b293-fbb9a2a09c7d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:04.634948 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1996/3979] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:05.229039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:05.231130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:05.293606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9132c-5100-4fd9-b2d6-b6fc6b93855f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:05.322071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:05.324021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:05.325267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:05.326537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] PATH_INFO: `/v3/users/ff8b3ef6725c4e3b8947604fe1867889` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:05.336914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8b3ef6725c4e3b8947604fe1867889)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:05.348461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:05.516548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8b3ef6725c4e3b8947604fe1867889'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:05.518537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8b3ef6725c4e3b8947604fe1867889'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:05.554253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8b3ef6725c4e3b8947604fe1867889'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:05.555849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8b3ef6725c4e3b8947604fe1867889'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:05.598533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8b3ef6725c4e3b8947604fe1867889'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:05.633346 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1984/3980] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:05 2022] DELETE /identity/v3/users/ff8b3ef6725c4e3b8947604fe1867889 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:05.639194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:05.640168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:05.641263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:05.664607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:05.665946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:05.774759 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1997/3981] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:05.819300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:05.820957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:05.927187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:05.975226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:05.976973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:05.989602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736210f6-d278-4820-bb0e-c983806eeccb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:06.002190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91bd47-b207-498f-85b5-b76ec5f8b3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:06.002858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91bd47-b207-498f-85b5-b76ec5f8b3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:06.003140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91bd47-b207-498f-85b5-b76ec5f8b3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:06.003410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91bd47-b207-498f-85b5-b76ec5f8b3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:06.006503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b108d5-32d5-4ff0-862b-bc5e43ba299a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:06.025284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:06.026419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:06.028098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:06.028709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] PATH_INFO: `/v3/projects/0d42cba2664646f6adb4202574b7fa5a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:06.039375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d42cba2664646f6adb4202574b7fa5a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:06.041752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:06.042172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91bd47-b207-498f-85b5-b76ec5f8b3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:06.043692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91bd47-b207-498f-85b5-b76ec5f8b3c9 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:06.077381 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1985/3982] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:06.171623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d42cba2664646f6adb4202574b7fa5a', 'request_id': 'req-f782b266-a901-4397-8f00-193071a66dae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:06.190933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d42cba2664646f6adb4202574b7fa5a', 'request_id': 'req-f782b266-a901-4397-8f00-193071a66dae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:06.208551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d42cba2664646f6adb4202574b7fa5a', 'request_id': 'req-f782b266-a901-4397-8f00-193071a66dae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:06.233998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] The token cache is being invalidate because project 0d42cba2664646f6adb4202574b7fa5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:06.234891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:06.235781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f782b266-a901-4397-8f00-193071a66dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:06.240000 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1998/3983] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:05 2022] DELETE /identity/v3/projects/0d42cba2664646f6adb4202574b7fa5a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:06.245370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91746bac-a885-4902-8e03-69847aacbb7d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:06.246016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91746bac-a885-4902-8e03-69847aacbb7d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:06.246016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91746bac-a885-4902-8e03-69847aacbb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:06.308038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-91746bac-a885-4902-8e03-69847aacbb7d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:06.309327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-91746bac-a885-4902-8e03-69847aacbb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:06.439961 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1986/3984] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:06.443241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e713fb1-d13a-49e3-9942-f5fc61aa22f7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:06.445431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e713fb1-d13a-49e3-9942-f5fc61aa22f7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:06.446066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e713fb1-d13a-49e3-9942-f5fc61aa22f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:06.509739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0e713fb1-d13a-49e3-9942-f5fc61aa22f7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:06.511333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0e713fb1-d13a-49e3-9942-f5fc61aa22f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:06.535630 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 1999/3985] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:06.542145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91746bac-a885-4902-8e03-69847aacbb7d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:06.557412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91746bac-a885-4902-8e03-69847aacbb7d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:06.572030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:06.573111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:06.575388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:06.575836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:06.576804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:06.580413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:06.582906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:06.617298 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1987/3986] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:06.618661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e713fb1-d13a-49e3-9942-f5fc61aa22f7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:06.635828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e713fb1-d13a-49e3-9942-f5fc61aa22f7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:06.643347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:06.645130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:06.645891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:06.646653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:06.647812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:06.650493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:06.703151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:06.703578 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2000/3987] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:06 2022] POST /identity/v3/projects => generated 389 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:06.721283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7eb7-666f-4576-b13d-e76484e4a38e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:06.900642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:06.908070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:06.908351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:06.908801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:06.909674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:06.914674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:06.986778 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1988/3988] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:06 2022] POST /identity/v3/users => generated 295 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:06.989448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:06.997604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:07.014963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769ccd5-6a1a-45b3-b149-43249bcc2a3d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:07.049026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:07.049693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.050969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.051249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.053359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:07.060344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:07.093922 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2001/3989] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:07.095295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:07.108280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1f88-eb0f-43c6-aef6-dc61e29e886f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:07.148174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99766027-0aee-465f-9f30-a7c3cdc5190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:07.148925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99766027-0aee-465f-9f30-a7c3cdc5190c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.149208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99766027-0aee-465f-9f30-a7c3cdc5190c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.155796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99766027-0aee-465f-9f30-a7c3cdc5190c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.156738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99766027-0aee-465f-9f30-a7c3cdc5190c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:07.158462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99766027-0aee-465f-9f30-a7c3cdc5190c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:07.231554 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1989/3990] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:07.237597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:07.286378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032beef5-ca53-4c37-b009-84c042b112dc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:07.292491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:07.293279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.295181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.295526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] PATH_INFO: `/v3/projects/a1f37517f6b44de988179cf62fd527a9/users/456546fdd6cc4081ac3eff91382b717f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.357821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1f37517f6b44de988179cf62fd527a9, user_id=456546fdd6cc4081ac3eff91382b717f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:07.378117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:07.412853 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2002/3991] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:07 2022] PUT /identity/v3/projects/a1f37517f6b44de988179cf62fd527a9/users/456546fdd6cc4081ac3eff91382b717f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:07.415597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.416118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.416243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.462763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:07.464642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:07.651177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:07.784663 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1990/3992] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:07.872385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:07.875524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.084295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6318f-c4c2-44a2-b8a9-8b2c8bc755f0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.100142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e3a92-24f7-434f-b18f-7de6d397b627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.101491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3a92-24f7-434f-b18f-7de6d397b627 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.106991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3a92-24f7-434f-b18f-7de6d397b627 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.107466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3a92-24f7-434f-b18f-7de6d397b627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.132064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e3a92-24f7-434f-b18f-7de6d397b627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.135601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e3a92-24f7-434f-b18f-7de6d397b627 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.172000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.174666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.209582 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2003/3993] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:08.219558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f6e47-5ae6-4c03-961a-c4d93d00264a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.240691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.241630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.242188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.242560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] PATH_INFO: `/v3/projects/651b69d82def44e9b63b45e9dccb9ce4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.243873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=651b69d82def44e9b63b45e9dccb9ce4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.245978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.425903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651b69d82def44e9b63b45e9dccb9ce4', 'request_id': 'req-5a9f477c-27bf-45ea-a3bc-12db34fa9053'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:08.440191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651b69d82def44e9b63b45e9dccb9ce4', 'request_id': 'req-5a9f477c-27bf-45ea-a3bc-12db34fa9053'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:08.488045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651b69d82def44e9b63b45e9dccb9ce4', 'request_id': 'req-5a9f477c-27bf-45ea-a3bc-12db34fa9053'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:08.517705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] The token cache is being invalidate because project 651b69d82def44e9b63b45e9dccb9ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:08.518474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:08.519614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a9f477c-27bf-45ea-a3bc-12db34fa9053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:08.528340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a5c0b-204b-4c9e-93fe-11fa4fdc8ee2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.529070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a5c0b-204b-4c9e-93fe-11fa4fdc8ee2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.530192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a5c0b-204b-4c9e-93fe-11fa4fdc8ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.533193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1991/3994] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:08 2022] DELETE /identity/v3/projects/651b69d82def44e9b63b45e9dccb9ce4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:08.579926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2e2a5c0b-204b-4c9e-93fe-11fa4fdc8ee2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.584042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2e2a5c0b-204b-4c9e-93fe-11fa4fdc8ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.848868 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2004/3995] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.861607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2dea8-cf8a-473c-b69d-b71c3f3d93dd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.861949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2dea8-cf8a-473c-b69d-b71c3f3d93dd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.862224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2dea8-cf8a-473c-b69d-b71c3f3d93dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.934092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-01c2dea8-cf8a-473c-b69d-b71c3f3d93dd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.938710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-01c2dea8-cf8a-473c-b69d-b71c3f3d93dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.968402 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1992/3996] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.973868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a5c0b-204b-4c9e-93fe-11fa4fdc8ee2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.986297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a5c0b-204b-4c9e-93fe-11fa4fdc8ee2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.005195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.005899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.006186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.006455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.007122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:09.007791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.030888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.060187 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2005/3997] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:09.064593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2dea8-cf8a-473c-b69d-b71c3f3d93dd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.088727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2dea8-cf8a-473c-b69d-b71c3f3d93dd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.117862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.118760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.119209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.119637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.120460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.123662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.180638 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1993/3998] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.185074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.208066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3cd5c-2554-4921-9cff-55263ad1c868 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.362465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:09.385949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.386853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.387361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.387866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.388988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.395198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.449824 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2006/3999] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.453810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.467700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed280c-b21e-4537-9f45-8ce558e2d39d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.475914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.476802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.477356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.477817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.479869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.481912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.504475 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1994/4000] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.505428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.520923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3928826-3abd-478c-9b98-a1b6b656922c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.526771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e5cf4-87d9-4c1d-8dc8-fc8c348f8168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.527533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e5cf4-87d9-4c1d-8dc8-fc8c348f8168 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.528229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e5cf4-87d9-4c1d-8dc8-fc8c348f8168 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.528601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e5cf4-87d9-4c1d-8dc8-fc8c348f8168 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.529619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e5cf4-87d9-4c1d-8dc8-fc8c348f8168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.533746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e5cf4-87d9-4c1d-8dc8-fc8c348f8168 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.553161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2007/4001] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:04:09.557984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.573435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67a9b0-fc2a-449a-9024-16de739c4fc5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.579442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.580513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.581537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.582456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] PATH_INFO: `/v3/projects/130f997fde494ab887281d866d6aec31/users/cd454c25ddef46d993ff15440a64a438/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.654526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=130f997fde494ab887281d866d6aec31, user_id=cd454c25ddef46d993ff15440a64a438, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.659158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.698544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1995/4002] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:09 2022] PUT /identity/v3/projects/130f997fde494ab887281d866d6aec31/users/cd454c25ddef46d993ff15440a64a438/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:09.703478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863d077-b94b-4ee4-8040-c41857c098fe None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.703764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863d077-b94b-4ee4-8040-c41857c098fe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.704037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863d077-b94b-4ee4-8040-c41857c098fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.743136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6863d077-b94b-4ee4-8040-c41857c098fe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:09.744608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6863d077-b94b-4ee4-8040-c41857c098fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:09.903416 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2008/4003] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:09.922733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.924989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.166373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22758e9c-cb78-44ca-9577-1a872a5df38e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.178286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.178920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.179196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.179466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.201701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.203190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.237704 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1996/4004] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:10.241747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.241747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.241747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.280460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.281821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.417387 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2009/4005] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.423918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.426127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.441047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:10.626537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0bc7-6dcd-46d7-a139-187d6e1dac29 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.639379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.640820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.641350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.641916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.652799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.655068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.682712 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1997/4006] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:20.394642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:20.671756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=29 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:21.528515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:21.532217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:21.570918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2dd6d-0003-408f-87bb-bb4ec75b5842 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:21.586202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:21.586878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:21.587167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:21.587456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] PATH_INFO: `/v3/users/43e3adea586f4772b39b62323b1cc911` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:21.589656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e3adea586f4772b39b62323b1cc911)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:21.594609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:21.804090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e3adea586f4772b39b62323b1cc911'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.846699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e3adea586f4772b39b62323b1cc911'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.868133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e3adea586f4772b39b62323b1cc911'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.869087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e3adea586f4772b39b62323b1cc911'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.869976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e3adea586f4772b39b62323b1cc911'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.901711 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2010/4007] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:21 2022] DELETE /identity/v3/users/43e3adea586f4772b39b62323b1cc911 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:22.862331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.869201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.874826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:22.883429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df44dc-d77d-4311-8d16-7390220c37fe service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.046574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.049028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.050224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.050664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] PATH_INFO: `/v3/projects/d9024c27866542428191da960659ac2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.055673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9024c27866542428191da960659ac2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.061554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.084058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced984dd-4fee-447f-81c4-a48770b52956 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.098137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.099441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.103576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.104100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] PATH_INFO: `/v3/users/00e4ba59294f4529ba188e07cfc6681a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.109393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e4ba59294f4529ba188e07cfc6681a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.112306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.244486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9024c27866542428191da960659ac2e', 'request_id': 'req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.257291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9024c27866542428191da960659ac2e', 'request_id': 'req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.273389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e4ba59294f4529ba188e07cfc6681a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.274551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9024c27866542428191da960659ac2e', 'request_id': 'req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.295540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] The token cache is being invalidate because project d9024c27866542428191da960659ac2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:23.295814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e4ba59294f4529ba188e07cfc6681a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.295927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.296839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb97ad3d-c417-4b2b-9b6b-5296c1101bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.302993 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1998/4008] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:22 2022] DELETE /identity/v3/projects/d9024c27866542428191da960659ac2e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:23.307922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539c738-822e-4e83-800d-898f988f68fc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.308385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539c738-822e-4e83-800d-898f988f68fc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.308740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539c738-822e-4e83-800d-898f988f68fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.313180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e4ba59294f4529ba188e07cfc6681a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.314044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e4ba59294f4529ba188e07cfc6681a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.314767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e80a05a-a7c0-41db-8872-82a429ad1b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e4ba59294f4529ba188e07cfc6681a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.320052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:23.359447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2011/4009] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:22 2022] DELETE /identity/v3/users/00e4ba59294f4529ba188e07cfc6681a => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:23.372501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b539c738-822e-4e83-800d-898f988f68fc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:23.373877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b539c738-822e-4e83-800d-898f988f68fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:23.549599 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 1999/4010] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:23.557843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc63103-e3a5-4fb4-942f-167793326425 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.558194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc63103-e3a5-4fb4-942f-167793326425 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.558471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc63103-e3a5-4fb4-942f-167793326425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.597618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ffc63103-e3a5-4fb4-942f-167793326425 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:23.603254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ffc63103-e3a5-4fb4-942f-167793326425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:23.630653 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2012/4011] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:23.635175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539c738-822e-4e83-800d-898f988f68fc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:23.671378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539c738-822e-4e83-800d-898f988f68fc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.729754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.730827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.731269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.731693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.732524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:23.732992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.734828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.758925 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2000/4012] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:23.766970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc63103-e3a5-4fb4-942f-167793326425 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:23.781283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:23.783303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:23.790118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc63103-e3a5-4fb4-942f-167793326425 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.829265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.829937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.830296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.830408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a626-1cee-4a39-8644-a18085988b7e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.830631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.831288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.834640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.846538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.847207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.847492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.847766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] PATH_INFO: `/v3/projects/b33b687e3d75472bb38e672e90871a0b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.849102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33b687e3d75472bb38e672e90871a0b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.851146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.881536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:23.916932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2013/4013] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:23 2022] POST /identity/v3/projects => generated 397 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:23.917920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.026665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33b687e3d75472bb38e672e90871a0b', 'request_id': 'req-297f1489-42b2-4b6c-bd2f-5f892c139568'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.044490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33b687e3d75472bb38e672e90871a0b', 'request_id': 'req-297f1489-42b2-4b6c-bd2f-5f892c139568'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.066412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33b687e3d75472bb38e672e90871a0b', 'request_id': 'req-297f1489-42b2-4b6c-bd2f-5f892c139568'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.083669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] The token cache is being invalidate because project b33b687e3d75472bb38e672e90871a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:24.084090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.085016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-297f1489-42b2-4b6c-bd2f-5f892c139568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.091882 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2001/4014] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:23 2022] DELETE /identity/v3/projects/b33b687e3d75472bb38e672e90871a0b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:24.099877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de9be-3e16-442a-a2b9-43070fc6a037 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.100172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de9be-3e16-442a-a2b9-43070fc6a037 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.100461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065de9be-3e16-442a-a2b9-43070fc6a037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.107547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c422d8-c505-4b17-9383-a52e93a4da0c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.130004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-065de9be-3e16-442a-a2b9-43070fc6a037 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:24.131667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-065de9be-3e16-442a-a2b9-43070fc6a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.193956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.194713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.195126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.195465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.196431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.199573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.250983 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2014/4015] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:23 2022] POST /identity/v3/users => generated 299 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:24.254739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.256625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.270354 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2002/4016] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:24.277416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7274-9ebd-4fce-bbe8-6285fd7134a1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.277878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7274-9ebd-4fce-bbe8-6285fd7134a1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.278256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7274-9ebd-4fce-bbe8-6285fd7134a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.282805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ada0f-47b1-404e-a693-b08074d75b77 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.288257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.289434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.289939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.290404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.291366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.302435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.313716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-baad7274-9ebd-4fce-bbe8-6285fd7134a1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:24.315182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-baad7274-9ebd-4fce-bbe8-6285fd7134a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.328516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2003/4017] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:24.332775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7274-9ebd-4fce-bbe8-6285fd7134a1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.335858 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2015/4018] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:04:24.339087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.347845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7274-9ebd-4fce-bbe8-6285fd7134a1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.362172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.362801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.363074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.363191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dae003-745b-4b2a-bbac-659bc0f5008a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.363343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.364060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:24.364439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.366002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.373734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff253c-5b26-46f3-8460-d0bd3bb4f235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.373953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff253c-5b26-46f3-8460-d0bd3bb4f235 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.375752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff253c-5b26-46f3-8460-d0bd3bb4f235 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.376049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff253c-5b26-46f3-8460-d0bd3bb4f235 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.377099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff253c-5b26-46f3-8460-d0bd3bb4f235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.378926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff253c-5b26-46f3-8460-d0bd3bb4f235 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.397656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2016/4019] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:24.398979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adff9d-c2fd-4f94-99e8-af2da1035e7e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.399267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adff9d-c2fd-4f94-99e8-af2da1035e7e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.399536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adff9d-c2fd-4f94-99e8-af2da1035e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.402491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2004/4020] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:24.404428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.422965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87031ec9-ab6e-4f00-bd51-6dad4fb51a9e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.426921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-67adff9d-c2fd-4f94-99e8-af2da1035e7e None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:24.429015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67adff9d-c2fd-4f94-99e8-af2da1035e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.430040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.430982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.431467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.431949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] PATH_INFO: `/v3/projects/6cc3ec11e1374785afe8d3f73bc26891/users/9e38b0461884497e8c5d399051b3d50e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.457474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc3ec11e1374785afe8d3f73bc26891, user_id=9e38b0461884497e8c5d399051b3d50e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.460659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.471669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:24.482939 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2005/4021] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:24 2022] PUT /identity/v3/projects/6cc3ec11e1374785afe8d3f73bc26891/users/9e38b0461884497e8c5d399051b3d50e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:24.483779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.539668 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2017/4022] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:24.542763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.543116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.543450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.564614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:24.571038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702af45-af8d-4f89-afb0-94299d0cc7bb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.577487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.578273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.578928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.579346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.580416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.582279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:24.583964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.584695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.620673 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2018/4023] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:24.630094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.637029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.652784 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2006/4024] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:24.653310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.673498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.748213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16f0c9-f19d-4007-8614-90c7df042708 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.752454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.753077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.753367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.753660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.828917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.831654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.844829 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2007/4025] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:24.846524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.857938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d8489-5ac7-4a65-8223-880af103968f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.863376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.864077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.864357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.864629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.864971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ae171-72e3-4ff3-ad6a-5abc53611d3e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.865389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.868191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.911550 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2008/4026] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/users => generated 296 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:24.916000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.943416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d755c34-53b6-4077-9046-c266ad764965 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.956608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.958267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.958847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.959335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.961250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.965419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.968368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593199e-3b7a-41db-ae80-fdf0aca1f53c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.970172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593199e-3b7a-41db-ae80-fdf0aca1f53c service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.970172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593199e-3b7a-41db-ae80-fdf0aca1f53c service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.970172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593199e-3b7a-41db-ae80-fdf0aca1f53c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.003571 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2009/4027] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:25.006620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.040288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634e057-a161-4434-b3cb-388041390c27 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.054438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.055094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.055373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.055702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.056349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.058057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.070903 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2010/4028] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:25.076909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.092018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df9625-3005-4b2e-b54c-f35dbd9aac62 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.098250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.099363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.099719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.100127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] PATH_INFO: `/v3/projects/765dd4c95190458594be8f9a7915210e/users/e8f7671df52e4e95b829f03defeb6cf5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.101617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593199e-3b7a-41db-ae80-fdf0aca1f53c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.103858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593199e-3b7a-41db-ae80-fdf0aca1f53c service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.116971 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2019/4029] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:25.120270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=765dd4c95190458594be8f9a7915210e, user_id=e8f7671df52e4e95b829f03defeb6cf5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.123145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.151408 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2011/4030] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:25 2022] PUT /identity/v3/projects/765dd4c95190458594be8f9a7915210e/users/e8f7671df52e4e95b829f03defeb6cf5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:25.157053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639156f-7981-42e3-b4eb-77093211463f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.157173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639156f-7981-42e3-b4eb-77093211463f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.157348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639156f-7981-42e3-b4eb-77093211463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.234158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9639156f-7981-42e3-b4eb-77093211463f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:25.235577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9639156f-7981-42e3-b4eb-77093211463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:25.286832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:25.423011 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2020/4031] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:25.482038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.484205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:25.579096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:25.682388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ba384-a7c0-4a0a-9912-74a28b5e0459 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.687161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.687691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.688725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.689083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.707339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.719410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.815080 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2012/4032] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:26.817786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46623397-e0b5-47fc-9973-625da7102970 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:26.818343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46623397-e0b5-47fc-9973-625da7102970 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:26.818343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46623397-e0b5-47fc-9973-625da7102970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:26.842841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-46623397-e0b5-47fc-9973-625da7102970 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:26.844244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-46623397-e0b5-47fc-9973-625da7102970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:27.014793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2021/4033] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:27.040305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:27.264829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f07ea8-1588-40da-9ba2-e62ed665552a service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:27.299737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:27.304386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.304386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.305826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.334819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:27.337176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:27.363361 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2013/4034] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:30.835678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5622-d9c6-4aed-be1b-362b3255ed8f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.836551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5622-d9c6-4aed-be1b-362b3255ed8f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.837159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5622-d9c6-4aed-be1b-362b3255ed8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.876660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c7bd5622-d9c6-4aed-be1b-362b3255ed8f None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:30.879698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c7bd5622-d9c6-4aed-be1b-362b3255ed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.895376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2022/4035] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:30.910032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.930132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134e57a-0434-4f10-9697-67cd51f9ae15 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.949817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.950820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.951283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.951743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.981280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.988089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:31.024325 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2014/4036] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:32.026426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728caf3-d45a-41ed-85a0-cb3705f8fddc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.026797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728caf3-d45a-41ed-85a0-cb3705f8fddc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.026797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728caf3-d45a-41ed-85a0-cb3705f8fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.108305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1728caf3-d45a-41ed-85a0-cb3705f8fddc None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:32.111232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1728caf3-d45a-41ed-85a0-cb3705f8fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.168614 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2023/4037] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:32.190835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:32.215339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969aeed-1615-4c62-a842-448ec62139b7 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:32.232225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:32.233588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.234194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.234595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.245320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:32.247369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:32.287553 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2015/4038] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:04:33.391061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:33.391683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:33.391683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:33.453974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] MFA Rules not processed for user `456546fdd6cc4081ac3eff91382b717f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:33.455226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:33.601439 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2024/4039] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:34.150267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:34.154271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:34.350802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89feb6da-7e88-4452-9931-108597a08641 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:34.365564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:34.366517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:34.366971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:34.367401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:34.384862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:34.386605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:34.404003 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2016/4040] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:39.942232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.946527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:40.074343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:40.104312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fbfbc-9570-4d4b-933a-5e3b5a19c8d5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:40.112428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f83ce7-7bd0-4003-af59-04eb80073c55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:40.113417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f83ce7-7bd0-4003-af59-04eb80073c55 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:40.114223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f83ce7-7bd0-4003-af59-04eb80073c55 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:40.114713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f83ce7-7bd0-4003-af59-04eb80073c55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:40.116810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a8f83ce7-7bd0-4003-af59-04eb80073c55 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:40.150587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f83ce7-7bd0-4003-af59-04eb80073c55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:40.152895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f83ce7-7bd0-4003-af59-04eb80073c55 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:40.173524 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2025/4041] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:40.405312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:40.422642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc75a80-2f61-4d76-9dfe-1064b462371c service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:40.446115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:40.447139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:40.447585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:40.450583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:40.454764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:40.485556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:40.496859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:40.525924 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2017/4042] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:04:41.386145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5241bb-b0fd-421b-9860-3ba8980f75ee None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:41.387680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5241bb-b0fd-421b-9860-3ba8980f75ee None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:41.388465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5241bb-b0fd-421b-9860-3ba8980f75ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:41.462055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6a5241bb-b0fd-421b-9860-3ba8980f75ee None None] MFA Rules not processed for user `cd454c25ddef46d993ff15440a64a438`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:41.463757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6a5241bb-b0fd-421b-9860-3ba8980f75ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:41.623370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2026/4043] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:42.103090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:42.127232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a806174-3415-4af8-a545-ff7f9f341659 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:42.146823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28353f-3dfb-427e-8523-3c6136406ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:42.149884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28353f-3dfb-427e-8523-3c6136406ed6 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.150633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28353f-3dfb-427e-8523-3c6136406ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.151695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28353f-3dfb-427e-8523-3c6136406ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.164280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:42.174649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28353f-3dfb-427e-8523-3c6136406ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:42.180650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28353f-3dfb-427e-8523-3c6136406ed6 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:42.210703 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2018/4044] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:42.718349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.719197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.719653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.760161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] MFA Rules not processed for user `9e38b0461884497e8c5d399051b3d50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:42.767507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:42.937253 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2027/4045] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:42.948646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f754f-b30b-4116-a180-8bfa3de2b8cc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.950464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f754f-b30b-4116-a180-8bfa3de2b8cc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.951107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f754f-b30b-4116-a180-8bfa3de2b8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.979107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a46f754f-b30b-4116-a180-8bfa3de2b8cc None None] MFA Rules not processed for user `9e38b0461884497e8c5d399051b3d50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:42.981078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a46f754f-b30b-4116-a180-8bfa3de2b8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:43.000700 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2019/4046] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:43.604036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:43.605814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:43.639841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9c70a-48ca-4080-a0ba-0d22c11c8c9a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:43.679825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:43.680419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:43.680753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:43.680967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] PATH_INFO: `/v3/users/1032dca1542443a9a0ae30f1bbfb2d6e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:43.682208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1032dca1542443a9a0ae30f1bbfb2d6e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:43.684520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:43.826682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1032dca1542443a9a0ae30f1bbfb2d6e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:43.849897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:43.866258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1032dca1542443a9a0ae30f1bbfb2d6e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:43.920436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1032dca1542443a9a0ae30f1bbfb2d6e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:43.921640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1032dca1542443a9a0ae30f1bbfb2d6e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:43.922758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1032dca1542443a9a0ae30f1bbfb2d6e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:43.939993 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2028/4047] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:43 2022] DELETE /identity/v3/users/1032dca1542443a9a0ae30f1bbfb2d6e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:43.944660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503efbb-dc87-4cd7-b9e7-b5403e7a0fb6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:43.945157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503efbb-dc87-4cd7-b9e7-b5403e7a0fb6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:43.945245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503efbb-dc87-4cd7-b9e7-b5403e7a0fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:43.979848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b503efbb-dc87-4cd7-b9e7-b5403e7a0fb6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:43.983332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b503efbb-dc87-4cd7-b9e7-b5403e7a0fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:44.109181 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2020/4048] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:44.118832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:44.120284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:44.299747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027092d5-3f60-41a3-b799-d5eeafa29b26 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:44.354455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:44.362800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:44.364377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:44.365113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:44.402179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:44.440158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:44.503341 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2029/4049] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:44.567789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:44.568471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:44.568711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:44.620959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:44.622264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:44.724972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:44.726697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:44.794737 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2021/4050] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:44.809832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:44.813362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:44.917666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f08cee-eb77-4d76-b119-4c9368d89aea service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:44.930595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:44.931225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:44.931716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:44.932158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:44.936399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb6fb2-d40f-4c86-bf6a-66c3e53ae101 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:44.938149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:44.944345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090c12f-5e8e-4d31-81f6-368a6bbc2df9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:44.945135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090c12f-5e8e-4d31-81f6-368a6bbc2df9 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:44.945730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090c12f-5e8e-4d31-81f6-368a6bbc2df9 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:44.945935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090c12f-5e8e-4d31-81f6-368a6bbc2df9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:44.957400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090c12f-5e8e-4d31-81f6-368a6bbc2df9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:44.960730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090c12f-5e8e-4d31-81f6-368a6bbc2df9 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:44.973161 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2022/4051] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:04:45.170817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:45.172449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:45.200507 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2030/4052] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:46.795571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf30a-10fb-493b-ba42-5156b97cb186 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:46.796150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf30a-10fb-493b-ba42-5156b97cb186 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:46.796531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf30a-10fb-493b-ba42-5156b97cb186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:46.809713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:46.811217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:46.836743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8192e-29a3-448e-a96e-fe9e07218ddf service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:46.851319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:46.853042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:46.853788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:46.855409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] PATH_INFO: `/v3/projects/44687313f13747878e0369af9a9ef6f2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:46.858362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44687313f13747878e0369af9a9ef6f2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:46.858604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-536cf30a-10fb-493b-ba42-5156b97cb186 None None] MFA Rules not processed for user `cd454c25ddef46d993ff15440a64a438`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:46.861435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:46.862246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-536cf30a-10fb-493b-ba42-5156b97cb186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:46.893145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:46.939630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:46.998676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44687313f13747878e0369af9a9ef6f2', 'request_id': 'req-fd1d0916-354e-4a83-a991-16aa57bfac44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.023300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44687313f13747878e0369af9a9ef6f2', 'request_id': 'req-fd1d0916-354e-4a83-a991-16aa57bfac44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.033182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44687313f13747878e0369af9a9ef6f2', 'request_id': 'req-fd1d0916-354e-4a83-a991-16aa57bfac44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.079333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] The token cache is being invalidate because project 44687313f13747878e0369af9a9ef6f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:47.081191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.083085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fd1d0916-354e-4a83-a991-16aa57bfac44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.096028 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2031/4053] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:46 2022] DELETE /identity/v3/projects/44687313f13747878e0369af9a9ef6f2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:47.101762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769fa3-c670-4bec-824a-aeeb3b8cf254 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.102806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769fa3-c670-4bec-824a-aeeb3b8cf254 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.103496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769fa3-c670-4bec-824a-aeeb3b8cf254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.111697 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2023/4054] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:47.171615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e8769fa3-c670-4bec-824a-aeeb3b8cf254 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:47.175107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8769fa3-c670-4bec-824a-aeeb3b8cf254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:47.319749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2032/4055] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:47.321599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af66c24-bf28-4f0d-95bb-7f0486467c13 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.322046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af66c24-bf28-4f0d-95bb-7f0486467c13 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.322423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af66c24-bf28-4f0d-95bb-7f0486467c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.411108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1af66c24-bf28-4f0d-95bb-7f0486467c13 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:47.415507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1af66c24-bf28-4f0d-95bb-7f0486467c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:47.442255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769fa3-c670-4bec-824a-aeeb3b8cf254 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.443251 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2024/4056] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:47.454869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769fa3-c670-4bec-824a-aeeb3b8cf254 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:47.463724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:47.471113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.471740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.472243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.473348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:47.474474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:47.476420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:47.500356 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2033/4057] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:04:47.505097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af66c24-bf28-4f0d-95bb-7f0486467c13 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.518503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af66c24-bf28-4f0d-95bb-7f0486467c13 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:47.527700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:47.529349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.529971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.530494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.531256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:47.533423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:47.584666 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2025/4058] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:47 2022] POST /identity/v3/projects => generated 359 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:47.590077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.607395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051443ef-036c-40a7-b72e-17f4fd271307 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:47.799684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:47.802142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.802500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.802852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.803964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:47.807622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:47.862728 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2034/4059] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:47 2022] POST /identity/v3/users => generated 280 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:47.864160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.878272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910b212-837d-4bba-8e47-7ece9a76caf7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:47.914377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:47.915035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.915315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.915586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.916253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:47.917932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:47.951802 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2026/4060] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:47.954898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.972154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba30f0e-2295-4e92-b8ea-814501440ef8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.003105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb7a12-4865-474c-9b3e-ac13c14c508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.003744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7a12-4865-474c-9b3e-ac13c14c508b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.004088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7a12-4865-474c-9b3e-ac13c14c508b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.004361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7a12-4865-474c-9b3e-ac13c14c508b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.005124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb7a12-4865-474c-9b3e-ac13c14c508b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.007203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb7a12-4865-474c-9b3e-ac13c14c508b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.056275 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2035/4061] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:48.060577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.075057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c6cf-af82-456d-8e94-cab567e1cbc3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.086396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0c42f-d9ff-4774-8b88-687edfba4219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.090247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c42f-d9ff-4774-8b88-687edfba4219 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.091799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c42f-d9ff-4774-8b88-687edfba4219 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.092293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c42f-d9ff-4774-8b88-687edfba4219 admin admin] PATH_INFO: `/v3/projects/8849b5d5a21c4e35966bb1b345cd2af0/users/91ed8ec972d9495e820a0a23367acaf0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.150516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea9218-62a8-44a0-9cea-405e981bd34a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.150913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea9218-62a8-44a0-9cea-405e981bd34a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.151185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea9218-62a8-44a0-9cea-405e981bd34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.178477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0c42f-d9ff-4774-8b88-687edfba4219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8849b5d5a21c4e35966bb1b345cd2af0, user_id=91ed8ec972d9495e820a0a23367acaf0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.185867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0c42f-d9ff-4774-8b88-687edfba4219 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.186041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:48.224552 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2027/4062] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:48 2022] PUT /identity/v3/projects/8849b5d5a21c4e35966bb1b345cd2af0/users/91ed8ec972d9495e820a0a23367acaf0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:48.227121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.227857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.228241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.235809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ccea9218-62a8-44a0-9cea-405e981bd34a None None] MFA Rules not processed for user `e8f7671df52e4e95b829f03defeb6cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:48.235809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ccea9218-62a8-44a0-9cea-405e981bd34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:48.307223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] MFA Rules not processed for user `91ed8ec972d9495e820a0a23367acaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:48.308459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:48.439125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2036/4063] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:48.440617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.440902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.441172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.497861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] MFA Rules not processed for user `e8f7671df52e4e95b829f03defeb6cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:48.504535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:48.518503 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2037/4064] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:48.549294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2028/4065] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:48.552942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.554970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:48.709404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078bbb0-3285-4d3e-baec-00ce7db1bb17 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.732013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f4511-3109-4c09-accc-ded2ac7245fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.732688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f4511-3109-4c09-accc-ded2ac7245fd service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.733361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f4511-3109-4c09-accc-ded2ac7245fd service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.734101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f4511-3109-4c09-accc-ded2ac7245fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.761355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f4511-3109-4c09-accc-ded2ac7245fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.768482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f4511-3109-4c09-accc-ded2ac7245fd service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.800280 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2038/4066] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:50.205984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:50.207608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:50.406639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:50.515848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80f7ae-2a4c-42af-9fa1-b0a423c11bda None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:50.538527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b83176-e62d-4526-8b1f-9ba385cc754a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:50.544392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b83176-e62d-4526-8b1f-9ba385cc754a service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:50.544680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b83176-e62d-4526-8b1f-9ba385cc754a service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:50.548639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b83176-e62d-4526-8b1f-9ba385cc754a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:50.581793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b83176-e62d-4526-8b1f-9ba385cc754a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:50.588114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b83176-e62d-4526-8b1f-9ba385cc754a service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:50.626057 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2029/4067] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:57.484760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.485375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.485375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.556629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] MFA Rules not processed for user `2ef547e071fd43ec96c745c26eca119f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:57.562281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.691840 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2039/4068] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:00.609571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:01.170126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c56f4-e201-483c-a967-f91fd3f67d17 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:01.172578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c56f4-e201-483c-a967-f91fd3f67d17 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:01.173058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c56f4-e201-483c-a967-f91fd3f67d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:01.218596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f87c56f4-e201-483c-a967-f91fd3f67d17 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:01.219824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f87c56f4-e201-483c-a967-f91fd3f67d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.422416 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2030/4069] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:01.443889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:01.452685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.505418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:01.628559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa965e-2a45-476e-a493-67480dd63996 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:01.634418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2ffb0-ed95-4acf-bff4-eb79223ebd17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:01.635071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ffb0-ed95-4acf-bff4-eb79223ebd17 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:01.635355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ffb0-ed95-4acf-bff4-eb79223ebd17 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:01.635626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ffb0-ed95-4acf-bff4-eb79223ebd17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:01.645073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2ffb0-ed95-4acf-bff4-eb79223ebd17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:01.646724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2ffb0-ed95-4acf-bff4-eb79223ebd17 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:01.655620 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2040/4070] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:05:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:05:07.651936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c56f4-e201-483c-a967-f91fd3f67d17 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.724485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c56f4-e201-483c-a967-f91fd3f67d17 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.731068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.731993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.732270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.732539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] PATH_INFO: `/v3/users/91ed8ec972d9495e820a0a23367acaf0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.733742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ed8ec972d9495e820a0a23367acaf0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.735842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.806829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ed8ec972d9495e820a0a23367acaf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:07.807514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ed8ec972d9495e820a0a23367acaf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:07.822523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ed8ec972d9495e820a0a23367acaf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:07.823160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ed8ec972d9495e820a0a23367acaf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:07.854713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ed8ec972d9495e820a0a23367acaf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:07.860594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:07.870534 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2031/4071] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:07 2022] DELETE /identity/v3/users/91ed8ec972d9495e820a0a23367acaf0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:07.874990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.875471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.875594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.903549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:07.905151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:08.000365 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2041/4072] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:08.011118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:08.012611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:08.109568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ab09a-8c0b-498a-aa9d-1dd513766b48 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:08.115152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe5cd7-0762-4909-8f58-fe2deb7059dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:08.115667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe5cd7-0762-4909-8f58-fe2deb7059dd service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.116007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe5cd7-0762-4909-8f58-fe2deb7059dd service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.116310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe5cd7-0762-4909-8f58-fe2deb7059dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:08.127237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe5cd7-0762-4909-8f58-fe2deb7059dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:08.129783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe5cd7-0762-4909-8f58-fe2deb7059dd service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:08.146267 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2032/4073] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:08.659395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:08.661419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:08.674059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d23b9e-1e41-4178-8bcd-0556681d3262 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:08.679618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:08.680290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.680567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.680832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] PATH_INFO: `/v3/projects/8849b5d5a21c4e35966bb1b345cd2af0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:08.682105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8849b5d5a21c4e35966bb1b345cd2af0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:08.684072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:08.736954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8849b5d5a21c4e35966bb1b345cd2af0', 'request_id': 'req-38290664-9ec7-41fc-ba74-cf9f888bad80'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.746798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8849b5d5a21c4e35966bb1b345cd2af0', 'request_id': 'req-38290664-9ec7-41fc-ba74-cf9f888bad80'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.753069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8849b5d5a21c4e35966bb1b345cd2af0', 'request_id': 'req-38290664-9ec7-41fc-ba74-cf9f888bad80'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.808873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] The token cache is being invalidate because project 8849b5d5a21c4e35966bb1b345cd2af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:08.809198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.811563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-38290664-9ec7-41fc-ba74-cf9f888bad80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.818255 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2042/4074] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:08 2022] DELETE /identity/v3/projects/8849b5d5a21c4e35966bb1b345cd2af0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:08.823224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c1c11-fdc0-4ce5-a20a-fa51edb7fec3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.824893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c1c11-fdc0-4ce5-a20a-fa51edb7fec3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.825338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c1c11-fdc0-4ce5-a20a-fa51edb7fec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:08.851510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1f2c1c11-fdc0-4ce5-a20a-fa51edb7fec3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:08.853184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f2c1c11-fdc0-4ce5-a20a-fa51edb7fec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:08.965692 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2033/4075] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:08.972859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95dba-f377-4f49-a76b-10265425d6d0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.975048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95dba-f377-4f49-a76b-10265425d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.975717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95dba-f377-4f49-a76b-10265425d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.010478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-12c95dba-f377-4f49-a76b-10265425d6d0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:09.011791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-12c95dba-f377-4f49-a76b-10265425d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:09.025990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2043/4076] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:09.026924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c1c11-fdc0-4ce5-a20a-fa51edb7fec3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.038823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c1c11-fdc0-4ce5-a20a-fa51edb7fec3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.044275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.044954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.045235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.045531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.046189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:09.046491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.048229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.062237 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2034/4077] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:05:09.067480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95dba-f377-4f49-a76b-10265425d6d0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.077518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95dba-f377-4f49-a76b-10265425d6d0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.082662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.083126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.083404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.083674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.084310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.086285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.099002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:09.128276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.133759 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2044/4078] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:09 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:09.139170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6114d-c0c8-4e56-a386-be2b030b5f94 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.202304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.203343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.203966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.204649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.205148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.207622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.240894 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2035/4079] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:09 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:09.242163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.254543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41cbf6-8615-489a-b0ed-874f90493c27 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.267393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.268031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.268312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.268605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.269344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.271029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.288534 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2045/4080] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:09.289747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.301652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe568c3-ae1a-4101-86dd-4735c1a596d4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.306967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b8dd7-ba01-4920-b934-d23ce70c8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.307869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b8dd7-ba01-4920-b934-d23ce70c8081 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.308210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b8dd7-ba01-4920-b934-d23ce70c8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.308572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b8dd7-ba01-4920-b934-d23ce70c8081 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.309311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b8dd7-ba01-4920-b934-d23ce70c8081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.311364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b8dd7-ba01-4920-b934-d23ce70c8081 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.331254 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2036/4081] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:09.335678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.348300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad8a11-2101-4d90-95f2-686c8d716189 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.353347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.354119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.354456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.354807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] PATH_INFO: `/v3/projects/bc1f0a9a3f7f424ebdb6116ae4c8e826/users/ff5da5e62c1c460b9d405470910594a7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.372486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1f0a9a3f7f424ebdb6116ae4c8e826, user_id=ff5da5e62c1c460b9d405470910594a7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.375321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.393319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2046/4082] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:09 2022] PUT /identity/v3/projects/bc1f0a9a3f7f424ebdb6116ae4c8e826/users/ff5da5e62c1c460b9d405470910594a7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:09.405024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.405303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.405603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.414533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:09.436059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] MFA Rules not processed for user `ff5da5e62c1c460b9d405470910594a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:09.437356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:09.523999 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2037/4083] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:09.525411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.570152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.571269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:09.642827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4f1cd-bcce-45a8-b2f5-2a51e55bf7c2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.662974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.663612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.663894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.664161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.664811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.667438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.717972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3173e9-7410-48b3-9bee-35e61414dc9c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.724666 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2047/4084] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:09 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:09.728014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.729427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.729898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.730186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] PATH_INFO: `/v3/users/e570c2dfca474a089d281ac21a859e98` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.730434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.732264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e570c2dfca474a089d281ac21a859e98)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.742836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.824160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae4340-3986-4137-be26-695862662d39 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.826612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e570c2dfca474a089d281ac21a859e98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.828572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e570c2dfca474a089d281ac21a859e98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.833180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.834315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.834711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.836161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.836161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.838655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.849497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e570c2dfca474a089d281ac21a859e98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.850679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e570c2dfca474a089d281ac21a859e98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.868206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2048/4085] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:09 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:09.871633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.874144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:09.876660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4a53466b-54b3-436c-80e4-e6d7394e5843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e570c2dfca474a089d281ac21a859e98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.901856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92724b86-838c-4e20-aa38-b5c52011bc01 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.910057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.910776 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2038/4086] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:09 2022] DELETE /identity/v3/users/e570c2dfca474a089d281ac21a859e98 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:09.910919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.911096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.911503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.912231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.915616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.916062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.916362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.916906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.942060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:09.947296 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2049/4087] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:09.947473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:09.948508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.960530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:09.966786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab337a2d-eb3c-4713-bd29-3ab779f9bf66 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.987046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:10.051621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.052197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.052481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.052755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] PATH_INFO: `/v3/projects/7a353bef98d44cca917255e423bc57d7/users/3ada5d880d5f4e1c98c99f930547d11a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.071110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a353bef98d44cca917255e423bc57d7, user_id=3ada5d880d5f4e1c98c99f930547d11a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.074423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.110612 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2050/4088] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:09 2022] PUT /identity/v3/projects/7a353bef98d44cca917255e423bc57d7/users/3ada5d880d5f4e1c98c99f930547d11a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:10.116781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.128390 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2039/4089] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:10.167311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.168715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.218688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590277e-1c51-458d-a916-fc2e4df4cdaf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.224710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.227998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.228767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.229308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.230202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.232695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.246623 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2051/4090] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:10.250710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df1ac9-093e-4ded-a171-f2052070d81e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.252182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.267366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e801-f45d-4cda-b0d2-7f1aeec55dea admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.272335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eebb95-f061-4661-861b-9ac5a8ff62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.273003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eebb95-f061-4661-861b-9ac5a8ff62af admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.273279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eebb95-f061-4661-861b-9ac5a8ff62af admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.273576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eebb95-f061-4661-861b-9ac5a8ff62af admin admin] PATH_INFO: `/v3/projects/7a353bef98d44cca917255e423bc57d7/users/3ada5d880d5f4e1c98c99f930547d11a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.275687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eebb95-f061-4661-861b-9ac5a8ff62af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a353bef98d44cca917255e423bc57d7, user_id=3ada5d880d5f4e1c98c99f930547d11a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.278773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eebb95-f061-4661-861b-9ac5a8ff62af admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.292376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2052/4091] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:10 2022] PUT /identity/v3/projects/7a353bef98d44cca917255e423bc57d7/users/3ada5d880d5f4e1c98c99f930547d11a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:10.305026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214f194-f10f-4bee-aca5-8333fa143a0e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.305345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214f194-f10f-4bee-aca5-8333fa143a0e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.305829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214f194-f10f-4bee-aca5-8333fa143a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.359879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9214f194-f10f-4bee-aca5-8333fa143a0e None None] MFA Rules not processed for user `3ada5d880d5f4e1c98c99f930547d11a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:10.362698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9214f194-f10f-4bee-aca5-8333fa143a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.379971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.380752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.381517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.381892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.393409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.395390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.409887 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2040/4092] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:10.521579 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2053/4093] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:10.532686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.535607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.647587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c1c57-1048-4974-92e9-9a3c2702391b service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.657021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:10.736705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e22d3-57bd-4b5f-b36c-47ca5c642795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.737422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e22d3-57bd-4b5f-b36c-47ca5c642795 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.738254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e22d3-57bd-4b5f-b36c-47ca5c642795 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.738577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e22d3-57bd-4b5f-b36c-47ca5c642795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.757252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e22d3-57bd-4b5f-b36c-47ca5c642795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.760040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e22d3-57bd-4b5f-b36c-47ca5c642795 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.777868 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2041/4094] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:10.865540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214f194-f10f-4bee-aca5-8333fa143a0e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.878690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:11.053088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214f194-f10f-4bee-aca5-8333fa143a0e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.061963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.062691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.064041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.064320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] PATH_INFO: `/v3/users/ff5da5e62c1c460b9d405470910594a7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.065579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5da5e62c1c460b9d405470910594a7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.069263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.191745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5da5e62c1c460b9d405470910594a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.230063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5da5e62c1c460b9d405470910594a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.263079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5da5e62c1c460b9d405470910594a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.273911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5da5e62c1c460b9d405470910594a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.274815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5da5e62c1c460b9d405470910594a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.324953 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2054/4095] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:10 2022] DELETE /identity/v3/users/ff5da5e62c1c460b9d405470910594a7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:11.330927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.333381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.333733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.392528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:11.393953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:11.512737 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2042/4096] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:11.567484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.569374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:11.658457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf595-6308-4177-9bc0-4f914534337d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.662782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aeaeac-813b-4779-8af5-1890a829306a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.663570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeaeac-813b-4779-8af5-1890a829306a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.664701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeaeac-813b-4779-8af5-1890a829306a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.664874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeaeac-813b-4779-8af5-1890a829306a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.679586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aeaeac-813b-4779-8af5-1890a829306a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.681338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aeaeac-813b-4779-8af5-1890a829306a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.701822 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2055/4097] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:12.083416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:12.084739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.104738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3c749-4ce3-482e-abab-391b140d77e9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:12.130684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:12.131448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.131726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.131996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] PATH_INFO: `/v3/projects/1894770a3dfd4dd2aaa4a2b02f9a7f36` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.134856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1894770a3dfd4dd2aaa4a2b02f9a7f36)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:12.137028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:12.149565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22084-36cb-4fba-8d78-5b2826865ac4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.150516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22084-36cb-4fba-8d78-5b2826865ac4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.150975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec22084-36cb-4fba-8d78-5b2826865ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.191203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aec22084-36cb-4fba-8d78-5b2826865ac4 None None] MFA Rules not processed for user `9e38b0461884497e8c5d399051b3d50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:12.192826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aec22084-36cb-4fba-8d78-5b2826865ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.208820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:12.256245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1894770a3dfd4dd2aaa4a2b02f9a7f36', 'request_id': 'req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.270313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1894770a3dfd4dd2aaa4a2b02f9a7f36', 'request_id': 'req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.302460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1894770a3dfd4dd2aaa4a2b02f9a7f36', 'request_id': 'req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.340637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] The token cache is being invalidate because project 1894770a3dfd4dd2aaa4a2b02f9a7f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:12.341343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.341913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e18f4bfe-09c7-4c5e-a7d6-01569f3bc11a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.342571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2056/4098] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:12.346823 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2043/4099] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:12 2022] DELETE /identity/v3/projects/1894770a3dfd4dd2aaa4a2b02f9a7f36 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:12.351233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec22084-36cb-4fba-8d78-5b2826865ac4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:12.365380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec22084-36cb-4fba-8d78-5b2826865ac4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:12.447388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5dd55-6e3c-4793-a2b7-dea7eba01d92 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.447673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5dd55-6e3c-4793-a2b7-dea7eba01d92 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.447943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5dd55-6e3c-4793-a2b7-dea7eba01d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.483545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-59b5dd55-6e3c-4793-a2b7-dea7eba01d92 None None] MFA Rules not processed for user `9e38b0461884497e8c5d399051b3d50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:12.484470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-59b5dd55-6e3c-4793-a2b7-dea7eba01d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.508753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:12.509869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.510395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.510894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] PATH_INFO: `/v3/users/9128af1af1684c82ae2db52f2ccaa0b9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.513372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9128af1af1684c82ae2db52f2ccaa0b9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:12.517044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:12.539698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:12.626480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9128af1af1684c82ae2db52f2ccaa0b9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.643500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9128af1af1684c82ae2db52f2ccaa0b9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.662336 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2044/4100] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:12.666389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9128af1af1684c82ae2db52f2ccaa0b9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.669304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9128af1af1684c82ae2db52f2ccaa0b9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.670440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44beec60-4cfe-4bf2-a878-ec0b50e28e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9128af1af1684c82ae2db52f2ccaa0b9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.699497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.699755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.700034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.706064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2057/4101] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:12 2022] DELETE /identity/v3/users/9128af1af1684c82ae2db52f2ccaa0b9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:12.740644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] MFA Rules not processed for user `9e38b0461884497e8c5d399051b3d50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:12.743625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.870957 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2045/4102] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:12.976795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1972d0-8f1f-4a78-8e83-375fbc6900ec None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.977096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1972d0-8f1f-4a78-8e83-375fbc6900ec None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.977376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1972d0-8f1f-4a78-8e83-375fbc6900ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.017481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8e1972d0-8f1f-4a78-8e83-375fbc6900ec None None] MFA Rules not processed for user `9e38b0461884497e8c5d399051b3d50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:13.019021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8e1972d0-8f1f-4a78-8e83-375fbc6900ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.044908 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2058/4103] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:13.432454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.433929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.562973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7e94b-4817-4577-a2b9-9648958ce2e0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.572114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.572928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.573257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.573581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] PATH_INFO: `/v3/projects/c2d9a9f0412940ee9c92d54c7ac16d4e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.574811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d9a9f0412940ee9c92d54c7ac16d4e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.576882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.649360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d7bad-71ec-415f-b281-ed28cf24cdfa None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.649862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d7bad-71ec-415f-b281-ed28cf24cdfa None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.649994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d7bad-71ec-415f-b281-ed28cf24cdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.669092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-205d7bad-71ec-415f-b281-ed28cf24cdfa None None] MFA Rules not processed for user `9e38b0461884497e8c5d399051b3d50e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:13.670592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-205d7bad-71ec-415f-b281-ed28cf24cdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.702866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d9a9f0412940ee9c92d54c7ac16d4e', 'request_id': 'req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.703301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:13.710233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d9a9f0412940ee9c92d54c7ac16d4e', 'request_id': 'req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.732786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d9a9f0412940ee9c92d54c7ac16d4e', 'request_id': 'req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.749779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] The token cache is being invalidate because project c2d9a9f0412940ee9c92d54c7ac16d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:13.750076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.756368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2478a272-48fe-41cd-97dd-d1d31f9ccdbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.757258 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2059/4104] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:13.758215 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2046/4105] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:13 2022] DELETE /identity/v3/projects/c2d9a9f0412940ee9c92d54c7ac16d4e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:13.764915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.765485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.766512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.819585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:13.823529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.967050 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2060/4106] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:13.968799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17726-fb72-432d-811f-91d0cdf4a2d3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.969089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17726-fb72-432d-811f-91d0cdf4a2d3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.969362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17726-fb72-432d-811f-91d0cdf4a2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.975859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:14.090522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5ea17726-fb72-432d-811f-91d0cdf4a2d3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:14.095602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.097080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:14.112264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5ea17726-fb72-432d-811f-91d0cdf4a2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:14.124562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d720ac8-2d31-445b-8a8e-fa48911f196b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.131863 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2047/4107] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:14.141012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.141542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea17726-fb72-432d-811f-91d0cdf4a2d3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.143837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.144228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.144503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] PATH_INFO: `/v3/projects/bc1f0a9a3f7f424ebdb6116ae4c8e826` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.149353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1f0a9a3f7f424ebdb6116ae4c8e826)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.156949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.169006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea17726-fb72-432d-811f-91d0cdf4a2d3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.185353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.187169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.187973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.188287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.188928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:14.191520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.194348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.216848 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2048/4108] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:14.218184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.232449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed621-25e3-45dc-8d68-90c09a51c67c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.238777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.239734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.240110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.240479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.241207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.243260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.274228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1f0a9a3f7f424ebdb6116ae4c8e826', 'request_id': 'req-368b6ebf-511c-4233-b213-2274af5ed74d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.286804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1f0a9a3f7f424ebdb6116ae4c8e826', 'request_id': 'req-368b6ebf-511c-4233-b213-2274af5ed74d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.305274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1f0a9a3f7f424ebdb6116ae4c8e826', 'request_id': 'req-368b6ebf-511c-4233-b213-2274af5ed74d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.306960 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2049/4109] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:14 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:14.308221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.330186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] The token cache is being invalidate because project bc1f0a9a3f7f424ebdb6116ae4c8e826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:14.330589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.331291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.333591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2061/4110] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:14 2022] DELETE /identity/v3/projects/bc1f0a9a3f7f424ebdb6116ae4c8e826 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:14.338752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.340665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:14.456365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c6f6-38ec-4f7c-b70d-38ddc977b423 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.461533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.462578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.463035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.463474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.464276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.466424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.477897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:14.483960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b6ebf-511c-4233-b213-2274af5ed74d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.492563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.493404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.493884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.494310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] PATH_INFO: `/v3/users/3ada5d880d5f4e1c98c99f930547d11a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.496215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ada5d880d5f4e1c98c99f930547d11a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.500078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.508101 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2050/4111] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:14 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:14.512293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.514670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:14.528644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790f7f9-8a66-45a2-afeb-e8f94effcd2a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.535264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.536401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.536892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.537486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.539529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.541792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.562506 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2051/4112] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:14.566823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.583719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52a0fb-2c55-465e-b4b3-906e77439690 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.591870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.592906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.594085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.594878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.596495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.598703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.613648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ada5d880d5f4e1c98c99f930547d11a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.617742 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2052/4113] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:14.626138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.650463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ada5d880d5f4e1c98c99f930547d11a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.661372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ebd55-292a-4444-ac8c-f5dc7ada11f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:14.667022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ada5d880d5f4e1c98c99f930547d11a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.677866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:14.678698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ada5d880d5f4e1c98c99f930547d11a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.679492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ddfe730a-943f-4cdd-b57c-df55c917b72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ada5d880d5f4e1c98c99f930547d11a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:14.679651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.680428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.683784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] PATH_INFO: `/v3/projects/c3a83c9c29f1483b9ee9c48e94dc17ce/users/24c7a82126124ebb8baf5c656ff9849d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.717573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2062/4114] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:14 2022] DELETE /identity/v3/users/3ada5d880d5f4e1c98c99f930547d11a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:14.733928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a83c9c29f1483b9ee9c48e94dc17ce, user_id=24c7a82126124ebb8baf5c656ff9849d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:14.741375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:14.781875 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2053/4115] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:14 2022] PUT /identity/v3/projects/c3a83c9c29f1483b9ee9c48e94dc17ce/users/24c7a82126124ebb8baf5c656ff9849d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:14.802379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:14.804520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:14.804520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:14.854188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:14.856195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:14.948814 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2063/4116] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:14.957885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:14.961359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:15.105104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:15.155908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572b8af-4ec5-4382-8d22-f47d923b887a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:15.160664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149cf53-2516-48c0-ad5b-c56181e6dd51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:15.161390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cf53-2516-48c0-ad5b-c56181e6dd51 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:15.161783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cf53-2516-48c0-ad5b-c56181e6dd51 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:15.162332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149cf53-2516-48c0-ad5b-c56181e6dd51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:15.200216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149cf53-2516-48c0-ad5b-c56181e6dd51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:15.202186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149cf53-2516-48c0-ad5b-c56181e6dd51 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:15.238141 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2054/4117] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:16.578182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:16.579767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:16.592903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb935d3-0047-4748-8717-eb3254b3c9cc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:16.598702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:16.599823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:16.600103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:16.600552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] PATH_INFO: `/v3/projects/7a353bef98d44cca917255e423bc57d7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:16.601878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a353bef98d44cca917255e423bc57d7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:16.604029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:16.685868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a353bef98d44cca917255e423bc57d7', 'request_id': 'req-792e664d-709d-4429-b571-3a20a0690399'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:16.715424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a353bef98d44cca917255e423bc57d7', 'request_id': 'req-792e664d-709d-4429-b571-3a20a0690399'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:16.732283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a353bef98d44cca917255e423bc57d7', 'request_id': 'req-792e664d-709d-4429-b571-3a20a0690399'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:16.744707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:16.750260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] The token cache is being invalidate because project 7a353bef98d44cca917255e423bc57d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:16.751017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:16.753156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:16.755638 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2064/4118] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:16 2022] DELETE /identity/v3/projects/7a353bef98d44cca917255e423bc57d7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:16.760910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba41407-be9c-4c97-99ce-a73922bd1d1f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:16.762691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba41407-be9c-4c97-99ce-a73922bd1d1f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:16.763222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba41407-be9c-4c97-99ce-a73922bd1d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:16.785422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:16.787870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:16.806455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bba41407-be9c-4c97-99ce-a73922bd1d1f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:16.808420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bba41407-be9c-4c97-99ce-a73922bd1d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:16.903018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e664d-709d-4429-b571-3a20a0690399 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:16.911013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:16.911753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:16.912038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:16.912319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] PATH_INFO: `/v3/users/456546fdd6cc4081ac3eff91382b717f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:16.915244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=456546fdd6cc4081ac3eff91382b717f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:16.917309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:16.957380 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2055/4119] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:16.966722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705624e-db18-4bde-b35c-f133f033120f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:16.967114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705624e-db18-4bde-b35c-f133f033120f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:16.967516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705624e-db18-4bde-b35c-f133f033120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.002454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9705624e-db18-4bde-b35c-f133f033120f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:17.003989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9705624e-db18-4bde-b35c-f133f033120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:17.009651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456546fdd6cc4081ac3eff91382b717f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:17.016759 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2056/4120] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:17.021635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705624e-db18-4bde-b35c-f133f033120f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:17.046199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456546fdd6cc4081ac3eff91382b717f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:17.046457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705624e-db18-4bde-b35c-f133f033120f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:17.054024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:17.055515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:17.055801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:17.056239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.057153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:17.057425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:17.059339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:17.079145 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2057/4121] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:17.080230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:17.085348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456546fdd6cc4081ac3eff91382b717f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:17.086028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456546fdd6cc4081ac3eff91382b717f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:17.088429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456546fdd6cc4081ac3eff91382b717f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:17.097632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496122b-1f63-46bc-aa0a-016095d7e071 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:17.106841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:17.107439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:17.107719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:17.107988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.108740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:17.115518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:17.147376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2065/4122] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:16 2022] DELETE /identity/v3/users/456546fdd6cc4081ac3eff91382b717f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:17.209775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2058/4123] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:17 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:17.215061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:17.218290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:17.433934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5d8e2-9a04-49bd-9ac2-fa7af968dd75 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:17.485749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:17.598547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:17.600078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:17.600623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:17.601736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.602970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:17.605589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:17.678526 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2066/4124] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:17 2022] POST /identity/v3/users => generated 294 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:17.683002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:17.699705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c0576-cc49-4730-a43d-53e1414a0b58 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:17.713484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:17.714121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:17.714400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:17.714666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.715376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:17.717017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:17.751476 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2059/4125] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:17.756159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:17.767998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db193d-07db-4572-9b55-2ae027da8755 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:17.777290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636d7f1-ba44-4469-b529-8c72110b9d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:17.777971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636d7f1-ba44-4469-b529-8c72110b9d92 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:17.778246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636d7f1-ba44-4469-b529-8c72110b9d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:17.778520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636d7f1-ba44-4469-b529-8c72110b9d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.779176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636d7f1-ba44-4469-b529-8c72110b9d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:17.781043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636d7f1-ba44-4469-b529-8c72110b9d92 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:17.804818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2067/4126] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:17.808680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:17.820606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ea42-f813-48d1-a9fe-f28de1daac34 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:17.829843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:17.831436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:17.831941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:17.832389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] PATH_INFO: `/v3/projects/ed058afc9e9c4a6fa8eea4f882ec108e/users/65df6e06a0fd4e349c9a75fde1556870/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.851588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed058afc9e9c4a6fa8eea4f882ec108e, user_id=65df6e06a0fd4e349c9a75fde1556870, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:17.855753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:17.872644 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2060/4127] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:17 2022] PUT /identity/v3/projects/ed058afc9e9c4a6fa8eea4f882ec108e/users/65df6e06a0fd4e349c9a75fde1556870/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:17.885021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324589aa-dd97-4e7b-a0cf-76ad67a685a0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:17.885310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324589aa-dd97-4e7b-a0cf-76ad67a685a0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:17.885618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324589aa-dd97-4e7b-a0cf-76ad67a685a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:17.959349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-324589aa-dd97-4e7b-a0cf-76ad67a685a0 None None] MFA Rules not processed for user `65df6e06a0fd4e349c9a75fde1556870`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:17.961003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-324589aa-dd97-4e7b-a0cf-76ad67a685a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:18.037619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:18.040806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:18.124660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2068/4128] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:18.128680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324589aa-dd97-4e7b-a0cf-76ad67a685a0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:18.182683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:18.198970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b330e-32ac-4fdb-8c14-40f7701fb6c7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:18.220456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:18.221711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:18.222148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:18.226451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] PATH_INFO: `/v3/projects/a1f37517f6b44de988179cf62fd527a9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:18.231375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1f37517f6b44de988179cf62fd527a9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:18.234047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:18.241770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324589aa-dd97-4e7b-a0cf-76ad67a685a0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:18.248771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:18.249918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:18.250420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:18.250869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:18.251666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:18.254895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:18.305224 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2069/4129] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:18 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:18.309401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:18.362156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:18.394528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1f37517f6b44de988179cf62fd527a9', 'request_id': 'req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:18.410409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1f37517f6b44de988179cf62fd527a9', 'request_id': 'req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:18.425072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1f37517f6b44de988179cf62fd527a9', 'request_id': 'req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:18.466777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] The token cache is being invalidate because project a1f37517f6b44de988179cf62fd527a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:18.467057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:18.468721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7f5d24e-234c-4cb0-91c7-79b9ac6c74c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:18.477605 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2061/4130] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:18 2022] DELETE /identity/v3/projects/a1f37517f6b44de988179cf62fd527a9 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:18.478123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c0623-f908-4d90-9387-8dc3242a1045 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:18.479931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25390889-1408-47f7-bcee-68d0a1594ff7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:18.480219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25390889-1408-47f7-bcee-68d0a1594ff7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:18.480497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25390889-1408-47f7-bcee-68d0a1594ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:18.530049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-25390889-1408-47f7-bcee-68d0a1594ff7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:18.534264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-25390889-1408-47f7-bcee-68d0a1594ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:18.607983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:18.608650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:18.608907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:18.609353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:18.610094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:18.616259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:18.667356 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2070/4131] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:18 2022] POST /identity/v3/users => generated 299 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:05:18.675085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:18.677628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:18.704130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce51997-22b5-4487-888d-acf749547664 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:18.711276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:18.712722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:18.713166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:18.713646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:18.716632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:18.718535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:18.748043 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2071/4132] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:18.749618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:18.773028 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2062/4133] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:18.782333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0ba59-009e-47cc-bc55-cb7f1a71a1bd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:18.782852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0ba59-009e-47cc-bc55-cb7f1a71a1bd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:18.783065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf65b-e6db-4e4e-bcae-846e1262d0df admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:18.783488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0ba59-009e-47cc-bc55-cb7f1a71a1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:18.793353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:18.795167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:18.796217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:18.797387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] PATH_INFO: `/v3/projects/9be27e27d7b841a8bd6492a89a6e9e5d/users/2b62417b11174b0492069d5454ad2e2d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:18.830221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5cd0ba59-009e-47cc-bc55-cb7f1a71a1bd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:18.831875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5cd0ba59-009e-47cc-bc55-cb7f1a71a1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:18.832029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be27e27d7b841a8bd6492a89a6e9e5d, user_id=2b62417b11174b0492069d5454ad2e2d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:18.836258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:18.860590 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2063/4134] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:18.860798 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2072/4135] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:18 2022] PUT /identity/v3/projects/9be27e27d7b841a8bd6492a89a6e9e5d/users/2b62417b11174b0492069d5454ad2e2d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:18.864950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0ba59-009e-47cc-bc55-cb7f1a71a1bd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:18.867629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:18.884339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0ba59-009e-47cc-bc55-cb7f1a71a1bd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.032154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.045745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f2178-dd3e-4c72-a756-45bdc9270f2e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.046968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.047594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.052939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.054063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:19.054733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.055500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.055645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.055886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.056530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.056667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.058188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.062115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.084676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2073/4136] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:19.091452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:19.096138 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2064/4137] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:19.100650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:19.108110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc23e-3e05-473d-81d6-950b9bd46342 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.119829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672bb28-0c99-41ee-ad5a-fd9682632585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.120604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672bb28-0c99-41ee-ad5a-fd9682632585 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.120886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672bb28-0c99-41ee-ad5a-fd9682632585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.121157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672bb28-0c99-41ee-ad5a-fd9682632585 admin admin] PATH_INFO: `/v3/projects/9be27e27d7b841a8bd6492a89a6e9e5d/users/2b62417b11174b0492069d5454ad2e2d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.124044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4b5c3-4ab0-488a-8bbb-29ac621f4e53 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.125682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672bb28-0c99-41ee-ad5a-fd9682632585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be27e27d7b841a8bd6492a89a6e9e5d, user_id=2b62417b11174b0492069d5454ad2e2d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.132416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.133056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.133338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.133591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672bb28-0c99-41ee-ad5a-fd9682632585 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.133713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.134420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.136368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.149163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:19.168972 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2074/4138] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:19 2022] PUT /identity/v3/projects/9be27e27d7b841a8bd6492a89a6e9e5d/users/2b62417b11174b0492069d5454ad2e2d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:19.179673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.180084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.180371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.193499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:19.215551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2065/4139] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:19 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:19.217208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:19.241050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] MFA Rules not processed for user `2b62417b11174b0492069d5454ad2e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:19.242605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:19.369496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c449d1-970c-4b2a-81f4-26cfd6bed311 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.385432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.386245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.386591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.386928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.387652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.390009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.395267 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2075/4140] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:19.403007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:19.404537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:19.449414 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2066/4141] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:19 2022] POST /identity/v3/users => generated 291 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:19.453367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:19.473858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b0edc-2b29-4c96-9e56-96b2ca9c42c4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.487131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.487881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.488281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.488682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.489483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.496194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.531974 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2067/4142] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:19.533231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:19.562689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21286e69-fc94-4fa5-a756-f7bb535df770 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.565780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74153fe5-780d-48bb-8d06-3d19dd0119c4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.568471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e9015-f155-4152-b31f-369ee4e9dc3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.569392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e9015-f155-4152-b31f-369ee4e9dc3d service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.570036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e9015-f155-4152-b31f-369ee4e9dc3d service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.570499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e9015-f155-4152-b31f-369ee4e9dc3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.571512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.572239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.572620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.573754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.574617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.577313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.595533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e9015-f155-4152-b31f-369ee4e9dc3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.597082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e9015-f155-4152-b31f-369ee4e9dc3d service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.606208 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2068/4143] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:19.607761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:19.614748 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2076/4144] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:19.649391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f11188-cd9e-4036-a4b7-87c863aa204f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:19.663273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:19.664774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.665240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.665806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] PATH_INFO: `/v3/projects/1c307636beed407582da5779796cc6a7/users/30b19896bc4e4d14bfec46dd00519150/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.697375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c307636beed407582da5779796cc6a7, user_id=30b19896bc4e4d14bfec46dd00519150, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:19.701158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:19.734640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0526b2-d82c-4946-ba48-028c8d1cc9d0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:19.735846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0526b2-d82c-4946-ba48-028c8d1cc9d0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:19.736324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0526b2-d82c-4946-ba48-028c8d1cc9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:19.742817 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2069/4145] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:19 2022] PUT /identity/v3/projects/1c307636beed407582da5779796cc6a7/users/30b19896bc4e4d14bfec46dd00519150/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:19.786215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2b0526b2-d82c-4946-ba48-028c8d1cc9d0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:19.787542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2b0526b2-d82c-4946-ba48-028c8d1cc9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:19.941654 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2077/4146] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:20.000614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:20.002967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:20.152288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5ebde-7e6f-471a-a566-bd724165683d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:20.160759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704ffea-0b05-4dbf-a5fd-123008b63474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:20.161415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704ffea-0b05-4dbf-a5fd-123008b63474 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:20.161714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704ffea-0b05-4dbf-a5fd-123008b63474 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:20.161979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704ffea-0b05-4dbf-a5fd-123008b63474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:20.182104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704ffea-0b05-4dbf-a5fd-123008b63474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:20.183612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704ffea-0b05-4dbf-a5fd-123008b63474 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:20.197721 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2070/4147] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:20.655362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0526b2-d82c-4946-ba48-028c8d1cc9d0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:20.683042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0526b2-d82c-4946-ba48-028c8d1cc9d0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:20.698229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:20.699130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:20.699575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:20.700010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] PATH_INFO: `/v3/users/65df6e06a0fd4e349c9a75fde1556870` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:20.701383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65df6e06a0fd4e349c9a75fde1556870)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:20.703563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:20.776362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:20.827743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65df6e06a0fd4e349c9a75fde1556870'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:20.853949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65df6e06a0fd4e349c9a75fde1556870'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:20.866308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65df6e06a0fd4e349c9a75fde1556870'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:20.867271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65df6e06a0fd4e349c9a75fde1556870'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:20.868094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65df6e06a0fd4e349c9a75fde1556870'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:20.907741 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2078/4148] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:20 2022] DELETE /identity/v3/users/65df6e06a0fd4e349c9a75fde1556870 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:20.912193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:20.914036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:20.914770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:20.938523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:20.943445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:21.010905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:21.078985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2071/4149] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:21.089123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:21.091492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:21.291691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f381-abfc-4c74-b0d3-d221020d9206 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:21.299494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:21.300466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:21.301100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:21.303163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:21.318129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:21.322098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:21.346100 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2079/4150] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:21.704441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:21.706149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:21.730187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a4ae4-9c7b-4201-89d9-992ba7f99c30 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:21.742964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:21.744293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:21.744293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:21.744510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] PATH_INFO: `/v3/users/e8f7671df52e4e95b829f03defeb6cf5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:21.745634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f7671df52e4e95b829f03defeb6cf5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:21.747670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:21.847939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f7671df52e4e95b829f03defeb6cf5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:21.849019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f7671df52e4e95b829f03defeb6cf5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:21.860907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f7671df52e4e95b829f03defeb6cf5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:21.862503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f7671df52e4e95b829f03defeb6cf5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:21.879838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f7671df52e4e95b829f03defeb6cf5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:21.896900 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2072/4151] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:21 2022] DELETE /identity/v3/users/e8f7671df52e4e95b829f03defeb6cf5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:22.047871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.049405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:22.074873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ba49-2d38-4ddc-b7b8-013166fd2750 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.140708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.143187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:22.156581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.157168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.157546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.157894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] PATH_INFO: `/v3/projects/ed058afc9e9c4a6fa8eea4f882ec108e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.159372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed058afc9e9c4a6fa8eea4f882ec108e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.165705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.216659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:22.237173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed058afc9e9c4a6fa8eea4f882ec108e', 'request_id': 'req-ebc5a026-fbde-4ca3-97c6-6a970691556e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.239085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278b028-8e72-42e5-a88c-bee9b90c3d8b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.247995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed058afc9e9c4a6fa8eea4f882ec108e', 'request_id': 'req-ebc5a026-fbde-4ca3-97c6-6a970691556e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.255268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed058afc9e9c4a6fa8eea4f882ec108e', 'request_id': 'req-ebc5a026-fbde-4ca3-97c6-6a970691556e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.269528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] The token cache is being invalidate because project ed058afc9e9c4a6fa8eea4f882ec108e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:22.269890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.270491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.272401 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2080/4152] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:22 2022] DELETE /identity/v3/projects/ed058afc9e9c4a6fa8eea4f882ec108e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:22.276240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.277889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:22.302085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:22.312116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.312779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.313841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.314602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] PATH_INFO: `/v3/projects/765dd4c95190458594be8f9a7915210e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.316012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=765dd4c95190458594be8f9a7915210e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.318063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.372204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5a026-fbde-4ca3-97c6-6a970691556e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.384873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765dd4c95190458594be8f9a7915210e', 'request_id': 'req-2be2b79c-f2e0-4668-b6dc-b641d74617b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.394453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765dd4c95190458594be8f9a7915210e', 'request_id': 'req-2be2b79c-f2e0-4668-b6dc-b641d74617b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.407359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765dd4c95190458594be8f9a7915210e', 'request_id': 'req-2be2b79c-f2e0-4668-b6dc-b641d74617b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.424273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] The token cache is being invalidate because project 765dd4c95190458594be8f9a7915210e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:22.424734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.425602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2be2b79c-f2e0-4668-b6dc-b641d74617b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.429075 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2073/4153] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:22 2022] DELETE /identity/v3/projects/765dd4c95190458594be8f9a7915210e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:22.433384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ded8-8344-4de1-a32f-642ee5d61a99 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.433770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ded8-8344-4de1-a32f-642ee5d61a99 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.434122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ded8-8344-4de1-a32f-642ee5d61a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.465876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0a11ded8-8344-4de1-a32f-642ee5d61a99 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:22.467505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.467766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a11ded8-8344-4de1-a32f-642ee5d61a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:22.468544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.469778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.470181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] PATH_INFO: `/v3/users/2b62417b11174b0492069d5454ad2e2d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.471740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b62417b11174b0492069d5454ad2e2d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.474578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.479691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2074/4154] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:22.487388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a5388-4c09-445d-b99d-8911b16ad7d5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.489894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a5388-4c09-445d-b99d-8911b16ad7d5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.490431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a5388-4c09-445d-b99d-8911b16ad7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.518817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3f5a5388-4c09-445d-b99d-8911b16ad7d5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:22.520223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3f5a5388-4c09-445d-b99d-8911b16ad7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:22.530652 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2075/4155] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:22.535059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a5388-4c09-445d-b99d-8911b16ad7d5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.538328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b62417b11174b0492069d5454ad2e2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.544875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a5388-4c09-445d-b99d-8911b16ad7d5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.550371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.550992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.551318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.551996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.552431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:22.552633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.553515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b62417b11174b0492069d5454ad2e2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.554398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.559170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:22.567769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b62417b11174b0492069d5454ad2e2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.570007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b62417b11174b0492069d5454ad2e2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.571429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b62417b11174b0492069d5454ad2e2d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:22.572497 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2076/4156] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:05:22.573396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.575368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:22.586919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2081/4157] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:22 2022] DELETE /identity/v3/users/2b62417b11174b0492069d5454ad2e2d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:22.594615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229ad97-b91b-4706-9671-cf23919865cf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.664867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.665629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.666084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.666366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.667011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.675567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.717819 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2077/4158] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:22 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:22.722176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.739136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947b834-f9de-46be-9ce6-0e34f8df9caf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.818806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.819532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.819874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.820282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.821001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.823641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.853816 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2082/4159] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:22 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:22.855078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.866938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc8753-5273-415e-aaa7-776ba85143b8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.873597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.874611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.874895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.875172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.875845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.877521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.912920 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2078/4160] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:22.916621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:22.936550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1120e7-d914-423f-8cd1-713d0b3a9346 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:22.954112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:22.954882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:22.955106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:22.955496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:22.956390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:22.958781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:22.986908 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2083/4161] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:22.988401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:23.008505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbbe0-9f38-4706-96ae-f730df9b86b5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:23.022997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c08fb-b666-4e29-a7fe-9c1d8b5ebf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:23.024471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c08fb-b666-4e29-a7fe-9c1d8b5ebf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.025910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c08fb-b666-4e29-a7fe-9c1d8b5ebf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.026960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c08fb-b666-4e29-a7fe-9c1d8b5ebf3d admin admin] PATH_INFO: `/v3/projects/f8ed24e4bf6643f7b1474fcec78c4995/users/795813bb461e42ad8c2a758e9b8430d6/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.082539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c08fb-b666-4e29-a7fe-9c1d8b5ebf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ed24e4bf6643f7b1474fcec78c4995, user_id=795813bb461e42ad8c2a758e9b8430d6, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:23.085669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c08fb-b666-4e29-a7fe-9c1d8b5ebf3d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:23.099708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:23.101593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:23.122621 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2079/4162] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:22 2022] PUT /identity/v3/projects/f8ed24e4bf6643f7b1474fcec78c4995/users/795813bb461e42ad8c2a758e9b8430d6/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:23.128095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.128741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.129017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.130426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4874589-bb22-4f95-a05c-cb6aecff6e8c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:23.193444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:23.194728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:23.208080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:23.266144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:23.266857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.267142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.267416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] PATH_INFO: `/v3/projects/9be27e27d7b841a8bd6492a89a6e9e5d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.269146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be27e27d7b841a8bd6492a89a6e9e5d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:23.271875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:23.314361 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2080/4163] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:23.373323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:23.379801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:23.383863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be27e27d7b841a8bd6492a89a6e9e5d', 'request_id': 'req-99a969b7-95fc-4c8a-836e-31e4fd103d10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:23.432920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be27e27d7b841a8bd6492a89a6e9e5d', 'request_id': 'req-99a969b7-95fc-4c8a-836e-31e4fd103d10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:23.441406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be27e27d7b841a8bd6492a89a6e9e5d', 'request_id': 'req-99a969b7-95fc-4c8a-836e-31e4fd103d10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:23.442792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:23.473140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] The token cache is being invalidate because project 9be27e27d7b841a8bd6492a89a6e9e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:23.474801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:23.476565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-99a969b7-95fc-4c8a-836e-31e4fd103d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:23.478638 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2084/4164] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:23 2022] DELETE /identity/v3/projects/9be27e27d7b841a8bd6492a89a6e9e5d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:23.484008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9cbfb-2c61-4631-84f6-5ac6a12983bc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.484525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9cbfb-2c61-4631-84f6-5ac6a12983bc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.484525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9cbfb-2c61-4631-84f6-5ac6a12983bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.527817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e8b9cbfb-2c61-4631-84f6-5ac6a12983bc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:23.531661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8b9cbfb-2c61-4631-84f6-5ac6a12983bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:23.569770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d696882-b81e-4e69-99a9-2086f3a7e389 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:23.739378 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2085/4165] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:23.741701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f127f8-4690-40b0-b267-4b9d832c9568 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.741990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f127f8-4690-40b0-b267-4b9d832c9568 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.742265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f127f8-4690-40b0-b267-4b9d832c9568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.747663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:23.749227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.749522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.749796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.757029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:23.796365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-48f127f8-4690-40b0-b267-4b9d832c9568 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:23.797619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-48f127f8-4690-40b0-b267-4b9d832c9568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:23.805918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:23.808416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:23.812998 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2086/4166] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:23.814409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f127f8-4690-40b0-b267-4b9d832c9568 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:23.837259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f127f8-4690-40b0-b267-4b9d832c9568 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:23.839347 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2081/4167] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:23.845777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:23.846761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.847339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.847790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.849131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:23.849767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:23.852564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:23.890080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:23.901981 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2087/4168] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:23.904161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789d35c-458e-4e00-802f-6560fccd2984 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:23.918274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:23.919724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:23.920357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:23.921519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:23.922416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:23.930816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:23.988295 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2082/4169] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:23 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:23.992629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:24.004355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed4ff-4184-422c-9d1a-18292c39da62 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:24.075131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:24.075860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:24.076253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:24.076620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:24.077509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:24.079566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:24.127032 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2088/4170] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:23 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:24.132164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:24.153780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a4e7a-ec13-4cf2-b93f-a96ac52a9adf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:24.162537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:24.163336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:24.163620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:24.163890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:24.164597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:24.167616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:24.182868 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2083/4171] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:24.187341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:24.202499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef62bd-da14-43bf-8607-a177459e98b3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:24.216245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efb566-d212-4c56-a6e2-c2bd4049fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:24.217194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efb566-d212-4c56-a6e2-c2bd4049fff8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:24.217497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efb566-d212-4c56-a6e2-c2bd4049fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:24.217847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efb566-d212-4c56-a6e2-c2bd4049fff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:24.218502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efb566-d212-4c56-a6e2-c2bd4049fff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:24.220875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efb566-d212-4c56-a6e2-c2bd4049fff8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:24.238008 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2089/4172] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:24.239487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:24.265407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9d05f-ad04-4544-8041-50855d7b8e95 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:24.279668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:24.281115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:24.282032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:24.284041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] PATH_INFO: `/v3/projects/598139a6b6764e80822b82231f572770/users/1933f2aed04a45a28bb2617337faa719/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:24.311171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=598139a6b6764e80822b82231f572770, user_id=1933f2aed04a45a28bb2617337faa719, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:24.323936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:24.362251 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2084/4173] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:24 2022] PUT /identity/v3/projects/598139a6b6764e80822b82231f572770/users/1933f2aed04a45a28bb2617337faa719/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:24.368577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e3a5d-d138-479b-8617-01aa91e586df None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:24.368577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e3a5d-d138-479b-8617-01aa91e586df None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:24.369585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e3a5d-d138-479b-8617-01aa91e586df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:24.410367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f58e3a5d-d138-479b-8617-01aa91e586df None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:24.411731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f58e3a5d-d138-479b-8617-01aa91e586df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:24.513787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:24.545737 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2090/4174] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:24.573515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:24.576479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:24.613311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:24.749307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43497fbe-5a99-4392-840e-5116505a4997 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:24.756537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3e716-006b-4a28-b7fe-76a8f044e6ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:24.757599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3e716-006b-4a28-b7fe-76a8f044e6ee service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:24.758518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3e716-006b-4a28-b7fe-76a8f044e6ee service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:24.759056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3e716-006b-4a28-b7fe-76a8f044e6ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:24.769930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3e716-006b-4a28-b7fe-76a8f044e6ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:24.771832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3e716-006b-4a28-b7fe-76a8f044e6ee service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:24.792182 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2085/4175] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:26.394653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e3a5d-d138-479b-8617-01aa91e586df None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:26.416228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e3a5d-d138-479b-8617-01aa91e586df None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:26.420282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:26.421112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:26.421588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:26.422009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:26.424623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:26.464808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:26.466868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:26.493121 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2091/4176] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:27.362976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620ae7c-0ed4-4549-bcca-d5486d31c668 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:27.364039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620ae7c-0ed4-4549-bcca-d5486d31c668 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:27.364903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620ae7c-0ed4-4549-bcca-d5486d31c668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:27.443440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f620ae7c-0ed4-4549-bcca-d5486d31c668 None None] MFA Rules not processed for user `1933f2aed04a45a28bb2617337faa719`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:27.444779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f620ae7c-0ed4-4549-bcca-d5486d31c668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:27.553105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2086/4177] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:27.557732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:27.570633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703039a-cc3f-4587-a27c-aa05bc0f0cc6 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:27.577432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:27.578411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:27.578857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:27.579292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:27.580405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:27.582650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:27.634501 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2092/4178] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:27 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:27.638435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620ae7c-0ed4-4549-bcca-d5486d31c668 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:27.652795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620ae7c-0ed4-4549-bcca-d5486d31c668 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:27.810603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:27.812956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:27.813248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:27.813569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:27.825007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:27.827355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:27.882328 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2087/4179] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:27 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:27.886756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:27.900516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2196722-e4f9-45e9-bf0e-45a823c1ff9a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:27.917017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:27.917680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:27.918332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:27.918899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:27.919750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:27.921670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:27.948627 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2093/4180] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:27.952730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:27.967909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877cf5f-6fa6-4de2-8c96-984b7edff1a9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:27.979909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:27.982931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:27.983087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:27.983255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] PATH_INFO: `/v3/projects/801cfabc02454546b8a1488856e88798/users/c71dfa6b81844d6b84f1c3eaade8fc86/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:28.021726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=801cfabc02454546b8a1488856e88798, user_id=c71dfa6b81844d6b84f1c3eaade8fc86, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:28.024423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:28.059772 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2088/4181] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:27 2022] PUT /identity/v3/projects/801cfabc02454546b8a1488856e88798/users/c71dfa6b81844d6b84f1c3eaade8fc86/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:28.064545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:28.077625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bb3f9-2236-4908-bd56-ba0cf95b37b3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:28.176984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271034fc-5821-4713-a702-c10856afbbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:28.178166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271034fc-5821-4713-a702-c10856afbbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:28.178655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271034fc-5821-4713-a702-c10856afbbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:28.179122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271034fc-5821-4713-a702-c10856afbbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:28.180094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271034fc-5821-4713-a702-c10856afbbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:28.182104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271034fc-5821-4713-a702-c10856afbbb5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:28.206920 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2094/4182] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:28.211432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:28.227221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668265d-12d0-4943-84bf-3cd8d005cc6f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:28.236606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56caeb01-9777-4dbe-894c-e463d53547a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:28.237476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56caeb01-9777-4dbe-894c-e463d53547a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:28.239839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56caeb01-9777-4dbe-894c-e463d53547a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:28.240011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56caeb01-9777-4dbe-894c-e463d53547a3 admin admin] PATH_INFO: `/v3/projects/801cfabc02454546b8a1488856e88798/users/c71dfa6b81844d6b84f1c3eaade8fc86/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:28.244214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56caeb01-9777-4dbe-894c-e463d53547a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801cfabc02454546b8a1488856e88798, user_id=c71dfa6b81844d6b84f1c3eaade8fc86, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:28.253576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56caeb01-9777-4dbe-894c-e463d53547a3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:28.261210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:28.281079 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2089/4183] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:28 2022] PUT /identity/v3/projects/801cfabc02454546b8a1488856e88798/users/c71dfa6b81844d6b84f1c3eaade8fc86/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:31.914249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:31.914749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:31.914749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:31.999849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] MFA Rules not processed for user `c71dfa6b81844d6b84f1c3eaade8fc86`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:32.001537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:32.191555 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2095/4184] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:32.202753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4b3ab-4897-4dba-99f6-ca0d57aa9c81 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:32.203491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4b3ab-4897-4dba-99f6-ca0d57aa9c81 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:32.203944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4b3ab-4897-4dba-99f6-ca0d57aa9c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:32.255888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c2e4b3ab-4897-4dba-99f6-ca0d57aa9c81 None None] MFA Rules not processed for user `1933f2aed04a45a28bb2617337faa719`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:32.262780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c2e4b3ab-4897-4dba-99f6-ca0d57aa9c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:32.413430 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2090/4185] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:32.428325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:32.429846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:32.469334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:32.469949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:32.469949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:32.514047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:32.523047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] MFA Rules not processed for user `24c7a82126124ebb8baf5c656ff9849d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:32.524972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:32.648851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93381-ccc7-4d09-9d9c-e591a316bd01 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:32.663727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:32.664367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:32.666086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:32.666362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:32.703615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:32.708668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:32.724228 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2091/4186] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:32.725435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:32.733216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:32.748818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2096/4187] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:32.850433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:32.851755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:32.869089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7893f6b-8497-4fed-8068-48769b37dc7e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:32.886870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:32.888607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:32.889768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:32.890056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:32.890728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:32.902528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:32.994011 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2092/4188] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:32 2022] POST /identity/v3/projects => generated 385 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:32.998741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:33.026535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56794002-43d6-4dcb-9f14-57e3eb61f54a service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:33.038789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:33.166285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d427-943d-445b-93fa-9dea0c247082 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:33.167406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:33.168530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:33.168808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:33.170881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:33.173108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:33.177925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:33.179299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:33.180297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:33.180807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:33.181727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:33.184534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:33.246314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2093/4189] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:32 2022] POST /identity/v3/users => generated 299 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:33.247453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:33.264526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621554b-6958-465e-8540-d3cb63e2169e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:33.272121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:33.273180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:33.273709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:33.274218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:33.275141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:33.278147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:33.316241 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2094/4190] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:33.318549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:33.323448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:33.326697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:33.349560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7233f-6a1f-4973-9b85-fc68b7e07429 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:33.370907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:33.373002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:33.375175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:33.375774 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2097/4191] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:33.376019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] PATH_INFO: `/v3/projects/f1fca09044e249e497bc266060c3c3ab/users/fce8fdffbc354641b50a0d46ea6be26f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:33.400484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fca09044e249e497bc266060c3c3ab, user_id=fce8fdffbc354641b50a0d46ea6be26f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:33.403751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:33.427941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2095/4192] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:33 2022] PUT /identity/v3/projects/f1fca09044e249e497bc266060c3c3ab/users/fce8fdffbc354641b50a0d46ea6be26f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:33.434853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:33.452285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415de0a-4dff-411e-9608-50fd6be9a7df service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:33.487627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:33.617072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:33.617754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:33.618022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:33.618296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:33.618956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:33.620610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:33.651703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:33.657125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2098/4193] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:33.672792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621cc19-2933-4b0f-8395-18f936e34b75 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:33.683089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:33.683787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:33.684065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:33.684340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] PATH_INFO: `/v3/projects/f1fca09044e249e497bc266060c3c3ab/users/fce8fdffbc354641b50a0d46ea6be26f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:33.692304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fca09044e249e497bc266060c3c3ab, user_id=fce8fdffbc354641b50a0d46ea6be26f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:33.692304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:33.714582 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2096/4194] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:33 2022] PUT /identity/v3/projects/f1fca09044e249e497bc266060c3c3ab/users/fce8fdffbc354641b50a0d46ea6be26f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:33.838942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:33.840753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:34.012656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6808f-b8a5-4525-86a6-fbdf7db6f944 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:34.176404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:34.177025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:34.177302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:34.177602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:34.191327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:34.355922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:34.357423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:34.379434 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2099/4195] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:34.547716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:34.549633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:34.731768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e0b10-c356-439b-afb5-717500f94d33 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:34.745368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:34.746069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:34.746349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:34.746619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] PATH_INFO: `/v3/users/9e38b0461884497e8c5d399051b3d50e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:34.748828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e38b0461884497e8c5d399051b3d50e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:34.750881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:34.869791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e38b0461884497e8c5d399051b3d50e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:34.871321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e38b0461884497e8c5d399051b3d50e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:34.889981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e38b0461884497e8c5d399051b3d50e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:34.890591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e38b0461884497e8c5d399051b3d50e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:34.899034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:34.919931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e38b0461884497e8c5d399051b3d50e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:34.953875 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2097/4196] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:34 2022] DELETE /identity/v3/users/9e38b0461884497e8c5d399051b3d50e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:35.075687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:35.077306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:35.104158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:35.118590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cd7b8-ad95-4f2b-9eae-35c137b38d50 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:35.264557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:35.266331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:35.266850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:35.267368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] PATH_INFO: `/v3/users/33fa2d463884464fbe90666c98da0fd7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:35.269338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33fa2d463884464fbe90666c98da0fd7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:35.272208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:35.384983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33fa2d463884464fbe90666c98da0fd7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.411529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33fa2d463884464fbe90666c98da0fd7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.424202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33fa2d463884464fbe90666c98da0fd7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.425442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33fa2d463884464fbe90666c98da0fd7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.426598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f867ee48-ccd0-4f92-929d-7db797831cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33fa2d463884464fbe90666c98da0fd7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.467265 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2100/4197] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:35 2022] DELETE /identity/v3/users/33fa2d463884464fbe90666c98da0fd7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:35.637803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:35.641289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:35.665221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa375a53-72d6-40ba-a8b0-b7b981966f21 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:35.805682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:35.806579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:35.807035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:35.807766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] PATH_INFO: `/v3/projects/6cc3ec11e1374785afe8d3f73bc26891` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:35.812014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc3ec11e1374785afe8d3f73bc26891)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:35.814278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:35.913585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc3ec11e1374785afe8d3f73bc26891', 'request_id': 'req-333cdbe9-4bdc-4e81-9b93-d6028924eddd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.925805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc3ec11e1374785afe8d3f73bc26891', 'request_id': 'req-333cdbe9-4bdc-4e81-9b93-d6028924eddd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.960208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc3ec11e1374785afe8d3f73bc26891', 'request_id': 'req-333cdbe9-4bdc-4e81-9b93-d6028924eddd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.004518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] The token cache is being invalidate because project 6cc3ec11e1374785afe8d3f73bc26891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:36.004897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.006169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.009427 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2098/4198] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:35 2022] DELETE /identity/v3/projects/6cc3ec11e1374785afe8d3f73bc26891 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:36.015784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db301c-6423-4fd6-915d-087f25a0dd4e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.016067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db301c-6423-4fd6-915d-087f25a0dd4e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.016495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db301c-6423-4fd6-915d-087f25a0dd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.048290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:36.049796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:36.059880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f6db301c-6423-4fd6-915d-087f25a0dd4e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:36.064543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f6db301c-6423-4fd6-915d-087f25a0dd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:36.197157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cdbe9-4bdc-4e81-9b93-d6028924eddd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:36.219837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:36.220512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.220790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.221062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] PATH_INFO: `/v3/projects/36ce01157eec446dbcc93b2948b4be90` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.222438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ce01157eec446dbcc93b2948b4be90)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:36.224445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:36.244663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:36.260640 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2101/4199] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:36.263863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b81d2-0555-4a77-9883-b519a0b3e604 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.264563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b81d2-0555-4a77-9883-b519a0b3e604 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.265228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b81d2-0555-4a77-9883-b519a0b3e604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.329161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-db1b81d2-0555-4a77-9883-b519a0b3e604 None None] MFA Rules not processed for user `30b19896bc4e4d14bfec46dd00519150`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:36.331388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-db1b81d2-0555-4a77-9883-b519a0b3e604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:36.344983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ce01157eec446dbcc93b2948b4be90', 'request_id': 'req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.348892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:36.366654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ce01157eec446dbcc93b2948b4be90', 'request_id': 'req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.388572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ce01157eec446dbcc93b2948b4be90', 'request_id': 'req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.415028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] The token cache is being invalidate because project 36ce01157eec446dbcc93b2948b4be90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:36.415344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.416065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0db539c2-c3bd-4049-a6cf-159e1e24bb9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.417905 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2099/4200] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:36 2022] DELETE /identity/v3/projects/36ce01157eec446dbcc93b2948b4be90 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:36.419127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe3434-0779-4124-9bb6-0d650c070331 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.419406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe3434-0779-4124-9bb6-0d650c070331 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.419675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe3434-0779-4124-9bb6-0d650c070331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.469893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0cfe3434-0779-4124-9bb6-0d650c070331 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:36.471125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0cfe3434-0779-4124-9bb6-0d650c070331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:36.716541 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2100/4201] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:36.716541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec5c0c-39bf-419e-abe1-30bbd2fbabac None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.716541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec5c0c-39bf-419e-abe1-30bbd2fbabac None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.716541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec5c0c-39bf-419e-abe1-30bbd2fbabac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.758262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0ec5c0c-39bf-419e-abe1-30bbd2fbabac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:36.761374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0ec5c0c-39bf-419e-abe1-30bbd2fbabac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:36.771938 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2101/4202] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:36.772666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec5c0c-39bf-419e-abe1-30bbd2fbabac None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:36.798533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec5c0c-39bf-419e-abe1-30bbd2fbabac None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:36.805932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62ee08-0406-4ded-a2b3-6fdd59f90d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:36.826540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62ee08-0406-4ded-a2b3-6fdd59f90d95 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.827318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62ee08-0406-4ded-a2b3-6fdd59f90d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.828586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62ee08-0406-4ded-a2b3-6fdd59f90d95 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.829612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62ee08-0406-4ded-a2b3-6fdd59f90d95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:36.830111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62ee08-0406-4ded-a2b3-6fdd59f90d95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:36.832152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62ee08-0406-4ded-a2b3-6fdd59f90d95 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:36.854948 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2102/4203] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:36.857940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd59cf3-5a67-4221-80e9-4e0d2fa76bc4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.858348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd59cf3-5a67-4221-80e9-4e0d2fa76bc4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.858855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd59cf3-5a67-4221-80e9-4e0d2fa76bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.911844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2dd59cf3-5a67-4221-80e9-4e0d2fa76bc4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:36.915155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2dd59cf3-5a67-4221-80e9-4e0d2fa76bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:36.944377 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2103/4204] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:36.945182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd59cf3-5a67-4221-80e9-4e0d2fa76bc4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:36.969564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd59cf3-5a67-4221-80e9-4e0d2fa76bc4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:36.981012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:36.990622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2102/4205] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 729 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:36.991501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b81d2-0555-4a77-9883-b519a0b3e604 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.001706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.002814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.003996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.007348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.011545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.019655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b81d2-0555-4a77-9883-b519a0b3e604 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.032204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.033123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.033687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.034138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.035029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:37.045988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.048097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.068715 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2103/4206] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:37.074839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.078436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:37.110040 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2104/4207] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:36 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:37.112818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.225983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:37.274913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32a82a-f2c1-40f9-a2bc-e98c7a66ee8e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.283065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd4ecb-1833-4bbc-9d4a-e926b4dada1f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.289898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.290553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.290839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.292701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.293736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.299454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.382245 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2105/4208] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/projects => generated 385 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:37.383196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.463914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.464611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.464887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.465161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.568720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:37.588885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acbb36-d0a7-44f3-bea4-5ae11daf13e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.608898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.609722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.610231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.612440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.614215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.616964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.619167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.621884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.635319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2104/4209] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:37.636273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.656592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977924d0-dda2-4545-92d4-15788b2fafa6 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.666457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.667271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.667555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.668920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.670810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.675495 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2106/4210] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:37.675985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.680004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.708857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e78268-5622-478f-af78-853f17f24326 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.727643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.729022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.729576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.730655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.731663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.738099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.740734 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2105/4211] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:37.748697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.767368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14d58e-6733-4184-a9a9-c74f6603a076 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.773829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.774584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.774959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.775779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.776550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.779461 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2107/4212] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:37.784029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.784920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.800923 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2106/4213] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:37.805051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.833717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f5830-712f-4ea8-93ff-a631e346e485 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.839848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820cf59-89fb-488c-b457-b300b92b8ea6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.840493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.841124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.841400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.841761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.842414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.848982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.869886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.870594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.871071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.871352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.872002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.874499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2108/4214] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:37.879367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.882211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.902729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe4316-c22f-4390-98a4-c162d26c5b7c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.916649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811c961-2afe-412a-8d29-12b16a40c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.920186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811c961-2afe-412a-8d29-12b16a40c65c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.920492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811c961-2afe-412a-8d29-12b16a40c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.920771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811c961-2afe-412a-8d29-12b16a40c65c admin admin] PATH_INFO: `/v3/projects/4512f54506dd413a80df4b337a6c31c6/users/6b27ba12dd5a46b3b27479c3a7c241e5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.934702 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2107/4215] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:37.935939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.963813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811c961-2afe-412a-8d29-12b16a40c65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4512f54506dd413a80df4b337a6c31c6, user_id=6b27ba12dd5a46b3b27479c3a7c241e5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.973865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811c961-2afe-412a-8d29-12b16a40c65c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.003967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1263a-15ab-4865-ba0e-8feea5b69816 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.036006 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2109/4216] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:37 2022] PUT /identity/v3/projects/4512f54506dd413a80df4b337a6c31c6/users/6b27ba12dd5a46b3b27479c3a7c241e5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:38.040146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1761523-f171-4be2-931d-5fbf88644a38 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.040435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1761523-f171-4be2-931d-5fbf88644a38 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.040710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1761523-f171-4be2-931d-5fbf88644a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.135038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b90346-0366-4854-ac66-3f10c0e9f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.135726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b90346-0366-4854-ac66-3f10c0e9f608 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.136002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b90346-0366-4854-ac66-3f10c0e9f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.136274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b90346-0366-4854-ac66-3f10c0e9f608 admin admin] PATH_INFO: `/v3/projects/6fa7e14863254ab3964ac907c168d9e6/users/c544e8e6f7a741c985c92ed7437397cd/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.141453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f1761523-f171-4be2-931d-5fbf88644a38 None None] MFA Rules not processed for user `6b27ba12dd5a46b3b27479c3a7c241e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:38.143227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1761523-f171-4be2-931d-5fbf88644a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.171551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b90346-0366-4854-ac66-3f10c0e9f608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa7e14863254ab3964ac907c168d9e6, user_id=c544e8e6f7a741c985c92ed7437397cd, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.177283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b90346-0366-4854-ac66-3f10c0e9f608 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.208480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2108/4217] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:37 2022] PUT /identity/v3/projects/6fa7e14863254ab3964ac907c168d9e6/users/c544e8e6f7a741c985c92ed7437397cd/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:38.213642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816665ca-1397-4e8d-9b09-74578d73e015 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.213925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816665ca-1397-4e8d-9b09-74578d73e015 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.214195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816665ca-1397-4e8d-9b09-74578d73e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.248778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-816665ca-1397-4e8d-9b09-74578d73e015 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:38.252389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-816665ca-1397-4e8d-9b09-74578d73e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.315717 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2110/4218] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:38.317329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.317680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.317962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.340690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:38.407375 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2109/4219] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:38.408361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] MFA Rules not processed for user `795813bb461e42ad8c2a758e9b8430d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:38.408805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8c78d-e4e1-4230-8769-06ce95bc0ba4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.409301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8c78d-e4e1-4230-8769-06ce95bc0ba4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.409796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8c78d-e4e1-4230-8769-06ce95bc0ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.422615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.458967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b3b8c78d-e4e1-4230-8769-06ce95bc0ba4 None None] MFA Rules not processed for user `6b27ba12dd5a46b3b27479c3a7c241e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:38.460201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b3b8c78d-e4e1-4230-8769-06ce95bc0ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.494169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:38.563721 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2111/4220] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:38.564737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.566331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.610708 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2110/4221] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:38.616278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.616632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.618246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.660608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] MFA Rules not processed for user `795813bb461e42ad8c2a758e9b8430d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:38.662182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.690770 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2111/4222] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:38.691924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.693581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.780847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32da4c-1a2e-49aa-90e1-24619f7f2654 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.790670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.791407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.791686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.791954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.813879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.815931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.828666 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2112/4223] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:38.884248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d86bb-106c-4eff-859c-b19b625d9196 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.895424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d530eed-c8ff-4599-b68d-c0799d4c36bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.898158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530eed-c8ff-4599-b68d-c0799d4c36bd service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.898622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530eed-c8ff-4599-b68d-c0799d4c36bd service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.898952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530eed-c8ff-4599-b68d-c0799d4c36bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.914366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d530eed-c8ff-4599-b68d-c0799d4c36bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.920273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d530eed-c8ff-4599-b68d-c0799d4c36bd service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.951848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2112/4224] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:39.193779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:39.200138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc3910-3bff-48b7-a9f7-6b18e60c296f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:39.202356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc3910-3bff-48b7-a9f7-6b18e60c296f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:39.202884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc3910-3bff-48b7-a9f7-6b18e60c296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:39.211933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e40e-6076-4ade-ba9a-42acf2d6a8b2 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:39.228811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:39.229496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:39.229784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:39.230087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:39.240285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-53fc3910-3bff-48b7-a9f7-6b18e60c296f None None] MFA Rules not processed for user `30b19896bc4e4d14bfec46dd00519150`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:39.244662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-53fc3910-3bff-48b7-a9f7-6b18e60c296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:39.248950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:39.252580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:39.284469 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2113/4225] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:39.358834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:39.361449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:39.452634 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2113/4226] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:39.505814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8269f55-3965-4a34-9904-3decdd089172 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:39.513322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bedb44-6320-4512-ac14-6c3b5711f465 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:39.513860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bedb44-6320-4512-ac14-6c3b5711f465 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:39.514137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bedb44-6320-4512-ac14-6c3b5711f465 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:39.514406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bedb44-6320-4512-ac14-6c3b5711f465 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:39.516849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-04bedb44-6320-4512-ac14-6c3b5711f465 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:39.533261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bedb44-6320-4512-ac14-6c3b5711f465 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:39.535248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bedb44-6320-4512-ac14-6c3b5711f465 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:39.568611 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2114/4227] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:39.734049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b17dc-a878-45c2-90af-e1b35a83f777 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:39.734348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b17dc-a878-45c2-90af-e1b35a83f777 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:39.734741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b17dc-a878-45c2-90af-e1b35a83f777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:39.772760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-083b17dc-a878-45c2-90af-e1b35a83f777 None None] MFA Rules not processed for user `6b27ba12dd5a46b3b27479c3a7c241e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:39.780419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-083b17dc-a878-45c2-90af-e1b35a83f777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:39.826260 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2114/4228] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:42.733569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aa9f9-9a1d-4c57-b6b8-f56a49bcbd8a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:42.734124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aa9f9-9a1d-4c57-b6b8-f56a49bcbd8a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:42.734124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aa9f9-9a1d-4c57-b6b8-f56a49bcbd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:42.809157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-681aa9f9-9a1d-4c57-b6b8-f56a49bcbd8a None None] MFA Rules not processed for user `795813bb461e42ad8c2a758e9b8430d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:42.829529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-681aa9f9-9a1d-4c57-b6b8-f56a49bcbd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:42.879410 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2115/4229] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:45.931350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b17dc-a878-45c2-90af-e1b35a83f777 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:45.955506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b17dc-a878-45c2-90af-e1b35a83f777 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:45.963552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:45.964895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:45.966424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:45.966908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:45.971897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:46.007576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:46.012423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:46.046787 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2115/4230] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:46.122447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bb93a-9c1b-4d24-aecf-1e6ec5c37360 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:46.122885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bb93a-9c1b-4d24-aecf-1e6ec5c37360 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:46.123077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bb93a-9c1b-4d24-aecf-1e6ec5c37360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:46.166491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bf1bb93a-9c1b-4d24-aecf-1e6ec5c37360 None None] MFA Rules not processed for user `2ef547e071fd43ec96c745c26eca119f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:46.169451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bf1bb93a-9c1b-4d24-aecf-1e6ec5c37360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:46.185324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:46.314744 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2116/4231] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:50.995102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:51.017034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1ecd-34f3-49e2-9edc-aae6eecff02d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:51.025806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:51.026703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:51.027145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:51.029348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:51.033374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:51.066820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:51.070547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:51.093003 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2116/4232] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:53.302937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a050a-ec22-4c7f-92d0-58927ca72d2b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:53.303947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a050a-ec22-4c7f-92d0-58927ca72d2b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:53.304229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a050a-ec22-4c7f-92d0-58927ca72d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:53.366484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-724a050a-ec22-4c7f-92d0-58927ca72d2b None None] MFA Rules not processed for user `fce8fdffbc354641b50a0d46ea6be26f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:53.370306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-724a050a-ec22-4c7f-92d0-58927ca72d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:53.543711 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2117/4233] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:54.217805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:54.243723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad101c-00af-4228-bd63-022c93ffc3c8 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:54.266854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2896c79-35c0-4aeb-be94-222450e97c51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:54.277769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2896c79-35c0-4aeb-be94-222450e97c51 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:54.277769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2896c79-35c0-4aeb-be94-222450e97c51 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:54.279514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2896c79-35c0-4aeb-be94-222450e97c51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:54.289875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f2896c79-35c0-4aeb-be94-222450e97c51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:54.331026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:54.456441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2896c79-35c0-4aeb-be94-222450e97c51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:54.463544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2896c79-35c0-4aeb-be94-222450e97c51 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:54.522752 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2117/4234] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:54.682837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a050a-ec22-4c7f-92d0-58927ca72d2b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:54.710456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a050a-ec22-4c7f-92d0-58927ca72d2b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:54.762683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23309c09-0c3e-4a75-a546-0e8cbf23bf5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:54.763916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309c09-0c3e-4a75-a546-0e8cbf23bf5d service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:54.768939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309c09-0c3e-4a75-a546-0e8cbf23bf5d service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:54.771563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309c09-0c3e-4a75-a546-0e8cbf23bf5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:54.783278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-23309c09-0c3e-4a75-a546-0e8cbf23bf5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:54.807799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23309c09-0c3e-4a75-a546-0e8cbf23bf5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:54.812086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23309c09-0c3e-4a75-a546-0e8cbf23bf5d service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:54.836789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2118/4235] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:58.308797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63536885-5923-407a-997c-e28a4fa4d31f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:58.309981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63536885-5923-407a-997c-e28a4fa4d31f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:58.310442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63536885-5923-407a-997c-e28a4fa4d31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:58.372514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-63536885-5923-407a-997c-e28a4fa4d31f None None] MFA Rules not processed for user `24c7a82126124ebb8baf5c656ff9849d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:58.374114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-63536885-5923-407a-997c-e28a4fa4d31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:58.382405 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2118/4236] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:01.496195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.497931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.498741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.578847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] MFA Rules not processed for user `c544e8e6f7a741c985c92ed7437397cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:01.582146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:01.762022 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2119/4237] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:02.338115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63536885-5923-407a-997c-e28a4fa4d31f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.361748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63536885-5923-407a-997c-e28a4fa4d31f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.378149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65eefa-be38-45f1-a53b-450249eda1c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.378963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65eefa-be38-45f1-a53b-450249eda1c6 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.379325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65eefa-be38-45f1-a53b-450249eda1c6 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.379733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65eefa-be38-45f1-a53b-450249eda1c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.402230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65eefa-be38-45f1-a53b-450249eda1c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.407925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65eefa-be38-45f1-a53b-450249eda1c6 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.431034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2119/4238] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:02.811626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.813754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.845175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.900375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dab76-8af2-4c9f-b5eb-b7cfba78092e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.907671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.908661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.909266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.909752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.911895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:03.038531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.041427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.081835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2120/4239] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:08.703265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524f89-a134-4f0b-bfa8-8ffad42c2238 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.704123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524f89-a134-4f0b-bfa8-8ffad42c2238 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.704388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524f89-a134-4f0b-bfa8-8ffad42c2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.729283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-82524f89-a134-4f0b-bfa8-8ffad42c2238 None None] MFA Rules not processed for user `c544e8e6f7a741c985c92ed7437397cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:08.730758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-82524f89-a134-4f0b-bfa8-8ffad42c2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:08.741775 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2120/4240] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:10.069054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.070676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:10.084408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769de09-61a1-4326-8304-7b6fa0a4590c service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.089974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.090669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.091014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.091356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] PATH_INFO: `/v3/users/cd454c25ddef46d993ff15440a64a438` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.092567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd454c25ddef46d993ff15440a64a438)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.094615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.151188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd454c25ddef46d993ff15440a64a438'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.151853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd454c25ddef46d993ff15440a64a438'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.160172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd454c25ddef46d993ff15440a64a438'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.161070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd454c25ddef46d993ff15440a64a438'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.175178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4d40892a-4c42-4d87-8ac0-96e07272ac79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd454c25ddef46d993ff15440a64a438'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.190985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2121/4241] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:10 2022] DELETE /identity/v3/users/cd454c25ddef46d993ff15440a64a438 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:10.406785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82524f89-a134-4f0b-bfa8-8ffad42c2238 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.429247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82524f89-a134-4f0b-bfa8-8ffad42c2238 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.506115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.506775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.507232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.507509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] PATH_INFO: `/v3/projects/130f997fde494ab887281d866d6aec31` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.508702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] RBAC: Authorizing `identity:delete_project(project_id=130f997fde494ab887281d866d6aec31)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.510752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.568716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130f997fde494ab887281d866d6aec31', 'request_id': 'req-7743787f-1673-4f6b-a67f-dd88579187db'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.579567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130f997fde494ab887281d866d6aec31', 'request_id': 'req-7743787f-1673-4f6b-a67f-dd88579187db'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.587147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130f997fde494ab887281d866d6aec31', 'request_id': 'req-7743787f-1673-4f6b-a67f-dd88579187db'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.600912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] The token cache is being invalidate because project 130f997fde494ab887281d866d6aec31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:10.601215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.602101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7743787f-1673-4f6b-a67f-dd88579187db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.603824 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2121/4242] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:10 2022] DELETE /identity/v3/projects/130f997fde494ab887281d866d6aec31 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:10.610855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad04177-472c-4abb-af75-cd4a9f6b68bc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.611455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad04177-472c-4abb-af75-cd4a9f6b68bc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.612435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad04177-472c-4abb-af75-cd4a9f6b68bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.634955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-aad04177-472c-4abb-af75-cd4a9f6b68bc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:10.636088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aad04177-472c-4abb-af75-cd4a9f6b68bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:10.677961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:10.721431 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2122/4243] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:10.723111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20b339-98da-430c-83f6-f6fa29289579 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.725640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20b339-98da-430c-83f6-f6fa29289579 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.726008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20b339-98da-430c-83f6-f6fa29289579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.750361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ff20b339-98da-430c-83f6-f6fa29289579 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:10.751722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ff20b339-98da-430c-83f6-f6fa29289579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:10.761512 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2122/4244] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:10.765440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad04177-472c-4abb-af75-cd4a9f6b68bc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.777225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad04177-472c-4abb-af75-cd4a9f6b68bc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.783201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.784024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.784376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.784720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.785493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:10.785858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.787749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.801198 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2123/4245] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:10.805566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20b339-98da-430c-83f6-f6fa29289579 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.815095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20b339-98da-430c-83f6-f6fa29289579 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.819865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.820564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.820846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.821116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.821785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.823742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.833568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:10.855881 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2123/4246] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:10 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:10.860320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.872727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb20e-009b-4f57-ac92-78e164b024f4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.935205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.935912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.936244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.936550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.937283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.939625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.963536 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2124/4247] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:10 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:10.967804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.978520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493c97e-7adc-4e9d-b0c5-4f59787861f1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.983260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d0bca-b590-4129-923b-61487834855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.983957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0bca-b590-4129-923b-61487834855c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.984303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0bca-b590-4129-923b-61487834855c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.984640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0bca-b590-4129-923b-61487834855c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.985393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d0bca-b590-4129-923b-61487834855c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.987144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d0bca-b590-4129-923b-61487834855c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.997369 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2124/4248] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:11.001666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.013445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f6c45-5823-4f8e-bedb-3e38868ff327 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.018971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30f688-a2c6-4722-ac68-181c8b856ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.019735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f688-a2c6-4722-ac68-181c8b856ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.020076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f688-a2c6-4722-ac68-181c8b856ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.020415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30f688-a2c6-4722-ac68-181c8b856ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.021115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30f688-a2c6-4722-ac68-181c8b856ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.022856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30f688-a2c6-4722-ac68-181c8b856ed6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:11.035454 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2125/4249] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:11.040257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d0bca-b590-4129-923b-61487834855c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.050676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d0bca-b590-4129-923b-61487834855c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.055589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.056323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.056664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.057001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] PATH_INFO: `/v3/projects/d527ac7a0b864094a5c74dea23ec445b/users/599c7072545f4280988f6bc50e3ccc5f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.072117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d527ac7a0b864094a5c74dea23ec445b, user_id=599c7072545f4280988f6bc50e3ccc5f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.074813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:11.089982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2125/4250] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:11 2022] PUT /identity/v3/projects/d527ac7a0b864094a5c74dea23ec445b/users/599c7072545f4280988f6bc50e3ccc5f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:11.095930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.096439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.097065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.119902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:11.121635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.229056 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2126/4251] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:11.237587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.239825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.383237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b52bea-2ae5-4833-bdb6-e3535ba841b2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.395519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179996b-1094-476f-8815-c10812b72db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.396765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179996b-1094-476f-8815-c10812b72db8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.396765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179996b-1094-476f-8815-c10812b72db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.397173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179996b-1094-476f-8815-c10812b72db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.418586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179996b-1094-476f-8815-c10812b72db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.420140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179996b-1094-476f-8815-c10812b72db8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:11.443128 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2126/4252] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:17.588462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:17.590257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:17.701979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:17.764580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46098391-2529-40ec-a2ae-2129238bbf81 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:17.772389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735c3e9-a631-4ad5-ba3c-3107f071ef69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:17.773499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735c3e9-a631-4ad5-ba3c-3107f071ef69 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:17.774046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735c3e9-a631-4ad5-ba3c-3107f071ef69 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:17.774741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735c3e9-a631-4ad5-ba3c-3107f071ef69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:17.779569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b735c3e9-a631-4ad5-ba3c-3107f071ef69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:17.813251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735c3e9-a631-4ad5-ba3c-3107f071ef69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:17.815102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735c3e9-a631-4ad5-ba3c-3107f071ef69 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:17.844090 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2127/4253] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:18.130203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:18.130682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:18.130774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:18.165858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] MFA Rules not processed for user `599c7072545f4280988f6bc50e3ccc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:18.167084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:18.260148 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2127/4254] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:18.270726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b321-6d66-4fa8-8792-1863204c9399 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:18.271113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b321-6d66-4fa8-8792-1863204c9399 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:18.271463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2b321-6d66-4fa8-8792-1863204c9399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:18.302873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-82d2b321-6d66-4fa8-8792-1863204c9399 None None] MFA Rules not processed for user `599c7072545f4280988f6bc50e3ccc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:18.304146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-82d2b321-6d66-4fa8-8792-1863204c9399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:18.317645 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2128/4255] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:18.544224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:18.547260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:18.571556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f1dbd-96ab-4b71-aee7-484ffb3c3729 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:18.579418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:18.580418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:18.580979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:18.581425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:18.600206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:18.602316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:18.628701 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2128/4256] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:27.759919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd5d8d-c453-4091-803c-e09e92ab78c9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.760446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd5d8d-c453-4091-803c-e09e92ab78c9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.760669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd5d8d-c453-4091-803c-e09e92ab78c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.788135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-13cd5d8d-c453-4091-803c-e09e92ab78c9 None None] MFA Rules not processed for user `30b19896bc4e4d14bfec46dd00519150`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:27.789680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13cd5d8d-c453-4091-803c-e09e92ab78c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.867834 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2129/4257] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:28.609904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:29.395614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:29.397480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:29.418430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e18551-b822-4429-b93c-76606fc87a0d service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:29.424165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:29.424987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:29.425318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:29.426157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] PATH_INFO: `/v3/users/1933f2aed04a45a28bb2617337faa719` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:29.427779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1933f2aed04a45a28bb2617337faa719)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:29.430049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:29.510753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1933f2aed04a45a28bb2617337faa719'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:29.526267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1933f2aed04a45a28bb2617337faa719'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:29.537411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1933f2aed04a45a28bb2617337faa719'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:29.538522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1933f2aed04a45a28bb2617337faa719'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:29.539591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1933f2aed04a45a28bb2617337faa719'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:29.569031 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2129/4258] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:29 2022] DELETE /identity/v3/users/1933f2aed04a45a28bb2617337faa719 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:29.970204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd5d8d-c453-4091-803c-e09e92ab78c9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:29.990361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd5d8d-c453-4091-803c-e09e92ab78c9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:30.099375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:30.100022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:30.100300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:30.100574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] PATH_INFO: `/v3/projects/598139a6b6764e80822b82231f572770` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:30.101775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=598139a6b6764e80822b82231f572770)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:30.104411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:30.158963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '598139a6b6764e80822b82231f572770', 'request_id': 'req-1967837a-000e-46c1-bd88-39640b291e0b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.167241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '598139a6b6764e80822b82231f572770', 'request_id': 'req-1967837a-000e-46c1-bd88-39640b291e0b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.179648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '598139a6b6764e80822b82231f572770', 'request_id': 'req-1967837a-000e-46c1-bd88-39640b291e0b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.189346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:30.194491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] The token cache is being invalidate because project 598139a6b6764e80822b82231f572770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:30.194911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.195683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.200981 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2130/4259] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:29 2022] DELETE /identity/v3/projects/598139a6b6764e80822b82231f572770 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:30.203062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:30.204668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:30.219007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9530-213c-4626-9e24-b109e1d844c3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:30.340419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:30.340966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:30.341247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:30.341571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] PATH_INFO: `/v3/users/c71dfa6b81844d6b84f1c3eaade8fc86` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:30.342744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71dfa6b81844d6b84f1c3eaade8fc86)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:30.344693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:30.409129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71dfa6b81844d6b84f1c3eaade8fc86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.427919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71dfa6b81844d6b84f1c3eaade8fc86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.440313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71dfa6b81844d6b84f1c3eaade8fc86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.441272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71dfa6b81844d6b84f1c3eaade8fc86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.442150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-684222ff-77a7-4c44-abc4-1210cb477a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71dfa6b81844d6b84f1c3eaade8fc86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.446051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:30.489490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2130/4260] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:30 2022] DELETE /identity/v3/users/c71dfa6b81844d6b84f1c3eaade8fc86 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:30.717712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:30.719205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:30.732625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967837a-000e-46c1-bd88-39640b291e0b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:30.858617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:30.859274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:30.859576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:30.859851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] PATH_INFO: `/v3/projects/801cfabc02454546b8a1488856e88798` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:30.862322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] RBAC: Authorizing `identity:delete_project(project_id=801cfabc02454546b8a1488856e88798)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:30.870263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:30.959633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801cfabc02454546b8a1488856e88798', 'request_id': 'req-3262c753-1fad-4889-bbf4-f983e513babe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:30.981896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801cfabc02454546b8a1488856e88798', 'request_id': 'req-3262c753-1fad-4889-bbf4-f983e513babe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:31.009442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801cfabc02454546b8a1488856e88798', 'request_id': 'req-3262c753-1fad-4889-bbf4-f983e513babe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:31.024538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] The token cache is being invalidate because project 801cfabc02454546b8a1488856e88798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:31.024872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:31.025598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3262c753-1fad-4889-bbf4-f983e513babe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:31.027545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2131/4261] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:30 2022] DELETE /identity/v3/projects/801cfabc02454546b8a1488856e88798 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:31.032338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac7a49-4ea4-4276-a251-f33283cde5ed None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.032707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac7a49-4ea4-4276-a251-f33283cde5ed None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.033160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac7a49-4ea4-4276-a251-f33283cde5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.066546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-51ac7a49-4ea4-4276-a251-f33283cde5ed None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:31.067505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-51ac7a49-4ea4-4276-a251-f33283cde5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:31.144185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2131/4262] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:31.152794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24b7c-ae53-4cd4-8e82-948e96acebb7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.152794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24b7c-ae53-4cd4-8e82-948e96acebb7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.153131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24b7c-ae53-4cd4-8e82-948e96acebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.181818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6ba24b7c-ae53-4cd4-8e82-948e96acebb7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:31.183070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6ba24b7c-ae53-4cd4-8e82-948e96acebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:31.196646 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2132/4263] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:31.197430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac7a49-4ea4-4276-a251-f33283cde5ed None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:31.217560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac7a49-4ea4-4276-a251-f33283cde5ed None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:31.222828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:31.223641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.223917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.224185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.224836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:31.225135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:31.226967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:31.241504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2132/4264] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:31.245678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba24b7c-ae53-4cd4-8e82-948e96acebb7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:31.256041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba24b7c-ae53-4cd4-8e82-948e96acebb7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:31.263015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:31.264001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.264479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.264942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.265807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:31.267833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:31.318059 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2133/4265] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:31 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:31.322012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:31.334179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867dbfc-ec62-4590-8686-a3233e657572 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:31.412645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:31.413273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.413596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.413862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.414514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:31.416490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:31.452510 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2133/4266] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:31 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:31.457095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:31.467628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298d98-cd19-4862-be38-2add517bf002 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:31.472957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:31.473628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.473924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.474406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.475280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:31.476989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:31.488173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2134/4267] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:31.492494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:31.517527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df18d-3bd8-4535-852c-a04eead687ff admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:31.529857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd05692-c662-45c8-87d6-733d454d7a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:31.530592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd05692-c662-45c8-87d6-733d454d7a36 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.531172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd05692-c662-45c8-87d6-733d454d7a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.531381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd05692-c662-45c8-87d6-733d454d7a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.532060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd05692-c662-45c8-87d6-733d454d7a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:31.534148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd05692-c662-45c8-87d6-733d454d7a36 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:31.550558 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2134/4268] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:31.551389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:31.574274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c035353-19ea-4c6c-a8cb-0658ddc24e7b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:31.585378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:31.586202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.589226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.589657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] PATH_INFO: `/v3/projects/5816f5fcc813404e93f6fb6c810e38ae/users/2e26d30550f34e1aaf0628c3824040a7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.608777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5816f5fcc813404e93f6fb6c810e38ae, user_id=2e26d30550f34e1aaf0628c3824040a7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:31.612817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:31.657042 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2135/4269] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:31 2022] PUT /identity/v3/projects/5816f5fcc813404e93f6fb6c810e38ae/users/2e26d30550f34e1aaf0628c3824040a7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:31.674392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b979458-72ea-4239-a23e-0a1e82e97c5c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.675020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b979458-72ea-4239-a23e-0a1e82e97c5c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.675491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b979458-72ea-4239-a23e-0a1e82e97c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.721735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9b979458-72ea-4239-a23e-0a1e82e97c5c None None] MFA Rules not processed for user `2e26d30550f34e1aaf0628c3824040a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:31.722984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9b979458-72ea-4239-a23e-0a1e82e97c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:31.803877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:31.856573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2135/4270] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:31.861497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:31.901804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:31.964952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec27c10-b871-478a-9498-af37a72b1d9a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:31.969936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:31.970558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:31.971056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:31.971327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:31.972021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:31.974024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.005122 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2136/4271] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:31 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:32.009452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b979458-72ea-4239-a23e-0a1e82e97c5c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.024028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b979458-72ea-4239-a23e-0a1e82e97c5c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.101920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.103077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.103434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.103762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.105133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.108684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.139744 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2136/4272] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:32 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:32.144077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.155782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a4acd-0bc5-48fb-8cab-e1edf1b9b5d2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.160808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.161430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.161738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.162193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.163028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.164843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.177405 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2137/4273] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:32.181633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.193061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd7afa-d125-422d-b377-84ff27559f68 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.198901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ad789-8c09-462d-9481-d32667ffc40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.199547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ad789-8c09-462d-9481-d32667ffc40d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.199822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ad789-8c09-462d-9481-d32667ffc40d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.200090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ad789-8c09-462d-9481-d32667ffc40d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.200918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ad789-8c09-462d-9481-d32667ffc40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.203272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ad789-8c09-462d-9481-d32667ffc40d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.219968 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2137/4274] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:32.231452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.257251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd4b2c-4713-40ba-b8e6-f3411262607c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.265801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.266488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.266760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.271947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] PATH_INFO: `/v3/projects/d7ce234c3fb34227b11dce573056c78e/users/156d9b5e2ae34e07b1054aed56680e5d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.292808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ce234c3fb34227b11dce573056c78e, user_id=156d9b5e2ae34e07b1054aed56680e5d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.295905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.323549 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2138/4275] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:32 2022] PUT /identity/v3/projects/d7ce234c3fb34227b11dce573056c78e/users/156d9b5e2ae34e07b1054aed56680e5d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:32.338057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9a36c-a0c5-4eed-b575-c18c90487f1e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.338286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9a36c-a0c5-4eed-b575-c18c90487f1e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.338549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9a36c-a0c5-4eed-b575-c18c90487f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.379068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-66f9a36c-a0c5-4eed-b575-c18c90487f1e None None] MFA Rules not processed for user `156d9b5e2ae34e07b1054aed56680e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:32.380851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-66f9a36c-a0c5-4eed-b575-c18c90487f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:32.473481 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2138/4276] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:32.475307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.623197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eae89-7f33-4a45-9d82-16805ac59f4b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.634648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.635283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.635799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.636133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.636878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.638953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.699844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9a36c-a0c5-4eed-b575-c18c90487f1e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.702357 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2139/4277] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:32 2022] POST /identity/v3/projects => generated 381 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:32.713915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9a36c-a0c5-4eed-b575-c18c90487f1e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.873953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:32.892683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.894696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.894990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.895285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.897175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.900458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.933515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2139/4278] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:32 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:32.936848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.951002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34de3d-32bd-4437-b800-62dac0c2fbb5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.956865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.957512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.958710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.958710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.958710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.960289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.979229 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2140/4279] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:32.983343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.997445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1587703-ad9e-463a-974f-90fc6927b155 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:33.012538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:33.013222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:33.013634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:33.013898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] PATH_INFO: `/v3/projects/45988498b30f439fa6addefa8a66d65e/users/14cb6957da244aacac9548584d86cf9d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:33.047119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45988498b30f439fa6addefa8a66d65e, user_id=14cb6957da244aacac9548584d86cf9d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:33.050790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:33.067984 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2140/4280] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:32 2022] PUT /identity/v3/projects/45988498b30f439fa6addefa8a66d65e/users/14cb6957da244aacac9548584d86cf9d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:33.072090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:33.086399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a262f-ec4a-4ba8-bc37-83ccf9a36aa0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:33.149264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:33.174136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb855fd-8b73-43d9-8e13-1fe8d3029738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:33.177092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb855fd-8b73-43d9-8e13-1fe8d3029738 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:33.177425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb855fd-8b73-43d9-8e13-1fe8d3029738 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:33.179504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb855fd-8b73-43d9-8e13-1fe8d3029738 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:33.180194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb855fd-8b73-43d9-8e13-1fe8d3029738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:33.184665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb855fd-8b73-43d9-8e13-1fe8d3029738 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:33.208261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:33.208650 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2141/4281] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:33.222905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079ab06-0b94-4123-8e82-01e0586c53e2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:33.230728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:33.232680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:33.232975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:33.233250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] PATH_INFO: `/v3/projects/45988498b30f439fa6addefa8a66d65e/users/14cb6957da244aacac9548584d86cf9d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:33.236168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45988498b30f439fa6addefa8a66d65e, user_id=14cb6957da244aacac9548584d86cf9d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:33.240303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:33.266178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2141/4282] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:33 2022] PUT /identity/v3/projects/45988498b30f439fa6addefa8a66d65e/users/14cb6957da244aacac9548584d86cf9d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:33.269648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:33.270287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:33.270747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:33.325604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] MFA Rules not processed for user `14cb6957da244aacac9548584d86cf9d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:33.327786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:33.465249 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2142/4283] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:33.475257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:33.479115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:33.595883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29020eec-4e8c-453f-813f-b2dd0fe2dc44 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:33.624959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:33.627346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:33.632369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:33.632900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:33.758390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:33.758390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:33.759364 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2142/4284] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:34.854541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:34.856104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:34.938751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5f404-3df2-4e54-82e9-3ba270ea8ce5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:34.946953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537a9ae-05d0-4f52-928e-650259879595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:34.947646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537a9ae-05d0-4f52-928e-650259879595 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:34.947948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537a9ae-05d0-4f52-928e-650259879595 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:34.948219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537a9ae-05d0-4f52-928e-650259879595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:34.951257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7537a9ae-05d0-4f52-928e-650259879595 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:35.014739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537a9ae-05d0-4f52-928e-650259879595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:35.016544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537a9ae-05d0-4f52-928e-650259879595 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:35.040983 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2143/4285] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:35.062042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:35.062042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:35.134893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5925f87-501d-498a-95b2-cd3b528f492f service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:35.139595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:35.140316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:35.140592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:35.140861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] PATH_INFO: `/v3/users/6b27ba12dd5a46b3b27479c3a7c241e5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:35.142116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b27ba12dd5a46b3b27479c3a7c241e5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:35.144122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:35.146796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:35.208716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b27ba12dd5a46b3b27479c3a7c241e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.224397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b27ba12dd5a46b3b27479c3a7c241e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.237840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b27ba12dd5a46b3b27479c3a7c241e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.238603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b27ba12dd5a46b3b27479c3a7c241e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.239799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b27ba12dd5a46b3b27479c3a7c241e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.257666 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2143/4286] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:35 2022] DELETE /identity/v3/users/6b27ba12dd5a46b3b27479c3a7c241e5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:35.259382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:35.259667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:35.259938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:35.277739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:35.280985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:35.283549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:35.362878 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2144/4287] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:35.372941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:35.374780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:35.465788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a2beb-8053-4bb1-95d1-320f9ec4c35e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:35.469453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:35.470100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:35.470420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:35.470743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:35.480965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:35.482652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:35.484995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:35.486587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:35.494285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2144/4288] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:35.561926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9c1ad-e01a-4940-996f-02ddaee6f46b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:35.577011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b1eb8-8239-4e30-9c0b-2788f9565b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:35.577948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b1eb8-8239-4e30-9c0b-2788f9565b7f service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:35.578329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b1eb8-8239-4e30-9c0b-2788f9565b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:35.578737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b1eb8-8239-4e30-9c0b-2788f9565b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:35.584901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-845b1eb8-8239-4e30-9c0b-2788f9565b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:35.661908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b1eb8-8239-4e30-9c0b-2788f9565b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:35.667866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b1eb8-8239-4e30-9c0b-2788f9565b7f service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:35.679701 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2145/4289] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:35.863656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:35.874352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:35.892614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3618a32-bba8-4b67-b0ab-9bfbbebe5929 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:35.903340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:35.904463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:35.905597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:35.906104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] PATH_INFO: `/v3/projects/4512f54506dd413a80df4b337a6c31c6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:35.907391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4512f54506dd413a80df4b337a6c31c6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:35.909586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:35.961980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4512f54506dd413a80df4b337a6c31c6', 'request_id': 'req-eb979524-731c-4819-b185-73f1678c66f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.972241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4512f54506dd413a80df4b337a6c31c6', 'request_id': 'req-eb979524-731c-4819-b185-73f1678c66f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.980714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4512f54506dd413a80df4b337a6c31c6', 'request_id': 'req-eb979524-731c-4819-b185-73f1678c66f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.993815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] The token cache is being invalidate because project 4512f54506dd413a80df4b337a6c31c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:35.994274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.995062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb979524-731c-4819-b185-73f1678c66f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:35.996832 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2145/4290] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:35 2022] DELETE /identity/v3/projects/4512f54506dd413a80df4b337a6c31c6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:36.002147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cb324-34b9-4436-a911-bd1443ecc039 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.002696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cb324-34b9-4436-a911-bd1443ecc039 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.002978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cb324-34b9-4436-a911-bd1443ecc039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.026528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-613cb324-34b9-4436-a911-bd1443ecc039 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:36.027940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-613cb324-34b9-4436-a911-bd1443ecc039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:36.123278 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2146/4291] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:36.132822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30b9f0-321b-4192-9698-7ea92a994252 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.133394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30b9f0-321b-4192-9698-7ea92a994252 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.133394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30b9f0-321b-4192-9698-7ea92a994252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.149536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:36.156470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4a30b9f0-321b-4192-9698-7ea92a994252 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:36.158475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4a30b9f0-321b-4192-9698-7ea92a994252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:36.172145 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2146/4292] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:36.172457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cb324-34b9-4436-a911-bd1443ecc039 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.177579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:36.183831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cb324-34b9-4436-a911-bd1443ecc039 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.190957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.191786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.191951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.192109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.192757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:36.193065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.194843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.207688 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2147/4293] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:06:36.212709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30b9f0-321b-4192-9698-7ea92a994252 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.224756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30b9f0-321b-4192-9698-7ea92a994252 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.231856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.232823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.233118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.233502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.234511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.236910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.268962 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2147/4294] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:36 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:36.273419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.285163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccb685-16a6-4c1f-8926-d106f2deb3df admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.351580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.352268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.352628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.352962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.353779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.355895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.382816 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2148/4295] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:36 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:36.387093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.405793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df6de5-3299-401b-aa38-7756d4d4324e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.405793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.405793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.405793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.406242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.406242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.406242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.418737 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2148/4296] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:36.422539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.434206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bec14-395a-4d08-85e0-7af5054d8bd8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.442738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.442923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.443833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.444138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.444406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.444833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:36.445055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.447102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.458786 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2149/4297] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:36.462758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.473059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621c187-70bf-45c0-bec2-8201c41b553a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.477825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bd938-c73a-4072-9218-ab95524b76a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.478489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bd938-c73a-4072-9218-ab95524b76a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.478762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bd938-c73a-4072-9218-ab95524b76a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.479033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bd938-c73a-4072-9218-ab95524b76a9 admin admin] PATH_INFO: `/v3/projects/04743e084b9b4335adfd426861a62cb6/users/1c4115c0f0b7493cb4b5f4fb21bb3c9e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.505002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bd938-c73a-4072-9218-ab95524b76a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04743e084b9b4335adfd426861a62cb6, user_id=1c4115c0f0b7493cb4b5f4fb21bb3c9e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.507840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bd938-c73a-4072-9218-ab95524b76a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.510081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6a9b2-7ae6-4e9e-8faf-07ed358e7bd6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.514134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.514964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.515309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.515585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.517573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:36.521698 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2150/4298] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:36 2022] PUT /identity/v3/projects/04743e084b9b4335adfd426861a62cb6/users/1c4115c0f0b7493cb4b5f4fb21bb3c9e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:36.532032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff2feb-bc96-4951-aeb7-5ce90547e096 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.532387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff2feb-bc96-4951-aeb7-5ce90547e096 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.532659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff2feb-bc96-4951-aeb7-5ce90547e096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.559698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a7ff2feb-bc96-4951-aeb7-5ce90547e096 None None] MFA Rules not processed for user `1c4115c0f0b7493cb4b5f4fb21bb3c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:36.560819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a7ff2feb-bc96-4951-aeb7-5ce90547e096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:36.583410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.584858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.594384 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2149/4299] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:36.641968 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2151/4300] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:36.657137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.668714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f380c-e639-4021-a031-365d243ef6ae service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:36.746740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:36.747384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.747663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.747930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] PATH_INFO: `/v3/users/1c4115c0f0b7493cb4b5f4fb21bb3c9e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.749072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4115c0f0b7493cb4b5f4fb21bb3c9e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:36.751038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:36.772972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:36.804883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4115c0f0b7493cb4b5f4fb21bb3c9e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:36.818669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4115c0f0b7493cb4b5f4fb21bb3c9e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:36.825851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4115c0f0b7493cb4b5f4fb21bb3c9e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:36.826804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4115c0f0b7493cb4b5f4fb21bb3c9e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:36.827435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4115c0f0b7493cb4b5f4fb21bb3c9e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:36.840722 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2150/4301] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:36 2022] DELETE /identity/v3/users/1c4115c0f0b7493cb4b5f4fb21bb3c9e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:36.845897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:36.846183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:36.846460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:36.868273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:36.869240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:36.904152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:36.939101 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2152/4302] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:36.949353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:36.950905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:37.020278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f523a7e-6572-4d52-a6e2-4c4bf17a83d8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.024845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287056d5-316f-4458-8087-44b2627fda1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.025606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287056d5-316f-4458-8087-44b2627fda1b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.026831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287056d5-316f-4458-8087-44b2627fda1b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.026831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287056d5-316f-4458-8087-44b2627fda1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.043215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287056d5-316f-4458-8087-44b2627fda1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.045312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287056d5-316f-4458-8087-44b2627fda1b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:37.073037 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2151/4303] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:37.371949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:37.373501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:37.395022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c590e2-0c21-4790-972a-9e0b0aae914b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.409407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.410121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.410395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.410658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] PATH_INFO: `/v3/projects/04743e084b9b4335adfd426861a62cb6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.411821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04743e084b9b4335adfd426861a62cb6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.414031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:37.464345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04743e084b9b4335adfd426861a62cb6', 'request_id': 'req-294826ec-8d21-435d-bc55-a4ad13e8f95d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:37.471238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04743e084b9b4335adfd426861a62cb6', 'request_id': 'req-294826ec-8d21-435d-bc55-a4ad13e8f95d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:37.480551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04743e084b9b4335adfd426861a62cb6', 'request_id': 'req-294826ec-8d21-435d-bc55-a4ad13e8f95d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:37.492034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] The token cache is being invalidate because project 04743e084b9b4335adfd426861a62cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:37.492321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:37.493013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-294826ec-8d21-435d-bc55-a4ad13e8f95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:37.494626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2153/4304] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:37 2022] DELETE /identity/v3/projects/04743e084b9b4335adfd426861a62cb6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:37.500859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18430c11-78fb-468b-97cb-3c4993ee82a4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.501402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18430c11-78fb-468b-97cb-3c4993ee82a4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.501402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18430c11-78fb-468b-97cb-3c4993ee82a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.525926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-18430c11-78fb-468b-97cb-3c4993ee82a4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:37.527230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-18430c11-78fb-468b-97cb-3c4993ee82a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:37.591990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2152/4305] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:37.600282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c6989-565f-4101-aba3-54d8949b1315 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.600696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c6989-565f-4101-aba3-54d8949b1315 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.601056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c6989-565f-4101-aba3-54d8949b1315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.622130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5a5c6989-565f-4101-aba3-54d8949b1315 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:37.623494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5a5c6989-565f-4101-aba3-54d8949b1315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:37.632534 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2154/4306] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:37.637132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18430c11-78fb-468b-97cb-3c4993ee82a4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:37.647368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18430c11-78fb-468b-97cb-3c4993ee82a4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.652108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.652749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.652993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.653263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.653971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:37.654269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.655901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:37.671940 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2153/4307] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:37.673200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c6989-565f-4101-aba3-54d8949b1315 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:37.684129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c6989-565f-4101-aba3-54d8949b1315 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.689529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.690167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.690453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.690727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.691386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.698351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:37.732411 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2155/4308] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:37 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:37.736750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:37.747605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc12bf-1789-4467-8208-aac7f1c770dc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.776262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:37.811530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.812156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.812436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.812749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.813357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.815403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:37.854786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2154/4309] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:37 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:37.855587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:37.868903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd244c35-8b61-4174-8d8a-9ed7b0b42e71 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.874969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.875755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.876048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.876299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.877368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.878657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:37.890039 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2156/4310] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:37.895426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:37.907546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd1022-78d0-4d74-9f13-8ea0858be4e1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.912556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b5aa1-2ab3-40b5-8816-c1c1687f04b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.913172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b5aa1-2ab3-40b5-8816-c1c1687f04b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.913938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b5aa1-2ab3-40b5-8816-c1c1687f04b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.914205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b5aa1-2ab3-40b5-8816-c1c1687f04b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.914982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b5aa1-2ab3-40b5-8816-c1c1687f04b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.916679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b5aa1-2ab3-40b5-8816-c1c1687f04b9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:37.931847 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2155/4311] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:37.934220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:37.948304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc7df8-9fa9-4a9b-a323-f4a270541ede admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:37.953100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:37.954426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:37.954710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:37.955039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] PATH_INFO: `/v3/projects/e26e0c8443db417b9df0fd3b076b75aa/users/3aef0cd15055441a84ecf30f52d56d02/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:37.972843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26e0c8443db417b9df0fd3b076b75aa, user_id=3aef0cd15055441a84ecf30f52d56d02, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:37.976231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.002822 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2157/4312] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:37 2022] PUT /identity/v3/projects/e26e0c8443db417b9df0fd3b076b75aa/users/3aef0cd15055441a84ecf30f52d56d02/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:38.006823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e5e1e-db87-419a-9e96-51f8cac1198b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.007281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e5e1e-db87-419a-9e96-51f8cac1198b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.007761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e5e1e-db87-419a-9e96-51f8cac1198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.040150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6a8e5e1e-db87-419a-9e96-51f8cac1198b None None] MFA Rules not processed for user `3aef0cd15055441a84ecf30f52d56d02`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:38.041449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6a8e5e1e-db87-419a-9e96-51f8cac1198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:38.135229 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2156/4313] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:38.136697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.214283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:38.230905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bdbcc-4632-4f13-ba3b-87c81b01a7c2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.238053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.238794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.239148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.239488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.240213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.242254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.278480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2158/4314] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:38 2022] POST /identity/v3/projects => generated 391 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:38.282382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e5e1e-db87-419a-9e96-51f8cac1198b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.293281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e5e1e-db87-419a-9e96-51f8cac1198b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.363063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.364393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.364393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.364647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.365341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.367447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.397295 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2157/4315] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:38 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:38.401655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.414721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b04c0-9984-4585-aed2-b6592e9381ef admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.420056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.420747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.421234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.421816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.422566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.424688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.437816 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2159/4316] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:38.442181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.453389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e59e70-7e55-4251-acfc-786e54e642d2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.463173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.463996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.464340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.464619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] PATH_INFO: `/v3/projects/117de53009b149aeae9148901cd3206d/users/d97eb464e3314c46bbba5fb2c3ec2eb4/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.480164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117de53009b149aeae9148901cd3206d, user_id=d97eb464e3314c46bbba5fb2c3ec2eb4, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.482876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.492520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:38.497672 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2158/4317] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:38 2022] PUT /identity/v3/projects/117de53009b149aeae9148901cd3206d/users/d97eb464e3314c46bbba5fb2c3ec2eb4/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:38.501929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.513790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b89f-c919-41e1-a09f-e5af44b23747 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.580952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfba65d-ba0d-41ff-afff-27c5e0cd3cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.581617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfba65d-ba0d-41ff-afff-27c5e0cd3cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.581890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfba65d-ba0d-41ff-afff-27c5e0cd3cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.582149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfba65d-ba0d-41ff-afff-27c5e0cd3cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.582782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfba65d-ba0d-41ff-afff-27c5e0cd3cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.584381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfba65d-ba0d-41ff-afff-27c5e0cd3cc8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.595684 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2160/4318] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:38.600186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.612108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4f7bb-2a0d-4fe3-acfc-0aab6e174261 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.617740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.618403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.618678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.618948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] PATH_INFO: `/v3/projects/117de53009b149aeae9148901cd3206d/users/d97eb464e3314c46bbba5fb2c3ec2eb4/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.620886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117de53009b149aeae9148901cd3206d, user_id=d97eb464e3314c46bbba5fb2c3ec2eb4, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.623599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.634950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2159/4319] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:38 2022] PUT /identity/v3/projects/117de53009b149aeae9148901cd3206d/users/d97eb464e3314c46bbba5fb2c3ec2eb4/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:38.645572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23615-866b-4c5b-9dc1-8a3cf39ec112 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.645858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23615-866b-4c5b-9dc1-8a3cf39ec112 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.646099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc23615-866b-4c5b-9dc1-8a3cf39ec112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.681170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1dc23615-866b-4c5b-9dc1-8a3cf39ec112 None None] MFA Rules not processed for user `d97eb464e3314c46bbba5fb2c3ec2eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:38.682413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1dc23615-866b-4c5b-9dc1-8a3cf39ec112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:38.754238 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2161/4320] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:38.759967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.771245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de4d99-04d4-4ba2-b71a-251d6fa9c824 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.776309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.776906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.777178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.777447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.778141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.779767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.798708 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2160/4321] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:06:38 2022] POST /identity/v3/domains => generated 277 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:38.803095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23615-866b-4c5b-9dc1-8a3cf39ec112 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.812684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc23615-866b-4c5b-9dc1-8a3cf39ec112 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.859166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:38.873098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.873803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.874085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.874360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a/config` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.875043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=72dbc84f440f46f5938f57245fb3388a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.876065 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:38.876065 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:06:38.876065 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:06:38.877048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.919887 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2162/4322] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:06:38 2022] PUT /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a/config => generated 138 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:38.924521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.935251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148edd3c-ca1c-46c2-af2b-bbaa1198a576 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.939795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.940542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.940823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.941127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.942364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=72dbc84f440f46f5938f57245fb3388a, group=identity)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.943475 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:38.943475 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:06:38.943475 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:06:38.944370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.951659 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2161/4323] 10.222.0.26 () {58 vars in 1275 bytes} [Sat Sep 17 21:06:38 2022] GET /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/identity => generated 45 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:38.956048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.966668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade689a-ba9b-44e4-b9a5-4c8b8dbf666b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:38.971806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:38.972515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:38.972794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:38.973068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/identity/driver` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:38.974251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=72dbc84f440f46f5938f57245fb3388a, group=identity, option=driver)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:38.975295 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:38.975295 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:06:38.975295 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:06:38.976257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:38.982576 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2163/4324] 10.222.0.26 () {58 vars in 1296 bytes} [Sat Sep 17 21:06:38 2022] GET /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/identity/driver => generated 31 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:38.987084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:38.997414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10ec8-5f55-4a2d-8be5-cd72dd776648 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.002223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.002863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.003140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.003410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/ldap` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.004912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=72dbc84f440f46f5938f57245fb3388a, group=ldap)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.006580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.011380 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2162/4325] 10.222.0.26 () {58 vars in 1263 bytes} [Sat Sep 17 21:06:38 2022] GET /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/ldap => generated 106 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:39.015516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.026165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebaf78f-aba0-41cf-8b55-f59adb863de0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.031172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.031836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.032115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.032392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/ldap/url` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.033553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=72dbc84f440f46f5938f57245fb3388a, group=ldap, option=url)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.035194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.040648 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2164/4326] 10.222.0.26 () {58 vars in 1275 bytes} [Sat Sep 17 21:06:39 2022] GET /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/ldap/url => generated 46 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:39.050689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.061322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b9eaa-7e06-4736-8c90-4a0bb2bfd245 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.065961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.066600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.066876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.067145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/ldap/user_tree_dn` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.068265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=72dbc84f440f46f5938f57245fb3388a, group=ldap, option=user_tree_dn)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.069906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.075821 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2163/4327] 10.222.0.26 () {58 vars in 1302 bytes} [Sat Sep 17 21:06:39 2022] GET /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a/config/ldap/user_tree_dn => generated 63 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:39.080143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.090403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d70da-44bb-4eff-afca-e4e6b9b62a87 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.095016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.095679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.095957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.096230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a/config` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.096808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=72dbc84f440f46f5938f57245fb3388a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.097839 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:39.097839 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:06:39.097839 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:06:39.098730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.108980 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2165/4328] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:06:39 2022] DELETE /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a/config => generated 0 bytes in 29 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:39.113354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.123877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962c5e9-6120-48b1-b54e-27b92c11a5fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.130239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.131319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.131319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.131601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.133332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=72dbc84f440f46f5938f57245fb3388a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.134803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.157863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '72dbc84f440f46f5938f57245fb3388a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.158664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '72dbc84f440f46f5938f57245fb3388a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.190622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '72dbc84f440f46f5938f57245fb3388a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.191359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '72dbc84f440f46f5938f57245fb3388a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.194054 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2164/4329] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:06:39 2022] PATCH /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a => generated 278 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:39.199924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.206863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:39.228913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09838d5-0b82-4a4d-b30c-daa446190415 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.298870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.299612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.299956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.300294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.314677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=72dbc84f440f46f5938f57245fb3388a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.316370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.353694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '72dbc84f440f46f5938f57245fb3388a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.357874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '72dbc84f440f46f5938f57245fb3388a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.418415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72dbc84f440f46f5938f57245fb3388a', 'request_id': 'req-8ae9a845-744b-44fe-ad49-0357ec5f998a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.422845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72dbc84f440f46f5938f57245fb3388a', 'request_id': 'req-8ae9a845-744b-44fe-ad49-0357ec5f998a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.433809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72dbc84f440f46f5938f57245fb3388a', 'request_id': 'req-8ae9a845-744b-44fe-ad49-0357ec5f998a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.435345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:39.449081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] The token cache is being invalidate because project 72dbc84f440f46f5938f57245fb3388a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:39.449487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.451754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.462756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '72dbc84f440f46f5938f57245fb3388a', 'request_id': 'req-8ae9a845-744b-44fe-ad49-0357ec5f998a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.466571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '72dbc84f440f46f5938f57245fb3388a', 'request_id': 'req-8ae9a845-744b-44fe-ad49-0357ec5f998a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.467286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '72dbc84f440f46f5938f57245fb3388a', 'request_id': 'req-8ae9a845-744b-44fe-ad49-0357ec5f998a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:39.473501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2166/4330] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:39 2022] DELETE /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:39.480577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.510539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb2874-5a51-4adb-bbf8-0b81f859d0b2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.601255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.602674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.603292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.603769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.604556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.606690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.640691 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2165/4331] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:06:39 2022] POST /identity/v3/domains => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:39.641913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.644178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:39.660833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9a845-744b-44fe-ad49-0357ec5f998a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.736227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.736947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.737229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.737604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.738209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.739958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.797392 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2167/4332] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:06:39 2022] PUT /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config => generated 138 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:39.798966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.809666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da49184-d53d-4704-819b-7c84d49899d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.814677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.815399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.815743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.816077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.816839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.817964 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:39.817964 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:06:39.817964 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:06:39.819026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.849147 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2166/4333] 10.222.0.26 () {60 vars in 1271 bytes} [Sat Sep 17 21:06:39 2022] PATCH /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config => generated 142 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:39.853329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.864071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e6222-ad3f-43b2-9054-1b145102dcb9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.869605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.870256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.870530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.870800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.871939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.873735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.881884 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2168/4334] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:06:39 2022] GET /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config => generated 142 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:39.885828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.905390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634079-5216-4b85-a1de-85c03808a066 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.911116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.911767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.911954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.912486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.913121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.914250 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:39.914250 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:06:39.914250 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:06:39.915182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:39.926447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2167/4335] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:06:39 2022] DELETE /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:39.931231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:39.942858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9450ec7-873b-4884-ae5c-0fd2b79648c7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:39.948723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:39.950178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:39.950636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:39.950914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:39.952073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:39.954081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Could not find any options in domain configuration for domain bb93e8fe272c46ffa83e47d8638ea2c7.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain bb93e8fe272c46ffa83e47d8638ea2c7. Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:06:40.003739 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain bb93e8fe272c46ffa83e47d8638ea2c7. Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:40.009010 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2169/4336] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:06:39 2022] GET /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config => generated 151 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:06:40.010090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.021298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c2e36-69ca-40ba-8ab5-0ad33e010137 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.026979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.027692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.028060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.028285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.028948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.031025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.037745 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2168/4337] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:06:40 2022] DELETE /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7/config => generated 0 bytes in 28 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:40.042047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.052997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd528af-d76f-4ad9-a2e9-b8e7994d398d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.068140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.068880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.069223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.069659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.071041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.072242 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:40.072242 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:06:40.072242 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:06:40.073497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.095357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.096131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.130424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.131064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.136526 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2170/4338] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:06:40 2022] PATCH /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7 => generated 276 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:40.137856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.139611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:40.152845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647679c-6632-4e84-ac1f-8d3ded27bafd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.157500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:INSERT=6,keystone:UPDATE=1,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:40.241253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.242340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.242856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.243192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.266523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=bb93e8fe272c46ffa83e47d8638ea2c7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.267628 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:40.267628 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:06:40.267628 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:06:40.268574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.292247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.312004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.354921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7', 'request_id': 'req-eca48cac-9e81-4a03-abc0-04e23eaf191f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.363650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7', 'request_id': 'req-eca48cac-9e81-4a03-abc0-04e23eaf191f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.369353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7', 'request_id': 'req-eca48cac-9e81-4a03-abc0-04e23eaf191f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.383057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] The token cache is being invalidate because project bb93e8fe272c46ffa83e47d8638ea2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:40.383354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.384065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.394351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7', 'request_id': 'req-eca48cac-9e81-4a03-abc0-04e23eaf191f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.398256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7', 'request_id': 'req-eca48cac-9e81-4a03-abc0-04e23eaf191f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.399019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'bb93e8fe272c46ffa83e47d8638ea2c7', 'request_id': 'req-eca48cac-9e81-4a03-abc0-04e23eaf191f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:40.404929 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2169/4339] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:40 2022] DELETE /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:40.410633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.421025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c487-ba96-47b3-b11b-83f5a08c36a2 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.460778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:40.479256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.479875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.480152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.480422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.481092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.482138 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:40.482138 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:06:40.482138 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:06:40.483084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.500431 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2171/4340] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:06:40 2022] POST /identity/v3/domains => generated 275 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:40.505209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.506699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:40.519064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca48cac-9e81-4a03-abc0-04e23eaf191f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.566993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:40.586258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.586897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.587175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.587444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.588120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.589146 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:40.589146 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:06:40.589146 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:06:40.590066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.624374 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2170/4341] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:06:40 2022] PUT /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config => generated 138 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:40.628766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.639784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f2e74-b4d9-4209-b313-74d830e5af3d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.645100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.645807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.646060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.646333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.647331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7, group=identity)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.648340 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:40.648340 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:06:40.648340 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:06:40.649227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.683252 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2172/4342] 10.222.0.26 () {60 vars in 1297 bytes} [Sat Sep 17 21:06:40 2022] PATCH /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity => generated 177 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:40.684725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.696099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eec658-e32b-41cf-85a6-774fc3f4711e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.701068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.701778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.702055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.702355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity/driver` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.703045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7, group=identity, option=driver)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.704659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.730996 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2171/4343] 10.222.0.26 () {60 vars in 1318 bytes} [Sat Sep 17 21:06:40 2022] PATCH /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity/driver => generated 177 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:40.735911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.748126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294e1b3-71ca-4a06-851a-6a60dece2c26 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.753622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.754658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.755135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.755769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity/driver` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.756380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7, group=identity, option=driver)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.759703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.777448 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2173/4344] 10.222.0.26 () {58 vars in 1299 bytes} [Sat Sep 17 21:06:40 2022] DELETE /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity/driver => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:40.782226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.796940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800f66f-e447-43c9-afc5-8931eab70dc3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.803285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.804142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.804591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.805061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity/driver` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.817849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7, group=identity, option=driver)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.823070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.832802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Could not find option driver in group identity in domain configuration for domain aa4c3fe383d84c049c03373250eca4e7.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain aa4c3fe383d84c049c03373250eca4e7. Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:06:40.833925 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain aa4c3fe383d84c049c03373250eca4e7. Sep 17 21:06:40.835099 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:06:40.836903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:40.846653 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2172/4345] 10.222.0.26 () {58 vars in 1296 bytes} [Sat Sep 17 21:06:40 2022] GET /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity/driver => generated 171 bytes in 65 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:06:40.850963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.867703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5c623-8982-4360-b971-65be444636fc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.878314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.879729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.880078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.880442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.881075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7, group=identity)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.883742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.905871 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2173/4346] 10.222.0.26 () {58 vars in 1278 bytes} [Sat Sep 17 21:06:40 2022] DELETE /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:40.910304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.925432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924fc51-c268-4da7-a0d6-5cd986d8ab47 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.929690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326807e5-c7ea-4378-b97b-9202e11b54c0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.930457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.931260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.931547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.931823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.933051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7, group=identity)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.934831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.934982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.936407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.936407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.936407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] PATH_INFO: `/v3/users/2e26d30550f34e1aaf0628c3824040a7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.937552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e26d30550f34e1aaf0628c3824040a7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.939550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Could not find group identity in domain configuration for domain aa4c3fe383d84c049c03373250eca4e7.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain aa4c3fe383d84c049c03373250eca4e7. Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:06:40.946505 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain aa4c3fe383d84c049c03373250eca4e7. Sep 17 21:06:40.947603 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:06:40.948272 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2174/4347] 10.222.0.26 () {58 vars in 1275 bytes} [Sat Sep 17 21:06:40 2022] GET /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config/identity => generated 154 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:06:40.952542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:40.964248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84af171-b416-4c86-9d75-30a285c2954a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:40.970339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:40.971022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:40.971516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:40.971798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:40.972954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=aa4c3fe383d84c049c03373250eca4e7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:40.974635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:40.985346 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2175/4348] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:06:40 2022] DELETE /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7/config => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:40.990463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.001929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2014a-221c-49ad-bf7f-be1b209ee541 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.005323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e26d30550f34e1aaf0628c3824040a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.006061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e26d30550f34e1aaf0628c3824040a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.007613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.008688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.008974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.009265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.010822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=aa4c3fe383d84c049c03373250eca4e7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.012672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.016851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e26d30550f34e1aaf0628c3824040a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.017710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e26d30550f34e1aaf0628c3824040a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.034960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2736d1fe-03bf-4b17-a827-4cfeecb0eb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e26d30550f34e1aaf0628c3824040a7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.035521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.036334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.049997 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2174/4349] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:40 2022] DELETE /identity/v3/users/2e26d30550f34e1aaf0628c3824040a7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:41.054488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.054773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.055047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.063014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=80,keystone:INSERT=4,keystone:DELETE=16 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:41.071602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.072226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.078237 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2176/4350] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:06:40 2022] PATCH /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7 => generated 276 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:41.079282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.080846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:41.081342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:41.082568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:41.146112 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2175/4351] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:41.156189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.157538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:41.161834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab171404-a599-43f4-a564-ab769f8e6d10 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.165428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:41.217010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4ceb3-0ce4-45cb-a4c9-37032411b54f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.220746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.220940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.221384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.221531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.221724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.221854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.221984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.222094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.231084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.232520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.233042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=aa4c3fe383d84c049c03373250eca4e7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.234739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.242666 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2176/4352] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:41.260029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.273127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.330205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7', 'request_id': 'req-699ce19a-6494-44b9-9635-fceb8794a7bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.339511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7', 'request_id': 'req-699ce19a-6494-44b9-9635-fceb8794a7bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.352304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7', 'request_id': 'req-699ce19a-6494-44b9-9635-fceb8794a7bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.366452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] The token cache is being invalidate because project aa4c3fe383d84c049c03373250eca4e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:41.366727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.367728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.378001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7', 'request_id': 'req-699ce19a-6494-44b9-9635-fceb8794a7bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.381771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7', 'request_id': 'req-699ce19a-6494-44b9-9635-fceb8794a7bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.382832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'aa4c3fe383d84c049c03373250eca4e7', 'request_id': 'req-699ce19a-6494-44b9-9635-fceb8794a7bc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.389684 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2177/4353] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:41 2022] DELETE /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:41.395046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.406316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e05616-46dd-47a0-adf3-f0e76225595f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.483846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.484533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.485724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.486005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.486722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.487791 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:41.487791 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:06:41.487791 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:06:41.489032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.497734 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2177/4354] 10.222.0.26 () {58 vars in 1173 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:41.511575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.513720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:41.534822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce19a-6494-44b9-9635-fceb8794a7bc tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.552443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.553046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.553321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.553628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.554265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.555302 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:06:41.555302 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:06:41.555302 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:06:41.556212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.561996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.565681 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2178/4355] 10.222.0.26 () {58 vars in 1200 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.584662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42308e2b-4900-41b3-9e4d-65f65f687d3e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.599077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.599946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.600232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.600505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.601105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.608972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.612654 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2178/4356] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.616869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.628431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7529c-fd85-41a5-9e6e-949583ad99d0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.647414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.648092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.648386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.648641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.649236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.651164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.652634 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2179/4357] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.670178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.681900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6769e3d-45b1-4819-b802-469d86bc3206 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.687330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.688387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.688387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.688611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.689080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.690813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.695509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2179/4358] 10.222.0.26 () {58 vars in 1188 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:41.697518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.707900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aca76d-3aa0-461d-a632-8fe9f68e9ee9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.713083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.713852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.714190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.714692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.715921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.717692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.721702 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2180/4359] 10.222.0.26 () {58 vars in 1200 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.722856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.722988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.724725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:41.736965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae5cc6-6e0d-4473-b5f3-8964671b882f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.760108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.760894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.761229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.761371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.762232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.764146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.771139 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2181/4360] 10.222.0.26 () {58 vars in 1203 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.772094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.779429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:41.783213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade638-6116-4bd4-9d18-8fff6dbf786c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.796434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.797094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.797374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.797696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.798291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.800430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.805006 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2182/4361] 10.222.0.26 () {58 vars in 1209 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.805161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60a6e7-f515-4b5e-a25c-861f36ea51e3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.806121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.810384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.811232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.811505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.811797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] PATH_INFO: `/v3/projects/5816f5fcc813404e93f6fb6c810e38ae` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.812949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5816f5fcc813404e93f6fb6c810e38ae)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.814983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.818145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89fbb-64d2-44bb-bd8f-23e09b7bd2eb tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.822955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.823744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.824081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.824442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.825078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.826884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.828151 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2183/4362] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.831983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.841698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46127be4-9376-468a-97b4-015d9b8f131f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.846423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.847123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.847409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.847666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.848233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.849870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.851022 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2184/4363] 10.222.0.26 () {58 vars in 1218 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:41.855384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.865566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a315a5-af09-4464-afd5-cd883055a969 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.870175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5816f5fcc813404e93f6fb6c810e38ae', 'request_id': 'req-fba50f91-ce73-41ab-a924-e95644f9609a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.875879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:41.882544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5816f5fcc813404e93f6fb6c810e38ae', 'request_id': 'req-fba50f91-ce73-41ab-a924-e95644f9609a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.892221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5816f5fcc813404e93f6fb6c810e38ae', 'request_id': 'req-fba50f91-ce73-41ab-a924-e95644f9609a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.904242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] The token cache is being invalidate because project 5816f5fcc813404e93f6fb6c810e38ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:41.904242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.904766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:41.906314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2180/4364] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:41 2022] DELETE /identity/v3/projects/5816f5fcc813404e93f6fb6c810e38ae => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:41.911258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.913270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:41.934952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.935604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.935878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.936148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.936736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.938392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.939732 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2185/4365] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:41.943849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.945603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:41.958185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa27801-800b-4471-883d-4607f0839142 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.962774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.963399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.963709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.963982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.964559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.966186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.967327 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2186/4366] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:41.971293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:41.979847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba50f91-ce73-41ab-a924-e95644f9609a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.980693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780757af-fd21-42ca-8ff0-aa33d0ebe7df tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:41.984393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.985024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.985192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:41.985320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.985602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] PATH_INFO: `/v3/users/156d9b5e2ae34e07b1054aed56680e5d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.985829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:41.986140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:41.986415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:41.986762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=156d9b5e2ae34e07b1054aed56680e5d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.987091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:41.988686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.988809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:41.989828 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2187/4367] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:41.993528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.003819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9476fb-d5c8-46e2-84a0-d526789b3b86 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.008720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.009317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.009628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.009904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.010473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.012029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.013175 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2188/4368] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:06:41 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.017528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.030602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a35b90-7b3a-4079-96db-ede05fa2938f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.037825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.039611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.041065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.041701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.042422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.042574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156d9b5e2ae34e07b1054aed56680e5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.043183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156d9b5e2ae34e07b1054aed56680e5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.044519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.046553 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2189/4369] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.049910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.055104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156d9b5e2ae34e07b1054aed56680e5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.055982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156d9b5e2ae34e07b1054aed56680e5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.061104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fb618-1a62-47ae-9cbb-067d469618d6 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.066178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.066844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.067151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.067775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.068075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.069715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.069859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156d9b5e2ae34e07b1054aed56680e5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.070904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2190/4370] 10.222.0.26 () {58 vars in 1239 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.071112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:42.075078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.076895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:42.085793 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2181/4371] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:41 2022] DELETE /identity/v3/users/156d9b5e2ae34e07b1054aed56680e5d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:42.090152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f15c4-753b-4ab4-ad7e-11335c278e99 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.163833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.164631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.165016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.165636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.166354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.170579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.177762 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2191/4372] 10.222.0.26 () {58 vars in 1242 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.181947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.194061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85416b-c6de-458f-9144-63f6e60bc66d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.198738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.199404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.199688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.199956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.200722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.202450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.203927 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2182/4373] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.208733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.220023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a51d-d80a-4044-9ef5-a700f72d1efd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.226713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.227397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.227674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.228103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.228713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.230452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.235539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2192/4374] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.236153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.247733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a862293-af01-459f-bf70-1701bf586f08 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.256045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.257208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.257740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.258401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.259368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.261918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.269945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.282261 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2183/4375] 10.222.0.26 () {58 vars in 1269 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.286189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb425d55-14b5-4fed-95e3-8cc973fc38d4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.291740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.293046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.293332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.293691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.294490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.296126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.297514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2193/4376] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.302685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.320499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a35de-f81b-4a87-bc70-98eec14a843e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.327106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.328052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.329214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.329770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.330699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.332809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.334540 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2184/4377] 10.222.0.26 () {58 vars in 1257 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.338424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.350062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf3440-1e1a-49f1-bd4d-fde09d9daf73 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.355770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.357116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.357116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.357116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.357597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.359593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.361544 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2194/4378] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.365426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.376004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78524b53-6f4f-4719-b4a5-ac5a8ac96b9a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.381586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.382276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.382557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.382826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.383421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.385178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.386309 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2185/4379] 10.222.0.26 () {58 vars in 1242 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.390782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.403613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c6d8c-e8d3-4876-92d1-697551c60396 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.419136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.420814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.421375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.421822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.424769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.427034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.428625 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2195/4380] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.432748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.444273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.445434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ede081-fc16-467e-82f2-a05936f1a06e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.453345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:42.457631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.458406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.458797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.459141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.459798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.461962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.464175 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2186/4381] 10.222.0.26 () {58 vars in 1254 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.467778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.479450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32994823-fc10-4b83-b8e7-ddd9bf755ed4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.485337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.486039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.486364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.486635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.487250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.488837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.492835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2187/4382] 10.222.0.26 () {58 vars in 1290 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.493801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.503611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca90f43-4be3-4917-846e-d00e65ead0a8 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.508907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.509565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.509869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.510146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.510723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.512289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.513345 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2188/4383] 10.222.0.26 () {58 vars in 1257 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.519061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.531254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c314d-b208-4617-a06a-a25880da977f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.532864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530c935-189c-4f68-98dc-34c97dcc2608 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.537664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.538891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.539237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.539692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.539878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.540144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.540433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] PATH_INFO: `/v3/projects/d7ce234c3fb34227b11dce573056c78e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.540782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.541609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ce234c3fb34227b11dce573056c78e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.542000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.543563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.544014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.545335 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2189/4384] 10.222.0.26 () {58 vars in 1266 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.549739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.561549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a920-fade-45a2-871a-2061acbcc292 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.566688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.567327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.567605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.567878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.568462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.568607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:42.570496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.571735 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2190/4385] 10.222.0.26 () {58 vars in 1308 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.576203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.586928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765fc45-693c-42b4-8a33-f7a7aff45772 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.600812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ce234c3fb34227b11dce573056c78e', 'request_id': 'req-ccb97c56-36f2-4a02-8268-418630755f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.615420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ce234c3fb34227b11dce573056c78e', 'request_id': 'req-ccb97c56-36f2-4a02-8268-418630755f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.624709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ce234c3fb34227b11dce573056c78e', 'request_id': 'req-ccb97c56-36f2-4a02-8268-418630755f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.637747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] The token cache is being invalidate because project d7ce234c3fb34227b11dce573056c78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:42.638456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.638685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.643328 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2196/4386] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:42 2022] DELETE /identity/v3/projects/d7ce234c3fb34227b11dce573056c78e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:42.645227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.646635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:42.664257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.665650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.666443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.666822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.667996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.674403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.674403 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2191/4387] 10.222.0.26 () {58 vars in 1290 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.684628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.688045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:42.706794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40d21b-b524-4fdb-b19a-22379ddd4386 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.730159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.730159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.730159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.730539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.730539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.730539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.730539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2192/4388] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.730937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb97c56-36f2-4a02-8268-418630755f3d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.733099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.735851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.736454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.736632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.736907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] PATH_INFO: `/v3/users/14cb6957da244aacac9548584d86cf9d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.738392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14cb6957da244aacac9548584d86cf9d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.740585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.748005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f12f6a-7124-49a4-ac65-9ee45b29cb8d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.753546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.754727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.755057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.755510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.756039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.757912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.759587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2193/4389] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.763894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.774755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a73a-c0ec-4e75-82cb-50248c70eaca tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.779812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.780666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.781033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.781416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.782159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.784109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.788627 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2194/4390] 10.222.0.26 () {58 vars in 1242 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.789664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.800617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53d8fb-b003-47c3-a2e4-64c0127ec5e4 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.802596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14cb6957da244aacac9548584d86cf9d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.806218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.806994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.807460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.807695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.808536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.809941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.811058 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2195/4391] 10.222.0.26 () {58 vars in 1245 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.815228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.816711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14cb6957da244aacac9548584d86cf9d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.826019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca7664-a8c9-414e-9a73-46c807967dd9 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.828431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14cb6957da244aacac9548584d86cf9d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.829202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14cb6957da244aacac9548584d86cf9d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.829968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14cb6957da244aacac9548584d86cf9d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:42.831805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.832766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.832766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.833012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.833609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.835208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.836479 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2196/4392] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.840362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.841921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:42.847984 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2197/4393] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:42 2022] DELETE /identity/v3/users/14cb6957da244aacac9548584d86cf9d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:42.855405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c51c08-1ad3-41d7-9ade-c62e02f3d737 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.928599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.929865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.930184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.930864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.931952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.934400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.936630 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2197/4394] 10.222.0.26 () {58 vars in 1254 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.941817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.955803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b5c8a-3b61-4f89-82c6-cbe6dafdd093 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.961448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.962218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.962562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.963314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.964101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.965759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.969520 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2198/4395] 10.222.0.26 () {58 vars in 1257 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:42.971638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:42.982935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50c9d0-2542-4530-916c-d3c52dfa7177 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:42.988356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:42.989043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:42.989320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:42.989635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:42.990232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:42.991849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:42.993298 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2198/4396] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:42.997702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.017117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67339cb9-cdb2-42e1-93e2-b856c119bcd3 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.023304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.024135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.024628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.024902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.025890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.027416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.028989 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2199/4397] 10.222.0.26 () {58 vars in 1257 bytes} [Sat Sep 17 21:06:42 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.033384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.053660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697000-fe4d-4865-b29a-53a483787b9c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.060032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.061597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.061978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.063845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.064628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.067344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.068949 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2199/4398] 10.222.0.26 () {58 vars in 1293 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.073976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.090442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776a458-2d92-4378-8db3-19be4d0a5b50 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.095910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.096675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.097055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.097437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.098507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.100237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.104696 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2200/4399] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.105854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.118702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25308c79-9c81-4a4f-aed8-e4c7ad2df71c tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.127852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.128918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.129742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.130107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.130893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.132761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.134605 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2200/4400] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.141291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.158294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d400-3cc9-4e39-9677-b04562c80a8f tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.166632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.167577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.167921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.168266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.169015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.171036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.172474 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2201/4401] 10.222.0.26 () {58 vars in 1212 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.176896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.190106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d8a97-f73f-49f1-87be-24d1e3f92bd7 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.197821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.198590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.198875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.199153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.199970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.201705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.203100 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2201/4402] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.207516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.219206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54d9f0-5163-4323-9309-3263c3857d5e tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.224226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.225891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.226938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.227284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.228747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.231437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.232818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2202/4403] 10.222.0.26 () {58 vars in 1215 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.240812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.251122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4d3b2-62ed-48d6-8c3a-e86f2262808a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.256806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.257813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.258103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.258397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.259021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.260985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.261970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2202/4404] 10.222.0.26 () {58 vars in 1218 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.265989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.277113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad735736-0d16-4dc8-9c14-ffa44e31d9d1 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.282267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.283127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.283471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.283814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.284473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.286344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.287603 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2203/4405] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.291855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.302580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892d883-1071-48ba-a051-430b96a7128a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.307751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.308546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.308918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.309274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.309950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.311708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.313027 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2203/4406] 10.222.0.26 () {58 vars in 1239 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.317215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.322470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:43.329247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d136fa-365c-4b9d-be2c-ff5b9a7b2617 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.336401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.337132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.337746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.338261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.339315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.341376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.345261 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2204/4407] 10.222.0.26 () {58 vars in 1260 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.346803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.358530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d515174-25ed-46c7-898d-fb415236f261 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.364477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.365131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.365405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.365713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.366342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.367977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.369262 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2204/4408] 10.222.0.26 () {58 vars in 1263 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.373082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.384706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fa252-354e-4f98-bdbf-c7460ec7bf7b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.390451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.390647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.390928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.391224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.391785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.393446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.395000 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2205/4409] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:06:43.399059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.403308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.405024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:43.405586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:43.411639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7b438-e879-4eb2-b4dd-d1fce0941c1a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.416908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.418120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.418519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.418902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.419672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.422307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.424125 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2205/4410] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:43.428060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.470896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c658c9-b637-484a-8dde-97f165cb5bc0 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.496436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.497111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.497454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.497789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.498404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.500066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.502921 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2206/4411] 10.222.0.26 () {58 vars in 1278 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 17 21:06:43.509084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.510309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b6a5c-aa08-4391-bbc6-5d77ddab567b tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.516808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.518031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.518552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.518895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] PATH_INFO: `/v3/projects/45988498b30f439fa6addefa8a66d65e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.520884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45988498b30f439fa6addefa8a66d65e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.532607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925f7da-85f7-45ba-873e-853e1762e07a tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.534052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.539710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.540508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.540793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.541068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/aa4c3fe383d84c049c03373250eca4e7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.551057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Could not find domain: aa4c3fe383d84c049c03373250eca4e7.: keystone.exception.DomainNotFound: Could not find domain: aa4c3fe383d84c049c03373250eca4e7. Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: aa4c3fe383d84c049c03373250eca4e7. Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:06:43.574466 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:06:43.578810 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:06:43.580340 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: aa4c3fe383d84c049c03373250eca4e7. Sep 17 21:06:43.581917 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.581917 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2207/4412] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:06:43 2022] PATCH /identity/v3/domains/aa4c3fe383d84c049c03373250eca4e7 => generated 112 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:06:43.581917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.598079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45988498b30f439fa6addefa8a66d65e', 'request_id': 'req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.599103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e447c-d1ac-4a8f-9a69-d64c3df4a27d tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.611738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45988498b30f439fa6addefa8a66d65e', 'request_id': 'req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.622584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45988498b30f439fa6addefa8a66d65e', 'request_id': 'req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.637210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] The token cache is being invalidate because project 45988498b30f439fa6addefa8a66d65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:43.637790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.640731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84eab881-3da9-4cd4-9f82-d9afbb95ae9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.642771 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2206/4413] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:43 2022] DELETE /identity/v3/projects/45988498b30f439fa6addefa8a66d65e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:43.648391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ef434-b006-43ba-a8ee-4dd7d51de334 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.648905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ef434-b006-43ba-a8ee-4dd7d51de334 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.649442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ef434-b006-43ba-a8ee-4dd7d51de334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.688686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-548ef434-b006-43ba-a8ee-4dd7d51de334 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:43.690186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-548ef434-b006-43ba-a8ee-4dd7d51de334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:43.718017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.718627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.719512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.719512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.728044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Could not find domain: bb93e8fe272c46ffa83e47d8638ea2c7.: keystone.exception.DomainNotFound: Could not find domain: bb93e8fe272c46ffa83e47d8638ea2c7. Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: bb93e8fe272c46ffa83e47d8638ea2c7. Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:06:43.735169 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:06:43.736513 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:06:43.738209 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: bb93e8fe272c46ffa83e47d8638ea2c7. Sep 17 21:06:43.739599 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.739599 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2208/4414] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:06:43 2022] PATCH /identity/v3/domains/bb93e8fe272c46ffa83e47d8638ea2c7 => generated 112 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:06:43.740822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.742704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:43.757295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7eb09-a15e-4db3-8686-c03c2bde2335 tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.762410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97eb464e3314c46bbba5fb2c3ec2eb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '117de53009b149aeae9148901cd3206d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.763077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.763416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.763760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] PATH_INFO: `/v3/domains/72dbc84f440f46f5938f57245fb3388a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.775579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:06:43.776728 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2207/4415] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Could not find domain: 72dbc84f440f46f5938f57245fb3388a.: keystone.exception.DomainNotFound: Could not find domain: 72dbc84f440f46f5938f57245fb3388a. Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 72dbc84f440f46f5938f57245fb3388a. Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:06:43.781032 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:06:43.785294 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:06:43.786762 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 72dbc84f440f46f5938f57245fb3388a. Sep 17 21:06:43.788199 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:06:43.788199 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2209/4416] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:06:43 2022] PATCH /identity/v3/domains/72dbc84f440f46f5938f57245fb3388a => generated 112 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:06:43.788384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187330bf-53a5-43dd-8f47-680cf459c1da None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.788384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187330bf-53a5-43dd-8f47-680cf459c1da None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.788384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187330bf-53a5-43dd-8f47-680cf459c1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.791337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.793106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:43.807776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-187330bf-53a5-43dd-8f47-680cf459c1da None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:43.809064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-187330bf-53a5-43dd-8f47-680cf459c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:43.811622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694e79-6424-40ea-ac3e-c759a3b2c6fd tempest-DomainConfigurationTestJSON-1573044734 tempest-DomainConfigurationTestJSON-1573044734-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.817958 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2208/4417] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:43.821289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.821289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.821289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.821289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] PATH_INFO: `/v3/users/3aef0cd15055441a84ecf30f52d56d02` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.821720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aef0cd15055441a84ecf30f52d56d02)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.822794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187330bf-53a5-43dd-8f47-680cf459c1da None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.822961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.832414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187330bf-53a5-43dd-8f47-680cf459c1da None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.837300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.838009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.838341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.838677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.839387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:43.839749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.841501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.854159 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2209/4418] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:43.858718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.866152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:43.871934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39c38-99b4-405e-ae59-64731e582613 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:43.898643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:43.902360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.903333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.904399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:43.907658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:43.916580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:43.930466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aef0cd15055441a84ecf30f52d56d02'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.932006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aef0cd15055441a84ecf30f52d56d02'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.948344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aef0cd15055441a84ecf30f52d56d02'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.948989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aef0cd15055441a84ecf30f52d56d02'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.963793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-07af1515-1a1e-449d-a507-4d65977f8d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aef0cd15055441a84ecf30f52d56d02'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:43.967291 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2210/4419] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:43 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:43.971619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:43.973645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:43.980324 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2210/4420] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:43 2022] DELETE /identity/v3/users/3aef0cd15055441a84ecf30f52d56d02 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:43.982761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:43.983073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:43.983320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.007338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:44.008649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:44.043180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42f1dd-cf2f-4934-9bd7-5739eab14f07 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.060504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:44.082393 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2211/4421] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:44.093329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.095264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:44.112927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.113822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.114098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.114419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.115047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.117269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.143688 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2211/4422] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:43 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:44.147636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.158736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bf37d-7471-4c81-876a-5482e4a289ef admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.165590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.165940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.166059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.166604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.167048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.167541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2fb9b-6b0b-44a1-8a91-f8fa67f3f6c3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.168702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.172399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.173090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.173431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.173821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.181807 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2212/4423] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:44.185097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.186119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.186851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.197787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc1589-e29b-4cb5-94fe-37e7f23938b7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.199190 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2212/4424] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:44.205083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed471a-2999-4b09-b3d5-1e2919d27096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.205868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed471a-2999-4b09-b3d5-1e2919d27096 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.206239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed471a-2999-4b09-b3d5-1e2919d27096 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.206606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed471a-2999-4b09-b3d5-1e2919d27096 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.207353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed471a-2999-4b09-b3d5-1e2919d27096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.209074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed471a-2999-4b09-b3d5-1e2919d27096 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.221197 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2213/4425] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:44.225553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.238561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bc6cd-06b7-4f59-a31d-b4eafc5829b2 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.245181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.246054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.246501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.246913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] PATH_INFO: `/v3/projects/3e857c532a264ea78e24fd8fe1a44fbc/users/33784305478f46ae9fdc4d4b82ebc6f5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.280624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e857c532a264ea78e24fd8fe1a44fbc, user_id=33784305478f46ae9fdc4d4b82ebc6f5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.283557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.317832 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2213/4426] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:44 2022] PUT /identity/v3/projects/3e857c532a264ea78e24fd8fe1a44fbc/users/33784305478f46ae9fdc4d4b82ebc6f5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:44.320226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.320514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.320785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.345805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:44.347157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:44.422781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:44.444056 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2214/4427] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:44.454121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.512939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46f545-da61-4546-b0b8-185dd42d7178 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.516674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.517560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.517837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.518108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.528104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.529620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.539273 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2214/4428] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:44.828237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.830096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:44.868556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac9bd4-1647-4e9f-9ea6-0ee9ef93c6aa None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.916224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.917174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.917642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.917963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] PATH_INFO: `/v3/projects/e26e0c8443db417b9df0fd3b076b75aa` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.920362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26e0c8443db417b9df0fd3b076b75aa)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.922861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.052128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26e0c8443db417b9df0fd3b076b75aa', 'request_id': 'req-85d5be5e-28cc-42d2-8180-c0cd86a608a2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.073242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26e0c8443db417b9df0fd3b076b75aa', 'request_id': 'req-85d5be5e-28cc-42d2-8180-c0cd86a608a2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.083060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26e0c8443db417b9df0fd3b076b75aa', 'request_id': 'req-85d5be5e-28cc-42d2-8180-c0cd86a608a2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.110657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] The token cache is being invalidate because project e26e0c8443db417b9df0fd3b076b75aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:45.111000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.113003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.124560 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2215/4429] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:44 2022] DELETE /identity/v3/projects/e26e0c8443db417b9df0fd3b076b75aa => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:45.129705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.143736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f9d76-19a6-4887-9ffe-18d1f79fd47a service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.286066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.289103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.289477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.289760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] PATH_INFO: `/v3/users/d97eb464e3314c46bbba5fb2c3ec2eb4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.291067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d97eb464e3314c46bbba5fb2c3ec2eb4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.297224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.323267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:45.393660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd97eb464e3314c46bbba5fb2c3ec2eb4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.394399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd97eb464e3314c46bbba5fb2c3ec2eb4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.403439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd97eb464e3314c46bbba5fb2c3ec2eb4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.404130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd97eb464e3314c46bbba5fb2c3ec2eb4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.423979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7053856f-cf0c-4047-93a8-520e231afa2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd97eb464e3314c46bbba5fb2c3ec2eb4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.442592 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2215/4430] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:45 2022] DELETE /identity/v3/users/d97eb464e3314c46bbba5fb2c3ec2eb4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:45.911911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.914319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:45.937422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5be5e-28cc-42d2-8180-c0cd86a608a2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.973829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.975800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.976254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.023393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] MFA Rules not processed for user `33784305478f46ae9fdc4d4b82ebc6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:46.025969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:46.052246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.053390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.054201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.054715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] PATH_INFO: `/v3/projects/117de53009b149aeae9148901cd3206d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.056603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=117de53009b149aeae9148901cd3206d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.059151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.123019 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2216/4431] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:46.132184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117de53009b149aeae9148901cd3206d', 'request_id': 'req-5c408d77-f095-4500-be27-92809db87196'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:46.140648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.142209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:46.145001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117de53009b149aeae9148901cd3206d', 'request_id': 'req-5c408d77-f095-4500-be27-92809db87196'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:46.152802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117de53009b149aeae9148901cd3206d', 'request_id': 'req-5c408d77-f095-4500-be27-92809db87196'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:46.164577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] The token cache is being invalidate because project 117de53009b149aeae9148901cd3206d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:46.165019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:46.165841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c408d77-f095-4500-be27-92809db87196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:46.167422 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2216/4432] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:45 2022] DELETE /identity/v3/projects/117de53009b149aeae9148901cd3206d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:46.189716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3168b7-c30f-4b0e-a226-6723624e1bc8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.189716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3168b7-c30f-4b0e-a226-6723624e1bc8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.189716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3168b7-c30f-4b0e-a226-6723624e1bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.213634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ec3168b7-c30f-4b0e-a226-6723624e1bc8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:46.214929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ec3168b7-c30f-4b0e-a226-6723624e1bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:46.221362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435f44a-6bbb-4e56-8e5c-03a973e5fce8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.222400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:46.264805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:46.291391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.292316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.292933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.293068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.296245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:46.309930 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2217/4433] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:46.318771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8819c-7212-43a4-af47-5c13ca0f5988 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.319039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8819c-7212-43a4-af47-5c13ca0f5988 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.319312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8819c-7212-43a4-af47-5c13ca0f5988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.340716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c7b8819c-7212-43a4-af47-5c13ca0f5988 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:46.341860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c7b8819c-7212-43a4-af47-5c13ca0f5988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:46.355017 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2218/4434] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:46.359115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8819c-7212-43a4-af47-5c13ca0f5988 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.369797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8819c-7212-43a4-af47-5c13ca0f5988 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.372971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.374451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.375416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.376820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.377630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.377630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.378421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:46.378821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.380944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.387140 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2217/4435] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:46.397490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2219/4436] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:06:46.398613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.410137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd79f-ccda-4791-929e-e8356087f4d9 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.416726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.417589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.418382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.418903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.419847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.422306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.457623 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2218/4437] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:46 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:46.461397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.471901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c20ed-fade-4438-a425-1ac7e819bfb2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.548346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.549031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.549306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.549611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.550320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.552346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.578065 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2220/4438] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:46 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:46.582298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.597003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277330e2-1359-4922-8eda-a33dda9b3424 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.602751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.603417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.603865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.604174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.604931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.606747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.620739 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2219/4439] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:46.624837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.649609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f5c48-45c5-4d9d-85ee-ef5255f0bc60 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.655432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdb9d1-0413-47fb-adf8-7cb5be92a7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.656260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb9d1-0413-47fb-adf8-7cb5be92a7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.656640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb9d1-0413-47fb-adf8-7cb5be92a7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.656925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb9d1-0413-47fb-adf8-7cb5be92a7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.658054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdb9d1-0413-47fb-adf8-7cb5be92a7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.660986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdb9d1-0413-47fb-adf8-7cb5be92a7c3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.678227 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2221/4440] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:46.680774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.693634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802b4bf-143b-467a-b92f-b2f75957bd5d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.701491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.702794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.703395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.703998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] PATH_INFO: `/v3/projects/13f57be9c654416ab973d4feef28bff6/users/67156d2a8edf422088b711ba7293ec61/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.726099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f57be9c654416ab973d4feef28bff6, user_id=67156d2a8edf422088b711ba7293ec61, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.729397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:46.758559 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2220/4441] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:46 2022] PUT /identity/v3/projects/13f57be9c654416ab973d4feef28bff6/users/67156d2a8edf422088b711ba7293ec61/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:46.769104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b68b85-4b39-4180-b01a-dd4715ad17cf None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.769433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b68b85-4b39-4180-b01a-dd4715ad17cf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.769675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b68b85-4b39-4180-b01a-dd4715ad17cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.800015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-01b68b85-4b39-4180-b01a-dd4715ad17cf None None] MFA Rules not processed for user `67156d2a8edf422088b711ba7293ec61`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:46.801295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-01b68b85-4b39-4180-b01a-dd4715ad17cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:46.876081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:46.882729 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2222/4442] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:46.886265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:46.964687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec151ed-212d-47e7-99fc-9ccf5dfe31fd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:46.970221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:46.971750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:46.971750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:46.972104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:46.972829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:46.974970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.007711 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2221/4443] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:46 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:47.011702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b68b85-4b39-4180-b01a-dd4715ad17cf None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.024978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b68b85-4b39-4180-b01a-dd4715ad17cf None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.089775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.090408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.090679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.090957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.091605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.093593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.120140 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2223/4444] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:47 2022] POST /identity/v3/users => generated 301 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:47.121883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.134282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e412bd-88bb-4884-ba12-cbd1c0f42caa admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.139159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.139770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.140046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.140314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.140956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.142648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.155970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2222/4445] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:47.160389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.170903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c152d-c68a-4520-ae38-6f32f4d2353c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.175962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.176640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.176918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.177192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] PATH_INFO: `/v3/projects/ce52c6babe4e45f592c3407c0753053e/users/e64fbc53d4874d13ae55bf4d3fb5499c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.193294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce52c6babe4e45f592c3407c0753053e, user_id=e64fbc53d4874d13ae55bf4d3fb5499c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.196069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.210252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2224/4446] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:47 2022] PUT /identity/v3/projects/ce52c6babe4e45f592c3407c0753053e/users/e64fbc53d4874d13ae55bf4d3fb5499c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:47.214974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.220935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:47.225032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116405a8-0b21-4360-a0c8-96910bbfddbc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.288955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e747a6-dcc1-45f0-9f58-cbc9dd085377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.289714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e747a6-dcc1-45f0-9f58-cbc9dd085377 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.290062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e747a6-dcc1-45f0-9f58-cbc9dd085377 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.290401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e747a6-dcc1-45f0-9f58-cbc9dd085377 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.291142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e747a6-dcc1-45f0-9f58-cbc9dd085377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.292777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e747a6-dcc1-45f0-9f58-cbc9dd085377 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.306666 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2223/4447] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:47.307884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.319670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b197c-fc15-4631-9987-b59fae42fba2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.324678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.325508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.325882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.326246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] PATH_INFO: `/v3/projects/ce52c6babe4e45f592c3407c0753053e/users/e64fbc53d4874d13ae55bf4d3fb5499c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.328438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce52c6babe4e45f592c3407c0753053e, user_id=e64fbc53d4874d13ae55bf4d3fb5499c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.331444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.343010 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2225/4448] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:47 2022] PUT /identity/v3/projects/ce52c6babe4e45f592c3407c0753053e/users/e64fbc53d4874d13ae55bf4d3fb5499c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:47.354828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83acf81-fcec-4100-a5f0-1bcfd725b43c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.354828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83acf81-fcec-4100-a5f0-1bcfd725b43c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.354828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83acf81-fcec-4100-a5f0-1bcfd725b43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.389117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b83acf81-fcec-4100-a5f0-1bcfd725b43c None None] MFA Rules not processed for user `e64fbc53d4874d13ae55bf4d3fb5499c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:47.390301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b83acf81-fcec-4100-a5f0-1bcfd725b43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:47.472868 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2224/4449] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:47.477414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.488816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc936c33-97a1-4e40-b173-9169757fd75f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.494361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.495191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.495531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.495872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects/13f57be9c654416ab973d4feef28bff6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.497071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:get_project(project_id=13f57be9c654416ab973d4feef28bff6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.499277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.501182 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2226/4450] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/projects/13f57be9c654416ab973d4feef28bff6 => generated 387 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:47.505692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83acf81-fcec-4100-a5f0-1bcfd725b43c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.514994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83acf81-fcec-4100-a5f0-1bcfd725b43c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.519656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.520264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.520542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.521783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.522672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.524916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.556087 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2225/4451] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:06:47 2022] POST /identity/v3/projects => generated 339 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:47.560511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.570662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5deef-0580-4c35-a418-1b26f05c621d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.646995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.648355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.648763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.649111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.650015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.652227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.701714 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2227/4452] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/projects => generated 6590 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:47.704324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.724932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b23219-ccb5-436f-baa1-0d2e894bb70e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.731315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.741115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.741909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.742368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects/13f57be9c654416ab973d4feef28bff6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.747060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:get_project(project_id=13f57be9c654416ab973d4feef28bff6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.749377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.753509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2226/4453] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/projects/13f57be9c654416ab973d4feef28bff6 => generated 387 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:47.756013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.766380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c02424-0bf2-4138-af26-520aafdd3f1f tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.771364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.772077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.772427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.772772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects/1e8c6c2ce8eb4bb2a68db244554fbefd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.774007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:get_project(project_id=1e8c6c2ce8eb4bb2a68db244554fbefd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.776158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.781506 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2228/4454] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/projects/1e8c6c2ce8eb4bb2a68db244554fbefd => generated 339 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:06:47.783293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.795666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ddbaf-48fb-4b18-920c-78594b4d818d tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.806137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022295ab-67fe-49fe-a519-9dee62753ae6 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.807106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022295ab-67fe-49fe-a519-9dee62753ae6 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.807548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022295ab-67fe-49fe-a519-9dee62753ae6 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.807830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022295ab-67fe-49fe-a519-9dee62753ae6 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.808561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022295ab-67fe-49fe-a519-9dee62753ae6 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.810867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022295ab-67fe-49fe-a519-9dee62753ae6 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.837540 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2227/4455] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/projects => generated 6590 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:47.845999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.868175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300c553-1d8d-4368-8cb2-fae32c708b74 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.873493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.874326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.874826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.875280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.876650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:47.876650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.878963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:47.928509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32b7c0-7316-4678-b6a8-ec32bd7f3060 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.928814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32b7c0-7316-4678-b6a8-ec32bd7f3060 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.929064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32b7c0-7316-4678-b6a8-ec32bd7f3060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.951374 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2229/4456] 10.222.0.26 () {58 vars in 1166 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/projects?domain_id=default => generated 6259 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:47.953814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ef32b7c0-7316-4678-b6a8-ec32bd7f3060 None None] MFA Rules not processed for user `33784305478f46ae9fdc4d4b82ebc6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:47.957308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:47.958924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ef32b7c0-7316-4678-b6a8-ec32bd7f3060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:47.969978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcee5c1-9e79-4f23-9df7-ee052abc9eb2 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:47.976056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:47.976671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:47.976973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:47.977248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:47.978377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:47.979803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:48.003624 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2230/4457] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:06:47 2022] GET /identity/v3/projects => generated 6590 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:48.005223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:48.011422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:48.018666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2c9b3-cae8-408c-b140-0881a5dd717e tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:48.030285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:48.031190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:48.031633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:48.032096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:48.032999 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2228/4458] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:48.033362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:48.033870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:48.036936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:48.047157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:48.061894 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2231/4459] 10.222.0.26 () {58 vars in 1166 bytes} [Sat Sep 17 21:06:48 2022] GET /identity/v3/projects?domain_id=default => generated 6259 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:48.067638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32b7c0-7316-4678-b6a8-ec32bd7f3060 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:48.081041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32b7c0-7316-4678-b6a8-ec32bd7f3060 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:48.085987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:48.086948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:48.087194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:48.087536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:48.088252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:48.090711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:48.117767 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2229/4460] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:06:48 2022] GET /identity/v3/projects => generated 6590 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:48.126571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:48.137619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daffcb-7270-4c13-b6e1-cad9305200a3 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:48.142666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:48.143402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:48.143793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:48.144090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:48.144832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-330322680) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:48.145205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:48.147386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:48.169123 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2232/4461] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:06:48 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-330322680 => generated 534 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:48.174846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:48.185657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8077b6-d7a9-463b-9a85-960d0e877cb4 tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:48.190717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e64fbc53d4874d13ae55bf4d3fb5499c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce52c6babe4e45f592c3407c0753053e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:48.191371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:48.191652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:48.191931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] PATH_INFO: `/v3/projects/1e8c6c2ce8eb4bb2a68db244554fbefd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:48.193578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1e8c6c2ce8eb4bb2a68db244554fbefd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:48.195260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:48.280893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8c6c2ce8eb4bb2a68db244554fbefd', 'request_id': 'req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.290360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8c6c2ce8eb4bb2a68db244554fbefd', 'request_id': 'req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.302299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8c6c2ce8eb4bb2a68db244554fbefd', 'request_id': 'req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.318226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] The token cache is being invalidate because project 1e8c6c2ce8eb4bb2a68db244554fbefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:48.318359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.319034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a41c68b2-e7a1-4e57-bc6c-e3ca2a3024df tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.320717 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2230/4462] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:48 2022] DELETE /identity/v3/projects/1e8c6c2ce8eb4bb2a68db244554fbefd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:48.324727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:48.326481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:48.410019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337034-7cf8-4e90-96c0-63cb4b0eff9b tempest-ListProjectsStaticTestJSON-2039840473 tempest-ListProjectsStaticTestJSON-2039840473-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:48.482842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:48.483730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:48.484041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:48.484408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] PATH_INFO: `/v3/users/67156d2a8edf422088b711ba7293ec61` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:48.486259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67156d2a8edf422088b711ba7293ec61)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:48.490178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:48.540563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67156d2a8edf422088b711ba7293ec61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.554230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67156d2a8edf422088b711ba7293ec61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.562972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67156d2a8edf422088b711ba7293ec61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.563668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67156d2a8edf422088b711ba7293ec61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.564368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67156d2a8edf422088b711ba7293ec61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:48.578784 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2233/4463] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:48 2022] DELETE /identity/v3/users/67156d2a8edf422088b711ba7293ec61 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:48.583429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:48.583713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:48.583985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:48.604393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:48.606050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:48.705837 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2231/4464] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:48.715247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:48.716674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:48.730864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:48.782009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcfc96-fe60-4fcd-9724-abeabf2747d5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:48.786336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:48.787038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:48.787389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:48.787732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:48.808481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:48.810007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:48.830773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2234/4465] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:49.312203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:49.313796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:49.341172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd689ce-9dcd-4ded-b67a-442a29ec1b88 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:49.349353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:49.350391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:49.350882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:49.351813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] PATH_INFO: `/v3/projects/13f57be9c654416ab973d4feef28bff6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:49.352718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f57be9c654416ab973d4feef28bff6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:49.355265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:49.413821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f57be9c654416ab973d4feef28bff6', 'request_id': 'req-ce949714-0078-4df8-b304-2f0e40dfdfa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.424311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f57be9c654416ab973d4feef28bff6', 'request_id': 'req-ce949714-0078-4df8-b304-2f0e40dfdfa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.427720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:49.438011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f57be9c654416ab973d4feef28bff6', 'request_id': 'req-ce949714-0078-4df8-b304-2f0e40dfdfa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.455742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] The token cache is being invalidate because project 13f57be9c654416ab973d4feef28bff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:49.456122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.457391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.461957 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2232/4466] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:49 2022] DELETE /identity/v3/projects/13f57be9c654416ab973d4feef28bff6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:49.463460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:49.474469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fa24-3455-4ab2-ab11-939edda78e92 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:49.559657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:49.560462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:49.560922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:49.561201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] PATH_INFO: `/v3/users/e64fbc53d4874d13ae55bf4d3fb5499c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:49.562597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e64fbc53d4874d13ae55bf4d3fb5499c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:49.564611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:49.640306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e64fbc53d4874d13ae55bf4d3fb5499c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.660076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e64fbc53d4874d13ae55bf4d3fb5499c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.670242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e64fbc53d4874d13ae55bf4d3fb5499c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.671443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e64fbc53d4874d13ae55bf4d3fb5499c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.672332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d3ce943-63e6-4e9c-a11c-058ce8b7e563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e64fbc53d4874d13ae55bf4d3fb5499c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.691609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2235/4467] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:49 2022] DELETE /identity/v3/users/e64fbc53d4874d13ae55bf4d3fb5499c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:50.085201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:50.086848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:50.120675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce949714-0078-4df8-b304-2f0e40dfdfa5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:50.249315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:50.250285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:50.250539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:50.250970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] PATH_INFO: `/v3/projects/ce52c6babe4e45f592c3407c0753053e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:50.254073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce52c6babe4e45f592c3407c0753053e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:50.256953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:50.366712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce52c6babe4e45f592c3407c0753053e', 'request_id': 'req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:50.384574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce52c6babe4e45f592c3407c0753053e', 'request_id': 'req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:50.399338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce52c6babe4e45f592c3407c0753053e', 'request_id': 'req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:50.445813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] The token cache is being invalidate because project ce52c6babe4e45f592c3407c0753053e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:50.446273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:50.447012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b70f34d-3f6c-4c2b-bf5c-d6a87300d15c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:50.449965 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2233/4468] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:50 2022] DELETE /identity/v3/projects/ce52c6babe4e45f592c3407c0753053e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:50.457362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab684100-9186-4adb-a55c-0bf9152dd3f6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:50.457745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab684100-9186-4adb-a55c-0bf9152dd3f6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:50.458021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab684100-9186-4adb-a55c-0bf9152dd3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:50.485161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ab684100-9186-4adb-a55c-0bf9152dd3f6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:50.486733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ab684100-9186-4adb-a55c-0bf9152dd3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:50.543027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:50.618115 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2236/4469] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:50.628292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36a6ef-3c70-4937-b024-e7b33510cfaf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:50.628998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36a6ef-3c70-4937-b024-e7b33510cfaf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:50.630784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36a6ef-3c70-4937-b024-e7b33510cfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:50.679147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2f36a6ef-3c70-4937-b024-e7b33510cfaf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:50.685419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2f36a6ef-3c70-4937-b024-e7b33510cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:50.718833 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2234/4470] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:50.723195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab684100-9186-4adb-a55c-0bf9152dd3f6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:50.735622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab684100-9186-4adb-a55c-0bf9152dd3f6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:50.744879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:50.745971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:50.746593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:50.746953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:50.747896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:50.748418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:50.750829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:50.768832 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2237/4471] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:50.773116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36a6ef-3c70-4937-b024-e7b33510cfaf None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:50.791581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36a6ef-3c70-4937-b024-e7b33510cfaf None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:50.802424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:50.807470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:50.809213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:50.810102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:50.813036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:50.818270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:50.880328 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2235/4472] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:50 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:50.881707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:50.898914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32754c-ed64-4d90-8a0e-6ab88f4450d4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:50.978601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:50.979243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:50.979669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:50.980235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:50.981146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:50.983141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.014668 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2238/4473] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:50 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:51.019227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.033308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72679ae-5fa0-48c4-8c85-2fa1acb3cc98 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.039479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.040307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.040687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.041056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.041858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.043793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.062689 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2236/4474] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:51.063511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.074921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eb944-5dfe-4634-b5bd-378d09ec3dd7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.080201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.081058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.081282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.081581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] PATH_INFO: `/v3/projects/10b0d812445a4f6a8049268e1b5d8443/users/baf684ee6c1d422a94713b520b56f62e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.099680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b0d812445a4f6a8049268e1b5d8443, user_id=baf684ee6c1d422a94713b520b56f62e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.102581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.118567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2239/4475] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:51 2022] PUT /identity/v3/projects/10b0d812445a4f6a8049268e1b5d8443/users/baf684ee6c1d422a94713b520b56f62e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:51.122046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.135111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cceb78-ab6e-4f65-a363-a189df2061e6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.173626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:51.218496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb3d8a-c1a8-4f52-bf64-57217274c614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.219123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb3d8a-c1a8-4f52-bf64-57217274c614 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.219395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb3d8a-c1a8-4f52-bf64-57217274c614 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.219666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb3d8a-c1a8-4f52-bf64-57217274c614 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.220363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb3d8a-c1a8-4f52-bf64-57217274c614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.221983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb3d8a-c1a8-4f52-bf64-57217274c614 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.235817 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2237/4476] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:51.237545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.249583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0525-1b73-45d5-8356-f1734f51c8d4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.254578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.255236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.255518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.255787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] PATH_INFO: `/v3/projects/10b0d812445a4f6a8049268e1b5d8443/users/baf684ee6c1d422a94713b520b56f62e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.257840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b0d812445a4f6a8049268e1b5d8443, user_id=baf684ee6c1d422a94713b520b56f62e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.260509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.271563 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2240/4477] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:51 2022] PUT /identity/v3/projects/10b0d812445a4f6a8049268e1b5d8443/users/baf684ee6c1d422a94713b520b56f62e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:51.282359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c104d4-6500-4305-a85b-d3eba26b5453 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.282650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c104d4-6500-4305-a85b-d3eba26b5453 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.285259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c104d4-6500-4305-a85b-d3eba26b5453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.315834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d6c104d4-6500-4305-a85b-d3eba26b5453 None None] MFA Rules not processed for user `baf684ee6c1d422a94713b520b56f62e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:51.317064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d6c104d4-6500-4305-a85b-d3eba26b5453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:51.392222 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2238/4478] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:51.396882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.465044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cc2d0-e1a3-47d8-9c15-cb9825d78500 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.470391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.471170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.471489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.471684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.472504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.474574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.517193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:51.520948 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2241/4479] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:51 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:51.525149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c104d4-6500-4305-a85b-d3eba26b5453 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.536577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c104d4-6500-4305-a85b-d3eba26b5453 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.596514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.597167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.597407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.597734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.598383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.600270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.630268 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2239/4480] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:51 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:51.633956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.661432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ab8a8-48ca-4c3c-8607-895b14a73c82 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.666446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.667094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.667379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.667647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.668457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.670322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.681360 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2242/4481] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:51.685547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.697382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f953-b2ac-4170-b6bc-fb905c8588c0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.702369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eaa4d5-4052-427e-adb1-fa79e58b1ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.702980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaa4d5-4052-427e-adb1-fa79e58b1ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.703258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaa4d5-4052-427e-adb1-fa79e58b1ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.703532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaa4d5-4052-427e-adb1-fa79e58b1ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.704310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eaa4d5-4052-427e-adb1-fa79e58b1ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.705866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eaa4d5-4052-427e-adb1-fa79e58b1ff1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.716135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2240/4482] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:51.720489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.731085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e07acc-c9c2-403d-b5ba-5afecf9e2783 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.736219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.736820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.737095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.737361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] PATH_INFO: `/v3/projects/ca2bef71179846efb45c1fb862f12594/users/c30732ceca204667b199e2e2fca633d5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.761917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2bef71179846efb45c1fb862f12594, user_id=c30732ceca204667b199e2e2fca633d5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:51.764591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:51.778899 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2243/4483] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:51 2022] PUT /identity/v3/projects/ca2bef71179846efb45c1fb862f12594/users/c30732ceca204667b199e2e2fca633d5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:51.789593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94057d-e712-4901-8f31-beb289887b5a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.789874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94057d-e712-4901-8f31-beb289887b5a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.790142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94057d-e712-4901-8f31-beb289887b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:51.819534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4c94057d-e712-4901-8f31-beb289887b5a None None] MFA Rules not processed for user `c30732ceca204667b199e2e2fca633d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:51.821405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4c94057d-e712-4901-8f31-beb289887b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:51.845850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:51.891750 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2241/4484] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:51.903537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:51.906750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:51.976178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b899721-8f0f-4c34-bf2d-e8374d15a957 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:51.979722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc4310-0126-45b4-93f2-92592565b50f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:51.980327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc4310-0126-45b4-93f2-92592565b50f service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:51.980601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc4310-0126-45b4-93f2-92592565b50f service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:51.980867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc4310-0126-45b4-93f2-92592565b50f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:52.048698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc4310-0126-45b4-93f2-92592565b50f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:52.050290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc4310-0126-45b4-93f2-92592565b50f service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:52.060031 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2244/4485] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:52.836180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94057d-e712-4901-8f31-beb289887b5a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:52.920127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94057d-e712-4901-8f31-beb289887b5a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:52.925198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:52.926048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:52.926436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:52.926800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] PATH_INFO: `/v3/users/baf684ee6c1d422a94713b520b56f62e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:52.928062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf684ee6c1d422a94713b520b56f62e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:52.930322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:52.984481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf684ee6c1d422a94713b520b56f62e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:52.985290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf684ee6c1d422a94713b520b56f62e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:52.998793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf684ee6c1d422a94713b520b56f62e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.003649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf684ee6c1d422a94713b520b56f62e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.030507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf684ee6c1d422a94713b520b56f62e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.045807 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2242/4486] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:52 2022] DELETE /identity/v3/users/baf684ee6c1d422a94713b520b56f62e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:53.050493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:53.051107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:53.051408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:53.076410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:53.077903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:53.173944 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2245/4487] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:53.385760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:53.387988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:53.452968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:53.471389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d2175-6245-49eb-a470-ab7524efa501 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:53.477437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:53.477683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:53.478122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:53.478350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:53.492763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:53.495370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:53.513683 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2243/4488] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:53.829851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:53.831333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:53.849040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119b0cd-fa1b-41e4-8afe-b489234a7efc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:53.853956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:53.854602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:53.854879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:53.855151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] PATH_INFO: `/v3/projects/10b0d812445a4f6a8049268e1b5d8443` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:53.856272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b0d812445a4f6a8049268e1b5d8443)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:53.858239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:53.868508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:53.956703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b0d812445a4f6a8049268e1b5d8443', 'request_id': 'req-dd53608e-082c-4701-a750-cf776e63357f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.967913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b0d812445a4f6a8049268e1b5d8443', 'request_id': 'req-dd53608e-082c-4701-a750-cf776e63357f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.975252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b0d812445a4f6a8049268e1b5d8443', 'request_id': 'req-dd53608e-082c-4701-a750-cf776e63357f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.987156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] The token cache is being invalidate because project 10b0d812445a4f6a8049268e1b5d8443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:53.987445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.988089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:53.989605 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2246/4489] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:53 2022] DELETE /identity/v3/projects/10b0d812445a4f6a8049268e1b5d8443 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:53.994281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:54.009189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718412ed-4420-4969-bfc2-357c8623d885 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:54.067680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:54.069932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:54.070227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:54.070617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] PATH_INFO: `/v3/users/c30732ceca204667b199e2e2fca633d5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:54.072034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c30732ceca204667b199e2e2fca633d5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:54.074381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:54.125351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c30732ceca204667b199e2e2fca633d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.126044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c30732ceca204667b199e2e2fca633d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.140682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c30732ceca204667b199e2e2fca633d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.141303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c30732ceca204667b199e2e2fca633d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.158594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a25209a-0cae-4760-890a-40b88ebb34bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c30732ceca204667b199e2e2fca633d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.175912 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2244/4490] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:53 2022] DELETE /identity/v3/users/c30732ceca204667b199e2e2fca633d5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:54.552841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:54.554596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:54.580764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53608e-082c-4701-a750-cf776e63357f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:54.651093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:54.651765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:54.652053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:54.652324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] PATH_INFO: `/v3/projects/ca2bef71179846efb45c1fb862f12594` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:54.653443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2bef71179846efb45c1fb862f12594)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:54.655513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:54.709104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2bef71179846efb45c1fb862f12594', 'request_id': 'req-faf1fddb-b4c9-4579-ab87-db3ad19cf427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.718896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2bef71179846efb45c1fb862f12594', 'request_id': 'req-faf1fddb-b4c9-4579-ab87-db3ad19cf427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.724855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2bef71179846efb45c1fb862f12594', 'request_id': 'req-faf1fddb-b4c9-4579-ab87-db3ad19cf427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.736315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] The token cache is being invalidate because project ca2bef71179846efb45c1fb862f12594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:54.736604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.737255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-faf1fddb-b4c9-4579-ab87-db3ad19cf427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:54.738784 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2247/4491] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:54 2022] DELETE /identity/v3/projects/ca2bef71179846efb45c1fb862f12594 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:54.743823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3762-b9ca-4c16-959c-83f730f5eec0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:54.744108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3762-b9ca-4c16-959c-83f730f5eec0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:54.744380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3762-b9ca-4c16-959c-83f730f5eec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:54.766799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e5aa3762-b9ca-4c16-959c-83f730f5eec0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:54.768295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e5aa3762-b9ca-4c16-959c-83f730f5eec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:54.844185 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2245/4492] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:54.853174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708476e-a6f6-4e6b-a478-3d67b9259621 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:54.853500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708476e-a6f6-4e6b-a478-3d67b9259621 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:54.853777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708476e-a6f6-4e6b-a478-3d67b9259621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:54.879748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9708476e-a6f6-4e6b-a478-3d67b9259621 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:54.880601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9708476e-a6f6-4e6b-a478-3d67b9259621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:54.890266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2248/4493] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:54.899314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa3762-b9ca-4c16-959c-83f730f5eec0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:54.911601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa3762-b9ca-4c16-959c-83f730f5eec0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:54.916553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:54.918494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:54.918494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:54.918494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:54.918809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:54.919043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:54.920780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:54.922551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:54.943632 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2246/4494] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:54.944766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708476e-a6f6-4e6b-a478-3d67b9259621 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:54.954723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708476e-a6f6-4e6b-a478-3d67b9259621 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:54.959265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:54.959872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:54.960148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:54.960417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:54.961067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:54.963023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:54.984801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:54.993737 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2249/4495] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:54 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:54.997512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.009488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2dafc-7281-44cf-8466-15d87ecf44cd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.072529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.073151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.073878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.073878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.074332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.076312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.113418 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2247/4496] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:54 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:55.126057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.137515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fafc2-a637-4b37-baa0-d4c5f00a6736 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.142767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.143379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.143656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.143920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.144561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.146323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.156847 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2250/4497] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:55.161106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.171690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc66dd6-64be-47b1-b0d5-4f324ed0cbdc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.176502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9766e5c-6d84-4f92-9571-b2d15bcfd229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.177243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766e5c-6d84-4f92-9571-b2d15bcfd229 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.177558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766e5c-6d84-4f92-9571-b2d15bcfd229 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.177827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9766e5c-6d84-4f92-9571-b2d15bcfd229 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.178469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9766e5c-6d84-4f92-9571-b2d15bcfd229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.180065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9766e5c-6d84-4f92-9571-b2d15bcfd229 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.191418 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2248/4498] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:55.195945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.206361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc395b86-f445-4800-af04-48fdcb7fdcf0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.212487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.213174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.213419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.213716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] PATH_INFO: `/v3/projects/4c965c4d170f4fff858e951b6798b2ea/users/0acd027181c74a5e825f37a9d565d17d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.229631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c965c4d170f4fff858e951b6798b2ea, user_id=0acd027181c74a5e825f37a9d565d17d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.232375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.249959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2251/4499] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:55 2022] PUT /identity/v3/projects/4c965c4d170f4fff858e951b6798b2ea/users/0acd027181c74a5e825f37a9d565d17d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:55.260966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3810d-0f47-4e67-b462-3d49526cbc57 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.261268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3810d-0f47-4e67-b462-3d49526cbc57 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.261660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3810d-0f47-4e67-b462-3d49526cbc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.291100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-abb3810d-0f47-4e67-b462-3d49526cbc57 None None] MFA Rules not processed for user `0acd027181c74a5e825f37a9d565d17d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:55.292394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-abb3810d-0f47-4e67-b462-3d49526cbc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:55.372210 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2249/4500] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:55.376948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.439780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac77a5e-f3a8-4cf5-baf7-d113a5bb7bbc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.444370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.444983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.445259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.445560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.446214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.448186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.475920 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2252/4501] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:55 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:55.480065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3810d-0f47-4e67-b462-3d49526cbc57 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.491403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3810d-0f47-4e67-b462-3d49526cbc57 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.554162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.554162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.554162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.554162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.554953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.557715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.579460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:55.596344 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2250/4502] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:55 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:55.597547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.609014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148e29e-8480-4e52-bf39-0e9914d7a609 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.615016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.615667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.615942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.616213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.616860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.618509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.630161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2253/4503] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:55.633759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.643854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8267-c5b4-4897-98ce-03263b735d42 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.648627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298651d8-2113-41b7-bf0f-e437da4fcd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.649226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298651d8-2113-41b7-bf0f-e437da4fcd8d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.649530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298651d8-2113-41b7-bf0f-e437da4fcd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.649802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298651d8-2113-41b7-bf0f-e437da4fcd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.650441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298651d8-2113-41b7-bf0f-e437da4fcd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.652024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298651d8-2113-41b7-bf0f-e437da4fcd8d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.667008 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2251/4504] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:55.668383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.679019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31924e10-db5a-4b9b-8cdb-a6c1f04caa47 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.683965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.684626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.684875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.685171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] PATH_INFO: `/v3/projects/d9772417722a4ed1ade68b5c2c67202e/users/4fc4923e2770406d9f9b6e402d78a961/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.706865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9772417722a4ed1ade68b5c2c67202e, user_id=4fc4923e2770406d9f9b6e402d78a961, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:55.709745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:55.723705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2254/4505] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:55 2022] PUT /identity/v3/projects/d9772417722a4ed1ade68b5c2c67202e/users/4fc4923e2770406d9f9b6e402d78a961/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:55.734860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16167-265e-4c97-80b1-454bcf52bf64 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.735135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16167-265e-4c97-80b1-454bcf52bf64 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.735409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16167-265e-4c97-80b1-454bcf52bf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.764541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a5e16167-265e-4c97-80b1-454bcf52bf64 None None] MFA Rules not processed for user `4fc4923e2770406d9f9b6e402d78a961`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:55.765787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a5e16167-265e-4c97-80b1-454bcf52bf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:55.835587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2252/4506] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:55.853995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:55.856404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:55.931980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81d8b5-106a-449b-af40-36b5a2610856 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:55.935839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:55.936474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:55.936745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:55.937013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:55.963943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:56.001503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:56.003075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:56.013254 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2255/4507] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:56.159686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:56.161217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:56.161799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:56.202248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:56.203451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:56.261206 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2253/4508] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:56.311661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:56.313199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:56.398705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf7d7a-0e89-4b97-ba80-8dc0eb80c923 service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:56.404872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0836c4b-c11a-4e02-962b-f3b51d39a67b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:56.405635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0836c4b-c11a-4e02-962b-f3b51d39a67b service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:56.405818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0836c4b-c11a-4e02-962b-f3b51d39a67b service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:56.406088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0836c4b-c11a-4e02-962b-f3b51d39a67b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:56.416849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0836c4b-c11a-4e02-962b-f3b51d39a67b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:56.419464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0836c4b-c11a-4e02-962b-f3b51d39a67b service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:56.429080 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2256/4509] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:06:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:06:56.606744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:56.608513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:56.622710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2469e-d059-45d1-81e0-ffd07494557b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:56.626434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:56.627146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:56.627420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:56.627808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:56.637369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:56.638912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:56.663798 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2254/4510] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:56.752917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394d35-406f-470a-8170-a9e1752537fd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:56.753427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394d35-406f-470a-8170-a9e1752537fd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:56.753645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394d35-406f-470a-8170-a9e1752537fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:56.773080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7d394d35-406f-470a-8170-a9e1752537fd None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:56.774308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7d394d35-406f-470a-8170-a9e1752537fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:56.781320 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2257/4511] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:56.794088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:56.806338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec51b17-3c8d-495f-8951-713656d7288e service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:56.811366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100beedf-6989-44a9-b2f8-c5c376ff409a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:56.811970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beedf-6989-44a9-b2f8-c5c376ff409a service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:56.812245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beedf-6989-44a9-b2f8-c5c376ff409a service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:56.812505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beedf-6989-44a9-b2f8-c5c376ff409a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:56.821155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100beedf-6989-44a9-b2f8-c5c376ff409a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:56.822771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100beedf-6989-44a9-b2f8-c5c376ff409a service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:56.831756 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2255/4512] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:06:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:06:56.961572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394d35-406f-470a-8170-a9e1752537fd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:57.029369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394d35-406f-470a-8170-a9e1752537fd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:57.034443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:57.035083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:57.035335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:57.035776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] PATH_INFO: `/v3/users/0acd027181c74a5e825f37a9d565d17d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:57.037135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0acd027181c74a5e825f37a9d565d17d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:57.039085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:57.092772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0acd027181c74a5e825f37a9d565d17d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.098258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:57.109439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0acd027181c74a5e825f37a9d565d17d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.118263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0acd027181c74a5e825f37a9d565d17d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.118949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0acd027181c74a5e825f37a9d565d17d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.119564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0acd027181c74a5e825f37a9d565d17d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.132500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2258/4513] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:56 2022] DELETE /identity/v3/users/0acd027181c74a5e825f37a9d565d17d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:57.136995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:57.137348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:57.137741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:57.157517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:57.158748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:57.193641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:57.219833 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2256/4514] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:57.229853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:57.231271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:57.311012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce368a51-b670-406e-8601-99ac659e1f04 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:57.315612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:57.316424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:57.316631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:57.316995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:57.328634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:57.331009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:57.341637 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2259/4515] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:57.713072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:57.715307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:57.731785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ccdd0-8d7c-4282-a497-0422790603c0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:57.738858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:57.739761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:57.740049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:57.740319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] PATH_INFO: `/v3/projects/4c965c4d170f4fff858e951b6798b2ea` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:57.741824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c965c4d170f4fff858e951b6798b2ea)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:57.743825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:57.810207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c965c4d170f4fff858e951b6798b2ea', 'request_id': 'req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.817009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c965c4d170f4fff858e951b6798b2ea', 'request_id': 'req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.828092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c965c4d170f4fff858e951b6798b2ea', 'request_id': 'req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.839420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] The token cache is being invalidate because project 4c965c4d170f4fff858e951b6798b2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:57.839711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.840372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:57.845135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2257/4516] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:57 2022] DELETE /identity/v3/projects/4c965c4d170f4fff858e951b6798b2ea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:57.846236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:57.859364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c2f3-0998-4acd-a59b-a2ec9ff40025 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:57.951436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:57.952482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:57.952859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:57.953239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] PATH_INFO: `/v3/users/4fc4923e2770406d9f9b6e402d78a961` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:57.954702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc4923e2770406d9f9b6e402d78a961)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:57.956751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.024546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc4923e2770406d9f9b6e402d78a961'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.039836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc4923e2770406d9f9b6e402d78a961'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.048125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc4923e2770406d9f9b6e402d78a961'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.048851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc4923e2770406d9f9b6e402d78a961'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.049445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b7427203-a8a1-48eb-a520-b097f8c93aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc4923e2770406d9f9b6e402d78a961'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.051597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:58.070964 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2260/4517] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:57 2022] DELETE /identity/v3/users/4fc4923e2770406d9f9b6e402d78a961 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:58.358071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:58.359741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:58.373843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bed16-547e-4c1a-beda-43d1d50ed1a1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:58.433680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:58.434233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.434504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.434797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] PATH_INFO: `/v3/projects/d9772417722a4ed1ade68b5c2c67202e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.436060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9772417722a4ed1ade68b5c2c67202e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:58.437994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.485207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9772417722a4ed1ade68b5c2c67202e', 'request_id': 'req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.491726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9772417722a4ed1ade68b5c2c67202e', 'request_id': 'req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.501127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9772417722a4ed1ade68b5c2c67202e', 'request_id': 'req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.505966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:58.514606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] The token cache is being invalidate because project d9772417722a4ed1ade68b5c2c67202e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:58.515062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.515942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a326c9c-e2f6-40eb-a27d-2af4c5b3082c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:58.517764 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2258/4518] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:58 2022] DELETE /identity/v3/projects/d9772417722a4ed1ade68b5c2c67202e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:58.523790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6464-6d5e-4465-9b6a-5d8c990de519 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.524091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6464-6d5e-4465-9b6a-5d8c990de519 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.524363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6464-6d5e-4465-9b6a-5d8c990de519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.544600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-995a6464-6d5e-4465-9b6a-5d8c990de519 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:58.545820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-995a6464-6d5e-4465-9b6a-5d8c990de519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:58.607785 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2261/4519] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:58.617427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba402c59-114f-4f38-8f44-073b32fcf6b4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.618265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba402c59-114f-4f38-8f44-073b32fcf6b4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.618698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba402c59-114f-4f38-8f44-073b32fcf6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.639981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ba402c59-114f-4f38-8f44-073b32fcf6b4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:58.641205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ba402c59-114f-4f38-8f44-073b32fcf6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:58.650415 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2259/4520] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:58.654843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a6464-6d5e-4465-9b6a-5d8c990de519 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:58.665137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a6464-6d5e-4465-9b6a-5d8c990de519 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:58.670495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:58.671157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.671427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.671688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.672335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:58.672631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:58.674411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.687214 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2262/4521] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:58.691905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba402c59-114f-4f38-8f44-073b32fcf6b4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:58.702305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba402c59-114f-4f38-8f44-073b32fcf6b4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:58.707803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:58.708443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.708723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.708996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.709698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:58.711627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.743383 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2260/4522] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:58.744626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:58.756110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddc018-60b0-4cb7-8d9a-eb3b0564e08f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:58.817405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:58.818058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.818342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.818610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.819252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:58.821220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.847358 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2263/4523] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:58 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:58.848578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:58.859319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8462a-cfcd-4694-a89c-fb23462e60d7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:58.866120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:58.866868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.867213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.867558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.868264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:58.870098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.881685 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2261/4524] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:58.885788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:58.899261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcde84-b078-4f39-a83d-2af48f16e055 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:58.904250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011a8b-8a04-4190-a924-5b141a90599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:58.904893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011a8b-8a04-4190-a924-5b141a90599f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.905169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011a8b-8a04-4190-a924-5b141a90599f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.905441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011a8b-8a04-4190-a924-5b141a90599f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.906133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93011a8b-8a04-4190-a924-5b141a90599f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:58.907811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93011a8b-8a04-4190-a924-5b141a90599f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.918959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2264/4525] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:58.924041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:58.935207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87def896-0a9f-49da-a9e2-7b1504d49e2d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:58.940515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:58.941184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.941559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.941893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] PATH_INFO: `/v3/projects/dc469090940849e1b5ec3711dd87cb98/users/f39ba5ea074b47ea9cc7ff996d1f1f50/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:58.959849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc469090940849e1b5ec3711dd87cb98, user_id=f39ba5ea074b47ea9cc7ff996d1f1f50, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:58.962550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:58.977843 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2262/4526] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:58 2022] PUT /identity/v3/projects/dc469090940849e1b5ec3711dd87cb98/users/f39ba5ea074b47ea9cc7ff996d1f1f50/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:58.989684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858aaec-add9-40f5-b8d9-83677ffdfbc5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:58.989956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858aaec-add9-40f5-b8d9-83677ffdfbc5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:58.990248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858aaec-add9-40f5-b8d9-83677ffdfbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:59.019376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a858aaec-add9-40f5-b8d9-83677ffdfbc5 None None] MFA Rules not processed for user `f39ba5ea074b47ea9cc7ff996d1f1f50`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:59.020602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a858aaec-add9-40f5-b8d9-83677ffdfbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:59.071280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:59.093211 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2265/4527] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:59.108679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:59.110564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:59.170225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524984a-60e4-4359-8f04-1910a80aadc1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:59.174115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:59.174725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:59.174997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:59.175264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:59.183844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:59.185271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:59.194371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2263/4528] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:59.360998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bca71-0ef9-4dbc-a776-50373a90d48e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:59.361337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bca71-0ef9-4dbc-a776-50373a90d48e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:59.361654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bca71-0ef9-4dbc-a776-50373a90d48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:59.382028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e49bca71-0ef9-4dbc-a776-50373a90d48e None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:59.383250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e49bca71-0ef9-4dbc-a776-50373a90d48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:59.453519 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2266/4529] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:59.471738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:59.473431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:59.566286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb632f7-f3fc-4fc2-9175-5e79163ae0e3 service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:59.571395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:59.572074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:59.572551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:59.572818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:59.584591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:59.586838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:59.595388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2264/4530] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:06:59.857956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5de29-368f-4ed0-bfcd-db0e85a2d7ca None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:59.858540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5de29-368f-4ed0-bfcd-db0e85a2d7ca None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:59.858540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5de29-368f-4ed0-bfcd-db0e85a2d7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:59.880868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-63c5de29-368f-4ed0-bfcd-db0e85a2d7ca None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:59.882122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-63c5de29-368f-4ed0-bfcd-db0e85a2d7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:59.889861 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2267/4531] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:59.902551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:59.913852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c566ed-7ddc-40c1-8e24-655e4b110571 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:59.932606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:59.942569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:59.942914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:59.943195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:59.956582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:59.959164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:59.977607 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2265/4532] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:00.271245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b1c1c-2efd-404d-a6fc-4b3aef25f1a4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.271735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b1c1c-2efd-404d-a6fc-4b3aef25f1a4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.272072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b1c1c-2efd-404d-a6fc-4b3aef25f1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.293006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e17b1c1c-2efd-404d-a6fc-4b3aef25f1a4 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:00.294327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e17b1c1c-2efd-404d-a6fc-4b3aef25f1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:00.305750 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2268/4533] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:00.332062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:00.348342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491dd27-f9fe-46e1-be25-259cebd5b9c1 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:00.354793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:00.355621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.356061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.356507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.366551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:00.368670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:00.370331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:00.390275 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2266/4534] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:07:00.673624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332eb95-6e33-41a6-82f6-fa018d83aa09 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.674088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332eb95-6e33-41a6-82f6-fa018d83aa09 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.674575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332eb95-6e33-41a6-82f6-fa018d83aa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.699507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2332eb95-6e33-41a6-82f6-fa018d83aa09 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:00.700719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2332eb95-6e33-41a6-82f6-fa018d83aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:00.708133 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2269/4535] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:00.720685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:00.731769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158f776-4ccc-42c3-9b4a-da9d5af933be service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:00.737130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:00.737778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.738058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.738329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.746902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:00.748568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:00.758369 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2267/4536] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:01.020995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466dca3-70eb-4143-95c8-eefecf4d7326 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.021487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466dca3-70eb-4143-95c8-eefecf4d7326 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.021701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466dca3-70eb-4143-95c8-eefecf4d7326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.042358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b466dca3-70eb-4143-95c8-eefecf4d7326 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:01.043618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b466dca3-70eb-4143-95c8-eefecf4d7326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:01.052362 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2270/4537] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:01.066454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.077916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671aef8-09dc-44ee-8f55-94acfc828a6d service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.085063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45440dad-4845-4864-be15-1815af0e496d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.085798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45440dad-4845-4864-be15-1815af0e496d service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.086138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45440dad-4845-4864-be15-1815af0e496d service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.086473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45440dad-4845-4864-be15-1815af0e496d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.098481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45440dad-4845-4864-be15-1815af0e496d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.100551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45440dad-4845-4864-be15-1815af0e496d service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.110306 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2268/4538] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:07:01.383357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98f99e-f93f-4f6d-9eb8-80f342f45b38 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.383839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98f99e-f93f-4f6d-9eb8-80f342f45b38 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.384267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98f99e-f93f-4f6d-9eb8-80f342f45b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.405263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ff98f99e-f93f-4f6d-9eb8-80f342f45b38 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:01.406526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ff98f99e-f93f-4f6d-9eb8-80f342f45b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:01.415453 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2271/4539] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:01.427561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45440dad-4845-4864-be15-1815af0e496d service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.438933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45440dad-4845-4864-be15-1815af0e496d service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.444220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.444950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.445261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.445569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.454270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.456031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.466295 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2269/4540] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:02.490297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb34fe7-c1b9-4f14-99e9-5a85fa4af144 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:02.490760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb34fe7-c1b9-4f14-99e9-5a85fa4af144 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:02.490901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb34fe7-c1b9-4f14-99e9-5a85fa4af144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:02.557206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ceb34fe7-c1b9-4f14-99e9-5a85fa4af144 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:02.558569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ceb34fe7-c1b9-4f14-99e9-5a85fa4af144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:02.614050 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2272/4541] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:02.629552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:02.643905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bfe72-6c74-41e9-a127-6b5a5711ce7e service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:02.649570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:02.650507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:02.650689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:02.650931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:02.661901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:02.663913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:02.684415 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2270/4542] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:07:02.880378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12059c2f-521d-4792-8ff8-1528194be1dd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:02.880656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12059c2f-521d-4792-8ff8-1528194be1dd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:02.881898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12059c2f-521d-4792-8ff8-1528194be1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:02.906010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-12059c2f-521d-4792-8ff8-1528194be1dd None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:02.907607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-12059c2f-521d-4792-8ff8-1528194be1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:02.916982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2273/4543] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:02.931379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:02.943921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e466dd-1bdc-4945-9bc4-1423a0b39972 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:02.950929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:02.952190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:02.952867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:02.953340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:02.963810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:02.966515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:02.988097 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2271/4544] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:03.152567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f3d5-2169-41ab-9cfd-511306b498f0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:03.153047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f3d5-2169-41ab-9cfd-511306b498f0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:03.153421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f3d5-2169-41ab-9cfd-511306b498f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:03.174523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6563f3d5-2169-41ab-9cfd-511306b498f0 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:03.175903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6563f3d5-2169-41ab-9cfd-511306b498f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:03.183814 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2274/4545] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:03.197671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:03.211119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68cd3b-9f47-414d-8306-1a94c725c5b7 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:03.224741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:03.225436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:03.225799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:03.226044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:03.239132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:03.241172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:03.258039 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2272/4546] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:03.457642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2ed2-8f87-410f-8b58-d33b9c007b70 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:03.457930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2ed2-8f87-410f-8b58-d33b9c007b70 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:03.458623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2ed2-8f87-410f-8b58-d33b9c007b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:03.478825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-108c2ed2-8f87-410f-8b58-d33b9c007b70 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:03.480162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-108c2ed2-8f87-410f-8b58-d33b9c007b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:03.487977 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2275/4547] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:03.500878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:03.513396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d494a-87a0-4c73-8659-509a9d4b3e3b service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:03.519006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:03.519645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:03.519924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:03.520195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:03.528484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:03.530127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:03.538649 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2273/4548] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:03.773748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d57818-c721-40b8-889f-2cbc07cb8e8b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:03.774660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d57818-c721-40b8-889f-2cbc07cb8e8b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:03.775165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d57818-c721-40b8-889f-2cbc07cb8e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:03.795813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e6d57818-c721-40b8-889f-2cbc07cb8e8b None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:03.797375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e6d57818-c721-40b8-889f-2cbc07cb8e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:03.805056 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2276/4549] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:03.824084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:03.839345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe03c1-8235-474c-a0c9-82323743362d service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:03.849886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:03.850557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:03.850932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:03.851320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:03.863172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:03.865394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:03.876038 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2274/4550] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:04.103920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e940f-1df4-4e16-b1f9-bb4ca91bbb3b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:04.104226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e940f-1df4-4e16-b1f9-bb4ca91bbb3b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:04.105543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e940f-1df4-4e16-b1f9-bb4ca91bbb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:04.133351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5e2e940f-1df4-4e16-b1f9-bb4ca91bbb3b None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:04.134742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e2e940f-1df4-4e16-b1f9-bb4ca91bbb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:04.153066 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2277/4551] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:07:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:04.268281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:04.268281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551e29d-741c-4b1f-8653-cdcd7b02ac86 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:04.268281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df365-4cee-4ccb-85ca-a308901b6d9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:04.268281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df365-4cee-4ccb-85ca-a308901b6d9d service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:04.269119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df365-4cee-4ccb-85ca-a308901b6d9d service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:04.269119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df365-4cee-4ccb-85ca-a308901b6d9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:04.269119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131df365-4cee-4ccb-85ca-a308901b6d9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:04.269119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131df365-4cee-4ccb-85ca-a308901b6d9d service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:04.269119 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2275/4552] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:04.472176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e940f-1df4-4e16-b1f9-bb4ca91bbb3b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:04.551911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e940f-1df4-4e16-b1f9-bb4ca91bbb3b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:04.560661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:04.561542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:04.561966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:04.562253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] PATH_INFO: `/v3/users/f39ba5ea074b47ea9cc7ff996d1f1f50` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:04.563497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39ba5ea074b47ea9cc7ff996d1f1f50)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:04.565854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:04.604483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:04.646629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39ba5ea074b47ea9cc7ff996d1f1f50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:04.660834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39ba5ea074b47ea9cc7ff996d1f1f50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:04.669817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39ba5ea074b47ea9cc7ff996d1f1f50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:04.670430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39ba5ea074b47ea9cc7ff996d1f1f50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:04.671095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39ba5ea074b47ea9cc7ff996d1f1f50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:04.688106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2278/4553] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:04 2022] DELETE /identity/v3/users/f39ba5ea074b47ea9cc7ff996d1f1f50 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:04.693272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:04.693868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:04.694232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:04.714104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:04.715338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:04.780189 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2276/4554] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:04.790765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:04.792308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:04.858821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2a771-2827-42f6-8555-7b081eea09b2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:04.863575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a18c39-73a2-44b1-93de-b4bbc84437af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:04.864577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a18c39-73a2-44b1-93de-b4bbc84437af service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:04.864646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a18c39-73a2-44b1-93de-b4bbc84437af service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:04.864976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a18c39-73a2-44b1-93de-b4bbc84437af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:04.891742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a18c39-73a2-44b1-93de-b4bbc84437af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:04.897534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a18c39-73a2-44b1-93de-b4bbc84437af service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:04.908674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2279/4555] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:05.303831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:05.306481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:05.320367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3858510-308f-428e-af66-b6aa9135f94d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:05.325883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:05.326728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:05.327146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:05.327555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] PATH_INFO: `/v3/projects/dc469090940849e1b5ec3711dd87cb98` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:05.329109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc469090940849e1b5ec3711dd87cb98)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:05.332809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:05.393022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:05.435135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc469090940849e1b5ec3711dd87cb98', 'request_id': 'req-cb610d65-9945-4cef-a7c1-846411cfba38'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:05.456908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc469090940849e1b5ec3711dd87cb98', 'request_id': 'req-cb610d65-9945-4cef-a7c1-846411cfba38'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:05.474815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc469090940849e1b5ec3711dd87cb98', 'request_id': 'req-cb610d65-9945-4cef-a7c1-846411cfba38'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:05.496280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] The token cache is being invalidate because project dc469090940849e1b5ec3711dd87cb98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:05.496676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:05.497412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb610d65-9945-4cef-a7c1-846411cfba38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:05.499274 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2277/4556] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:05 2022] DELETE /identity/v3/projects/dc469090940849e1b5ec3711dd87cb98 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:05.506143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231fc95-5d53-489b-ad2d-aeef00f2b52b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:05.506572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231fc95-5d53-489b-ad2d-aeef00f2b52b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:05.506946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231fc95-5d53-489b-ad2d-aeef00f2b52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:05.536852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1231fc95-5d53-489b-ad2d-aeef00f2b52b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:05.538268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1231fc95-5d53-489b-ad2d-aeef00f2b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:05.663608 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2280/4557] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:05.673259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d0c86-fc47-4a15-acbb-fca35d8c408b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:05.673570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d0c86-fc47-4a15-acbb-fca35d8c408b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:05.674068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d0c86-fc47-4a15-acbb-fca35d8c408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:05.716108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bf2d0c86-fc47-4a15-acbb-fca35d8c408b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:05.718113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bf2d0c86-fc47-4a15-acbb-fca35d8c408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:05.729112 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2278/4558] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:05.733910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231fc95-5d53-489b-ad2d-aeef00f2b52b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:05.750109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231fc95-5d53-489b-ad2d-aeef00f2b52b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:05.761998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:05.762633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:05.762910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:05.763228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:05.763843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:05.764448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:05.765865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:05.781011 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2281/4559] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:05.786355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d0c86-fc47-4a15-acbb-fca35d8c408b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:05.811497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d0c86-fc47-4a15-acbb-fca35d8c408b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:05.825064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:05.832260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:05.832549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:05.832828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:05.833534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:05.835525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:05.879667 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2279/4560] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:05 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:05.883837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:05.896998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731767d-5e9e-4e80-bcb4-924d4434516d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:05.935524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:06.036631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:06.036631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:06.036631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:06.036631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:06.037151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:06.040021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:06.082926 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2282/4561] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:05 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:06.088068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:06.102318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf53a10-ab31-435f-ba31-0fb7146d748e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:06.115233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:06.115866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:06.116142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:06.116415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:06.117068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:06.118720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:06.142373 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2280/4562] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:06.146063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:06.161562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36563def-e3a0-42b9-893e-0a27b452361a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:06.185530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46171c-542f-48bf-891d-b43fdccca66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:06.186333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46171c-542f-48bf-891d-b43fdccca66f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:06.186615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46171c-542f-48bf-891d-b43fdccca66f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:06.186887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46171c-542f-48bf-891d-b43fdccca66f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:06.187546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46171c-542f-48bf-891d-b43fdccca66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:06.189226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46171c-542f-48bf-891d-b43fdccca66f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:06.222890 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2283/4563] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:06.224975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:06.242525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe8c6f-37d9-4bf5-8ce0-ff7e913da67d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:06.247473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:06.248524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:06.248967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:06.249237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] PATH_INFO: `/v3/projects/2f30cf5d20404a3eb7c87bed3482220d/users/d4d43b9c25cf44b39bdda69485d50d9b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:06.268846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f30cf5d20404a3eb7c87bed3482220d, user_id=d4d43b9c25cf44b39bdda69485d50d9b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:06.272778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:06.306668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:06.306963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:06.308741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:06.309596 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2281/4564] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:06 2022] PUT /identity/v3/projects/2f30cf5d20404a3eb7c87bed3482220d/users/d4d43b9c25cf44b39bdda69485d50d9b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:06.369985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] MFA Rules not processed for user `d4d43b9c25cf44b39bdda69485d50d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:06.371234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:06.478948 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2284/4565] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:06.493103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:06.495237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:06.621375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14244079-f5d3-45fd-a0bf-f31f6e6c45a6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:06.644840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:06.645785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:06.646235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:06.646625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:06.669502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:06.671432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:06.682158 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2282/4566] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:06.813923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:06.815454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:06.886215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7ba91-8a59-49c2-839f-22d2c51dfe16 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:06.890673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:06.891310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:06.891586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:06.891931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] PATH_INFO: `/v3/users/30b19896bc4e4d14bfec46dd00519150` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:06.893267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30b19896bc4e4d14bfec46dd00519150)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:06.895294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:06.938005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:06.944469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30b19896bc4e4d14bfec46dd00519150'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:06.958004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30b19896bc4e4d14bfec46dd00519150'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:06.967031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30b19896bc4e4d14bfec46dd00519150'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:06.967653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30b19896bc4e4d14bfec46dd00519150'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:06.968238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30b19896bc4e4d14bfec46dd00519150'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:06.982135 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2285/4567] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:06 2022] DELETE /identity/v3/users/30b19896bc4e4d14bfec46dd00519150 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:07.083384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.094200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16845f-25dc-43e8-9155-c13312834d8e service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.152992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.153709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.153991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.154282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] PATH_INFO: `/v3/users/d4d43b9c25cf44b39bdda69485d50d9b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.155527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d43b9c25cf44b39bdda69485d50d9b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.157484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.159216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:07.162611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.164314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.180229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4bc0f-8245-4080-8ef6-fd27ce39c12a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.185504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.185892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.186168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.186441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] PATH_INFO: `/v3/projects/1c307636beed407582da5779796cc6a7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.187562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c307636beed407582da5779796cc6a7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.189592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.230288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d43b9c25cf44b39bdda69485d50d9b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.231075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d43b9c25cf44b39bdda69485d50d9b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.244105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d43b9c25cf44b39bdda69485d50d9b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.244539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c307636beed407582da5779796cc6a7', 'request_id': 'req-fde06b61-24bf-4fff-ac34-1fe332d6c88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.245585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d43b9c25cf44b39bdda69485d50d9b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.257530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c307636beed407582da5779796cc6a7', 'request_id': 'req-fde06b61-24bf-4fff-ac34-1fe332d6c88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.260826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-90645f85-9206-428a-b5a1-2add5eabd4f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d43b9c25cf44b39bdda69485d50d9b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.263870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c307636beed407582da5779796cc6a7', 'request_id': 'req-fde06b61-24bf-4fff-ac34-1fe332d6c88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.275944 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2283/4568] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:07 2022] DELETE /identity/v3/users/d4d43b9c25cf44b39bdda69485d50d9b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:07.279218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233407a4-282c-41e9-a2ac-e34b973169bd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.279218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233407a4-282c-41e9-a2ac-e34b973169bd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.279405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] The token cache is being invalidate because project 1c307636beed407582da5779796cc6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:07.279489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233407a4-282c-41e9-a2ac-e34b973169bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.279565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.279565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde06b61-24bf-4fff-ac34-1fe332d6c88e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:07.280554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2286/4569] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:07 2022] DELETE /identity/v3/projects/1c307636beed407582da5779796cc6a7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:07.282106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.282534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.283143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.298475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-233407a4-282c-41e9-a2ac-e34b973169bd None None] MFA Rules not processed for user `33784305478f46ae9fdc4d4b82ebc6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:07.299822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-233407a4-282c-41e9-a2ac-e34b973169bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.304701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:07.306144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.367722 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2284/4570] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:07.369008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419166-5f3a-4387-94d1-9e031a35b1e6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.369288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419166-5f3a-4387-94d1-9e031a35b1e6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.369592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419166-5f3a-4387-94d1-9e031a35b1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.376985 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2287/4571] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:07.387481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.388944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.401540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-af419166-5f3a-4387-94d1-9e031a35b1e6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:07.403095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af419166-5f3a-4387-94d1-9e031a35b1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.417748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2285/4572] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:07.424370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9928e5-5049-455b-b5e5-fc549bbc1fa9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.425395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9928e5-5049-455b-b5e5-fc549bbc1fa9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.425890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9928e5-5049-455b-b5e5-fc549bbc1fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.470525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:07.479374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6d9928e5-5049-455b-b5e5-fc549bbc1fa9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:07.481163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6d9928e5-5049-455b-b5e5-fc549bbc1fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.493245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27b883-4367-4dc2-bd1f-86463a41ad79 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.500016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.500916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.501509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.501981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.513602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.515123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.515813 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2286/4573] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:07.521000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9928e5-5049-455b-b5e5-fc549bbc1fa9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.532354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9928e5-5049-455b-b5e5-fc549bbc1fa9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.532555 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2288/4574] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:07:07.538249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.539405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.539405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.539790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.540674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:07.541090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.542904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.560564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2287/4575] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:07:07.562712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.572590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd29774-a0fc-4ccc-bef6-75bfc30f3373 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.581945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.582777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.583043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.583531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.584035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.586163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.625929 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2289/4576] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:07 2022] POST /identity/v3/projects => generated 395 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:07.630348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.645022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7060e1-07cf-49ad-92a2-8db3ecec2509 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.734166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:07.741494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.742276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.742894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.743429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.744079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.746262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.785535 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2288/4577] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:07 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:07:07.788556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.800266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.800819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d691e2-8ed1-4c72-a203-6eaa2798d40a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.802349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.806192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.806806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.807484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.807736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.808585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.810164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.825182 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2290/4578] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:07.827310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.842146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ed628-ed43-43eb-82b1-876ae810d0e5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.847647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.848728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.850237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.850349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.851026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.852948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.864271 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2291/4579] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:07.868144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:07.880712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195ff96-bd36-48ce-9ecf-faa2a3b5643f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.894937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaafe3c-97fd-4218-9ce8-147ebe1c02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.896321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaafe3c-97fd-4218-9ce8-147ebe1c02ac admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.897000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaafe3c-97fd-4218-9ce8-147ebe1c02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.897299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56101ffd-4bb6-4a33-beaf-5c6ed653c803 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:07.897636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaafe3c-97fd-4218-9ce8-147ebe1c02ac admin admin] PATH_INFO: `/v3/projects/d704de116c314d1e880d826915a43eae/users/02aa44fb4544430484ce6cf03e6d0aa3/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.903576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:07.904203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.905057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.905337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.907627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:07.929642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaafe3c-97fd-4218-9ce8-147ebe1c02ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d704de116c314d1e880d826915a43eae, user_id=02aa44fb4544430484ce6cf03e6d0aa3, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:07.932536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaafe3c-97fd-4218-9ce8-147ebe1c02ac admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:07.952780 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2292/4580] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:07 2022] PUT /identity/v3/projects/d704de116c314d1e880d826915a43eae/users/02aa44fb4544430484ce6cf03e6d0aa3/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:07.956322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:07.957014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:07.957291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:07.994701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:07.996611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:08.034618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:08.036594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:08.048997 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2289/4581] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:07:08.097437 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2293/4582] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:08.114833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:08.117021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:08.181371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:08.183188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:08.217746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003b73d-3a64-4c07-99f2-3040c5741278 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:08.232001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:08.232606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:08.232964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:08.233334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] PATH_INFO: `/v3/projects/2f30cf5d20404a3eb7c87bed3482220d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:08.233793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce16e0-6b6a-46a9-a284-5634384bce3b service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:08.234726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f30cf5d20404a3eb7c87bed3482220d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:08.237659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:08.306016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f30cf5d20404a3eb7c87bed3482220d', 'request_id': 'req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:08.322274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:08.322768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f30cf5d20404a3eb7c87bed3482220d', 'request_id': 'req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:08.332673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f30cf5d20404a3eb7c87bed3482220d', 'request_id': 'req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:08.341981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:08.347649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9bcb0-d4d0-4817-b067-bef11008196e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:08.348352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9bcb0-d4d0-4817-b067-bef11008196e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:08.348922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9bcb0-d4d0-4817-b067-bef11008196e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:08.349199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9bcb0-d4d0-4817-b067-bef11008196e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:08.350302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] The token cache is being invalidate because project 2f30cf5d20404a3eb7c87bed3482220d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:08.350670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:08.352074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79e9856a-44ab-4e8e-a8b8-4c2f9b23827c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:08.355030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2294/4583] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:08 2022] DELETE /identity/v3/projects/2f30cf5d20404a3eb7c87bed3482220d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:08.362233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc677c2-27e4-49f1-bd19-36b38199abf0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:08.362879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc677c2-27e4-49f1-bd19-36b38199abf0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:08.363441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc677c2-27e4-49f1-bd19-36b38199abf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:08.364790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9bcb0-d4d0-4817-b067-bef11008196e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:08.366821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9bcb0-d4d0-4817-b067-bef11008196e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:08.383557 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2290/4584] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:08.393684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ddc677c2-27e4-49f1-bd19-36b38199abf0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:08.395230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ddc677c2-27e4-49f1-bd19-36b38199abf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:08.555809 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2295/4585] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:08.556670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140b152-b9ba-42d5-9b31-aacdc01a3ca2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:08.556979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140b152-b9ba-42d5-9b31-aacdc01a3ca2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:08.557253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140b152-b9ba-42d5-9b31-aacdc01a3ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:08.597315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a140b152-b9ba-42d5-9b31-aacdc01a3ca2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:08.599420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a140b152-b9ba-42d5-9b31-aacdc01a3ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:08.627059 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2291/4586] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:08.632879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc677c2-27e4-49f1-bd19-36b38199abf0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:08.649095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc677c2-27e4-49f1-bd19-36b38199abf0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:08.664436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:08.665074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:08.665350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:08.665721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:08.666417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:08.667576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:08.669184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:08.688138 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2296/4587] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:08.692033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140b152-b9ba-42d5-9b31-aacdc01a3ca2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:08.709605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140b152-b9ba-42d5-9b31-aacdc01a3ca2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:08.721824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:08.722706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:08.730338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:08.730913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:08.733007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:08.736399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:08.797893 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2292/4588] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:08 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:08.799261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:08.812966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f49c2-7d2b-4778-bc5e-7d855c3e83ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:08.947203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:08.948043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:08.948688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:08.949050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:08.950041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:08.952233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:08.985842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2297/4589] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:08 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:07:08.990278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:09.003085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012ca16-2a67-48e8-a2c6-892212f882ec admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:09.017612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:09.018329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:09.018673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:09.019093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:09.020077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:09.022879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:09.038497 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2293/4590] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:09.043897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:09.057313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1ca55-478b-460e-b199-30b45a65f651 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:09.064174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36141a00-7524-4347-aa41-ee9a85950717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:09.064937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36141a00-7524-4347-aa41-ee9a85950717 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:09.065221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36141a00-7524-4347-aa41-ee9a85950717 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:09.065514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36141a00-7524-4347-aa41-ee9a85950717 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:09.066380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36141a00-7524-4347-aa41-ee9a85950717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:09.068283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36141a00-7524-4347-aa41-ee9a85950717 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:09.087889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:09.099520 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2298/4591] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:09.104646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf2a58-a6d6-44e2-a7b8-f229def008c5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:09.111128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde21df-c59b-407c-9112-1bc42eadb5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:09.111983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde21df-c59b-407c-9112-1bc42eadb5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:09.112268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde21df-c59b-407c-9112-1bc42eadb5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:09.112543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde21df-c59b-407c-9112-1bc42eadb5fd admin admin] PATH_INFO: `/v3/projects/849f803a6b93482a864d620bfd89b997/users/debac56df1e540c890c8147195f338f6/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:09.135560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde21df-c59b-407c-9112-1bc42eadb5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=849f803a6b93482a864d620bfd89b997, user_id=debac56df1e540c890c8147195f338f6, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:09.139325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde21df-c59b-407c-9112-1bc42eadb5fd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:09.173568 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2294/4592] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:09 2022] PUT /identity/v3/projects/849f803a6b93482a864d620bfd89b997/users/debac56df1e540c890c8147195f338f6/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:09.269140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2a3c-471a-4a9b-b249-078dd16a0007 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:09.269420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2a3c-471a-4a9b-b249-078dd16a0007 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:09.270202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2a3c-471a-4a9b-b249-078dd16a0007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:09.307334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08600d2-b807-494a-bc1e-584358c76d80 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:09.307804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08600d2-b807-494a-bc1e-584358c76d80 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:09.307921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08600d2-b807-494a-bc1e-584358c76d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:09.336262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6e9e2a3c-471a-4a9b-b249-078dd16a0007 None None] MFA Rules not processed for user `debac56df1e540c890c8147195f338f6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:09.337929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6e9e2a3c-471a-4a9b-b249-078dd16a0007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:09.346620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a08600d2-b807-494a-bc1e-584358c76d80 None None] MFA Rules not processed for user `599c7072545f4280988f6bc50e3ccc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:09.347898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a08600d2-b807-494a-bc1e-584358c76d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:09.428215 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2295/4593] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:09.470724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:09.514815 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2299/4594] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:09.518885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08600d2-b807-494a-bc1e-584358c76d80 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:09.683367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08600d2-b807-494a-bc1e-584358c76d80 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:09.705672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:09.706988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:09.707480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:09.707979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:09.710048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:09.713624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:09.769560 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2296/4595] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:09 2022] POST /identity/v3/projects => generated 375 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:09.772371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2a3c-471a-4a9b-b249-078dd16a0007 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:09.790731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2a3c-471a-4a9b-b249-078dd16a0007 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:09.923044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83651b87-9ea0-48c2-bd77-013490ac63e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:09.924009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83651b87-9ea0-48c2-bd77-013490ac63e6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:09.924694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83651b87-9ea0-48c2-bd77-013490ac63e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:09.925078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83651b87-9ea0-48c2-bd77-013490ac63e6 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:09.932002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83651b87-9ea0-48c2-bd77-013490ac63e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:09.943403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83651b87-9ea0-48c2-bd77-013490ac63e6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:09.991593 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2300/4596] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:09 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:09.992217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:09.998870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:10.024402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6f3c7-5dfd-4243-86ff-c3501fef0c5c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:10.030355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:10.031238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.031606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.032056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.033207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:10.034535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb492c-35f0-45ea-8bb6-9789cc75cbd7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.034835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb492c-35f0-45ea-8bb6-9789cc75cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.035006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:10.035113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb492c-35f0-45ea-8bb6-9789cc75cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.051000 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2297/4597] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:07:10.056947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:10.076010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c8cb492c-35f0-45ea-8bb6-9789cc75cbd7 None None] MFA Rules not processed for user `24c7a82126124ebb8baf5c656ff9849d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:10.077486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c8cb492c-35f0-45ea-8bb6-9789cc75cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:10.078319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe19cdd-e5cc-45b0-b526-6ab9ae5173f1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:10.088532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:10.090147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.090542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.090972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] PATH_INFO: `/v3/projects/315bf9eead2b4a06a173de4472b9b08f/users/b87bfe2e5b5c421191570a7db9e6da66/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.120171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] RBAC: Authorizing `identity:create_grant(project_id=315bf9eead2b4a06a173de4472b9b08f, user_id=b87bfe2e5b5c421191570a7db9e6da66, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:10.123954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:10.142933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2298/4598] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:10 2022] PUT /identity/v3/projects/315bf9eead2b4a06a173de4472b9b08f/users/b87bfe2e5b5c421191570a7db9e6da66/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:10.146637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:10.227019 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2301/4599] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:10.335129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c8df5-f9e6-4a4a-a6c7-52f08fcb334e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:10.346851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6bb2b-76df-4a28-98a0-29e6eab62900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:10.347531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bb2b-76df-4a28-98a0-29e6eab62900 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.347848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bb2b-76df-4a28-98a0-29e6eab62900 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.348119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bb2b-76df-4a28-98a0-29e6eab62900 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.351142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6bb2b-76df-4a28-98a0-29e6eab62900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:10.352789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6bb2b-76df-4a28-98a0-29e6eab62900 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:10.372972 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2299/4600] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:10.379266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb492c-35f0-45ea-8bb6-9789cc75cbd7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:10.396927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb492c-35f0-45ea-8bb6-9789cc75cbd7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:10.413366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:10.414767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.415124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.415401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] PATH_INFO: `/v3/projects/315bf9eead2b4a06a173de4472b9b08f/users/b87bfe2e5b5c421191570a7db9e6da66/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.418127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315bf9eead2b4a06a173de4472b9b08f, user_id=b87bfe2e5b5c421191570a7db9e6da66, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:10.423513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:10.457928 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2302/4601] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:10 2022] PUT /identity/v3/projects/315bf9eead2b4a06a173de4472b9b08f/users/b87bfe2e5b5c421191570a7db9e6da66/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:10.470997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60443b2-0f2b-4179-9175-d2a5367b927d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.471538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60443b2-0f2b-4179-9175-d2a5367b927d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.471613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60443b2-0f2b-4179-9175-d2a5367b927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.534659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a60443b2-0f2b-4179-9175-d2a5367b927d None None] MFA Rules not processed for user `b87bfe2e5b5c421191570a7db9e6da66`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:10.535992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a60443b2-0f2b-4179-9175-d2a5367b927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:10.693428 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2300/4602] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:10.778117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:10.781963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:10.939862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60443b2-0f2b-4179-9175-d2a5367b927d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:10.951869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60443b2-0f2b-4179-9175-d2a5367b927d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:10.954459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46353c8-82c6-44f4-baaa-287a55c19df8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:10.957269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:10.957932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.958208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.958475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.960772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:10.967151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6458a35-8c08-4fbe-a726-9ef238090107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:10.967885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6458a35-8c08-4fbe-a726-9ef238090107 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:10.968078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6458a35-8c08-4fbe-a726-9ef238090107 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:10.968343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6458a35-8c08-4fbe-a726-9ef238090107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:10.992153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:10.993679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:11.014876 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2301/4603] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:11.017328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6458a35-8c08-4fbe-a726-9ef238090107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:11.018879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6458a35-8c08-4fbe-a726-9ef238090107 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:11.029277 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2303/4604] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:14.652979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:14.728752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:14.765522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c5072-7a68-402f-86f9-06ea06846659 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:14.775150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:14.776084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:14.776541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:14.776993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] PATH_INFO: `/v3/users/debac56df1e540c890c8147195f338f6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:14.779698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=debac56df1e540c890c8147195f338f6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:14.783261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:14.864116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debac56df1e540c890c8147195f338f6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:14.864676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debac56df1e540c890c8147195f338f6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:14.877766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debac56df1e540c890c8147195f338f6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:14.878319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debac56df1e540c890c8147195f338f6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:14.896550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debac56df1e540c890c8147195f338f6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:14.917917 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2302/4605] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:14 2022] DELETE /identity/v3/users/debac56df1e540c890c8147195f338f6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:14.919290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:14.919778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:14.919866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:14.942052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:14.943465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:14.958591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:15.024430 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2304/4606] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:15.047622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:15.050472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:15.151563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc045e-f48c-47f7-b86a-0cfa56b1f0c3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:15.166131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:15.166781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:15.167064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:15.167327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:15.197157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:15.199040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:15.210759 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2303/4607] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:16.088116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:16.090266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:16.108872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750d6f-b9ed-420a-9ff5-4e184889e8ee None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:16.119117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:16.120124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:16.120542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:16.120971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] PATH_INFO: `/v3/projects/849f803a6b93482a864d620bfd89b997` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:16.122470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849f803a6b93482a864d620bfd89b997)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:16.124639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:16.202942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849f803a6b93482a864d620bfd89b997', 'request_id': 'req-56cd3a39-97fb-438f-8761-6b457fafd878'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.226338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849f803a6b93482a864d620bfd89b997', 'request_id': 'req-56cd3a39-97fb-438f-8761-6b457fafd878'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.239225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849f803a6b93482a864d620bfd89b997', 'request_id': 'req-56cd3a39-97fb-438f-8761-6b457fafd878'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.255001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] The token cache is being invalidate because project 849f803a6b93482a864d620bfd89b997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:16.255716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.257272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.260480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2305/4608] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:16 2022] DELETE /identity/v3/projects/849f803a6b93482a864d620bfd89b997 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:16.265090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:16.279604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d23c3f-e225-4543-be10-b3cbbc66a7bf service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:16.400647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:16.402773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:16.403578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:16.406300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] PATH_INFO: `/v3/users/b87bfe2e5b5c421191570a7db9e6da66` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:16.409359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87bfe2e5b5c421191570a7db9e6da66)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:16.414763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:16.417396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:16.513201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87bfe2e5b5c421191570a7db9e6da66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.514383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87bfe2e5b5c421191570a7db9e6da66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.543610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87bfe2e5b5c421191570a7db9e6da66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.544491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87bfe2e5b5c421191570a7db9e6da66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.571349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516c7e2a-ba1d-4366-90ad-5ab1bf9c4313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87bfe2e5b5c421191570a7db9e6da66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:16.601673 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2304/4609] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:16 2022] DELETE /identity/v3/users/b87bfe2e5b5c421191570a7db9e6da66 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:18.084163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:18.087391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:18.144497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd3a39-97fb-438f-8761-6b457fafd878 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:18.247076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:18.247925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.248416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.248886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] PATH_INFO: `/v3/projects/315bf9eead2b4a06a173de4472b9b08f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.250616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=315bf9eead2b4a06a173de4472b9b08f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:18.254391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:18.313910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:18.322917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315bf9eead2b4a06a173de4472b9b08f', 'request_id': 'req-65d25411-e049-433c-aeda-39eff9beb59c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:18.342486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315bf9eead2b4a06a173de4472b9b08f', 'request_id': 'req-65d25411-e049-433c-aeda-39eff9beb59c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:18.349089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315bf9eead2b4a06a173de4472b9b08f', 'request_id': 'req-65d25411-e049-433c-aeda-39eff9beb59c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:18.363511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] The token cache is being invalidate because project 315bf9eead2b4a06a173de4472b9b08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:18.364046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:18.364911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65d25411-e049-433c-aeda-39eff9beb59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:18.366998 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2306/4610] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:18 2022] DELETE /identity/v3/projects/315bf9eead2b4a06a173de4472b9b08f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:18.372872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4632ad5-a5a1-43d8-9b35-872930faa176 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.373407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4632ad5-a5a1-43d8-9b35-872930faa176 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.373407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4632ad5-a5a1-43d8-9b35-872930faa176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.421718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e4632ad5-a5a1-43d8-9b35-872930faa176 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:18.423164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e4632ad5-a5a1-43d8-9b35-872930faa176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:18.502678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2305/4611] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:18.511034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787e0b-89f1-4861-91a8-365b55ac0113 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.511352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787e0b-89f1-4861-91a8-365b55ac0113 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.511618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3787e0b-89f1-4861-91a8-365b55ac0113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.547817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b3787e0b-89f1-4861-91a8-365b55ac0113 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:18.549092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b3787e0b-89f1-4861-91a8-365b55ac0113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:18.559074 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2307/4612] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:18.563725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4632ad5-a5a1-43d8-9b35-872930faa176 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:18.579786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4632ad5-a5a1-43d8-9b35-872930faa176 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:18.586057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:18.587456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.587700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.597453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.597656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:18.597928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:18.598187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7f761-4f91-47b2-8157-a5028d5344c6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.598469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7f761-4f91-47b2-8157-a5028d5344c6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.598737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7f761-4f91-47b2-8157-a5028d5344c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.600445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:18.617615 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2306/4613] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:18.621348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:18.630070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-94d7f761-4f91-47b2-8157-a5028d5344c6 None None] MFA Rules not processed for user `795813bb461e42ad8c2a758e9b8430d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:18.632105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-94d7f761-4f91-47b2-8157-a5028d5344c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:18.643603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2ce78-110e-4678-bdaf-46604fe731bd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:18.649171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:18.649896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.650174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.650444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.651951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:18.653009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:18.712048 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2307/4614] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:18 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:18.716477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:18.745280 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2308/4615] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:18.778443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:18.801084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147f7c6-be33-4748-b634-8e58880f63b0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:18.815768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:18.816538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.816882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.817219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.817997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:18.821547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:18.863218 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2308/4616] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:18 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:18.867284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7f761-4f91-47b2-8157-a5028d5344c6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:18.886541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7f761-4f91-47b2-8157-a5028d5344c6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:18.892502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:18.893102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.893598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.893831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.894557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:18.897126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:18.912181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2309/4617] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:18.913654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:18.927580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9f5b3-71cb-4d4b-901d-31998e1b214a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:18.935070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd751f27-6384-4d16-8fc9-7d18f435ca6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:18.935794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751f27-6384-4d16-8fc9-7d18f435ca6a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.936140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751f27-6384-4d16-8fc9-7d18f435ca6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.936480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751f27-6384-4d16-8fc9-7d18f435ca6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.937223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd751f27-6384-4d16-8fc9-7d18f435ca6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:18.939068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd751f27-6384-4d16-8fc9-7d18f435ca6a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:18.953042 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2309/4618] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:07:18.956642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:18.967559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c57389-65fc-4920-9aaa-16b5ed77dd12 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:18.972584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:18.973357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:18.973760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:18.974119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] PATH_INFO: `/v3/projects/7f3a00831a7f4bd392cd7531454cec9c/users/b46cc35a8db44ded8650ef3573d9a977/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:18.991924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3a00831a7f4bd392cd7531454cec9c, user_id=b46cc35a8db44ded8650ef3573d9a977, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:18.994884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:19.012868 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2310/4619] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:18 2022] PUT /identity/v3/projects/7f3a00831a7f4bd392cd7531454cec9c/users/b46cc35a8db44ded8650ef3573d9a977/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:19.026361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20f78-8992-4659-96f4-1cacff00d75f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.026711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20f78-8992-4659-96f4-1cacff00d75f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.027800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20f78-8992-4659-96f4-1cacff00d75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.060261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d8a20f78-8992-4659-96f4-1cacff00d75f None None] MFA Rules not processed for user `b46cc35a8db44ded8650ef3573d9a977`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:19.061841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8a20f78-8992-4659-96f4-1cacff00d75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:19.138605 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2310/4620] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:19.143140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:19.236944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a43f9-2f04-4a65-9f81-7f78cee80edb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:19.242688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:19.243477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.243752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.244018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.244816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:19.247628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:19.292030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2311/4621] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:19 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:19.296299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a20f78-8992-4659-96f4-1cacff00d75f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:19.312600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a20f78-8992-4659-96f4-1cacff00d75f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:19.473803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:19.475240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.475821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.476284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.477137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:19.479330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:19.512253 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2311/4622] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:19 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:19.517902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:19.533157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6b308-b3c3-4278-afd6-af28d2917717 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:19.549518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:19.550160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.550436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.550709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.551365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:19.553003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:19.587822 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2312/4623] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:19.592821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:19.610787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fa5cd-b374-461d-97ac-f60a7d452fa2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:19.643284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:19.643980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.644250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.644519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] PATH_INFO: `/v3/projects/69a12224f5c642059d1ec86f99944f37/users/421a1653bf6f460abce6396b3be29e7d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.697608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a12224f5c642059d1ec86f99944f37, user_id=421a1653bf6f460abce6396b3be29e7d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:19.701824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:19.731565 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2312/4624] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:19 2022] PUT /identity/v3/projects/69a12224f5c642059d1ec86f99944f37/users/421a1653bf6f460abce6396b3be29e7d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:19.732623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:19.748943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2338bb-0116-42a5-893f-cf5b9ed02e6a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:19.762552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:19.881429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9e3b9-c2e5-416d-8ae8-03bd16821475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:19.882007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9e3b9-c2e5-416d-8ae8-03bd16821475 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.882327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9e3b9-c2e5-416d-8ae8-03bd16821475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.882981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9e3b9-c2e5-416d-8ae8-03bd16821475 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.883248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9e3b9-c2e5-416d-8ae8-03bd16821475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:19.884934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9e3b9-c2e5-416d-8ae8-03bd16821475 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:19.898525 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2313/4625] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:07:19.899913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:19.913085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633d723-2cd8-4d92-bd17-ad4bf054689f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:19.921251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:19.922401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.923706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.924168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] PATH_INFO: `/v3/projects/69a12224f5c642059d1ec86f99944f37/users/421a1653bf6f460abce6396b3be29e7d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.926699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a12224f5c642059d1ec86f99944f37, user_id=421a1653bf6f460abce6396b3be29e7d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:19.930373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:19.943097 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2313/4626] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:19 2022] PUT /identity/v3/projects/69a12224f5c642059d1ec86f99944f37/users/421a1653bf6f460abce6396b3be29e7d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:19.954780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7e85-f6d7-4cc5-bb88-f4edd2e0a74c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:19.955477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7e85-f6d7-4cc5-bb88-f4edd2e0a74c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:19.955922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7e85-f6d7-4cc5-bb88-f4edd2e0a74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:19.992598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5fdf7e85-f6d7-4cc5-bb88-f4edd2e0a74c None None] MFA Rules not processed for user `421a1653bf6f460abce6396b3be29e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:19.993922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5fdf7e85-f6d7-4cc5-bb88-f4edd2e0a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:20.106496 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2314/4627] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:20.118199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:20.120405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:20.131043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:20.196907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b86c-e300-4ce5-9a88-aca319591ffe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:20.201925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:20.202476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:20.202820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:20.203438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:20.213388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:20.215009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:20.228683 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2314/4628] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:07:20.986701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:20.987124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:20.988127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:21.029147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] MFA Rules not processed for user `02aa44fb4544430484ce6cf03e6d0aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:21.031014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:21.131271 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2315/4629] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:21.185359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:21.187875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:21.294169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2a87-a468-42ae-8102-32851d31a45e service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:21.300048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1cd6-2f53-4223-9879-d5536e73d66d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:21.300670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1cd6-2f53-4223-9879-d5536e73d66d service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:21.300957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1cd6-2f53-4223-9879-d5536e73d66d service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:21.301210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1cd6-2f53-4223-9879-d5536e73d66d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:21.313650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1cd6-2f53-4223-9879-d5536e73d66d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:21.315807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1cd6-2f53-4223-9879-d5536e73d66d service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:21.334239 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2315/4630] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:21 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:23.283510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:23.285696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:23.305153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d6207-e476-4bf4-957d-c0408e577b23 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:23.312468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:23.314158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:23.314874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:23.315358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:23.359715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:23.430621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:23.432135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:23.465674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2316/4631] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:25.132116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa802f-4f43-4f3f-a7f7-57a410ca6397 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:25.132710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa802f-4f43-4f3f-a7f7-57a410ca6397 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:25.132710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa802f-4f43-4f3f-a7f7-57a410ca6397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:25.163572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d5fa802f-4f43-4f3f-a7f7-57a410ca6397 None None] MFA Rules not processed for user `c544e8e6f7a741c985c92ed7437397cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:25.165015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d5fa802f-4f43-4f3f-a7f7-57a410ca6397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:25.283789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2316/4632] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:25.902727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:26.039528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e30d7a-6060-49c3-b6ac-14af4d96d295 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:26.044515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:26.045399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:26.045881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:26.046297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] PATH_INFO: `/v3/users/b46cc35a8db44ded8650ef3573d9a977` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:26.047674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b46cc35a8db44ded8650ef3573d9a977)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:26.050006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:26.120589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b46cc35a8db44ded8650ef3573d9a977'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.134611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b46cc35a8db44ded8650ef3573d9a977'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.144279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b46cc35a8db44ded8650ef3573d9a977'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.146742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b46cc35a8db44ded8650ef3573d9a977'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.147099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b46cc35a8db44ded8650ef3573d9a977'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.176985 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2317/4633] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:25 2022] DELETE /identity/v3/users/b46cc35a8db44ded8650ef3573d9a977 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:26.181695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:26.182264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:26.183445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:26.211077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:26.213555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:26.249774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:26.279944 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2317/4634] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:26.289963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:26.291375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:26.335205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:26.353067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b000a4-84f7-4c22-9193-c5b0815dd18e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:26.358080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:26.358609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:26.359289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:26.359526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:26.372935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:26.375005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:26.389378 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2318/4635] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:26.694971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:26.696755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:26.715233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcff5b-4083-49b1-8298-49bac6738914 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:26.721362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:26.723113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:26.723398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:26.723672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] PATH_INFO: `/v3/projects/7f3a00831a7f4bd392cd7531454cec9c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:26.724868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3a00831a7f4bd392cd7531454cec9c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:26.726853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:26.779932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3a00831a7f4bd392cd7531454cec9c', 'request_id': 'req-d41a52f7-6448-4a7b-a5ae-a1e03a973586'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.786686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3a00831a7f4bd392cd7531454cec9c', 'request_id': 'req-d41a52f7-6448-4a7b-a5ae-a1e03a973586'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.796032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3a00831a7f4bd392cd7531454cec9c', 'request_id': 'req-d41a52f7-6448-4a7b-a5ae-a1e03a973586'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.807950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] The token cache is being invalidate because project 7f3a00831a7f4bd392cd7531454cec9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:26.808288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.808895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.810459 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2318/4636] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:26 2022] DELETE /identity/v3/projects/7f3a00831a7f4bd392cd7531454cec9c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:26.814655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:26.824614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027c6d8-e534-4640-82a4-a128d3e1fac5 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:26.886460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:26.887336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:26.887529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:26.887691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] PATH_INFO: `/v3/users/421a1653bf6f460abce6396b3be29e7d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:26.889514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=421a1653bf6f460abce6396b3be29e7d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:26.890809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:26.943165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '421a1653bf6f460abce6396b3be29e7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.955721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '421a1653bf6f460abce6396b3be29e7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.964836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '421a1653bf6f460abce6396b3be29e7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.965667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '421a1653bf6f460abce6396b3be29e7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.966291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92d7d251-5c3f-4da2-aca0-7019421f4c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '421a1653bf6f460abce6396b3be29e7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:26.979608 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2319/4637] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:26 2022] DELETE /identity/v3/users/421a1653bf6f460abce6396b3be29e7d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:27.304141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:27.307790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:27.348671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a52f7-6448-4a7b-a5ae-a1e03a973586 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:27.412661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:27.413687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:27.413963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:27.414226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] PATH_INFO: `/v3/projects/69a12224f5c642059d1ec86f99944f37` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:27.415408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a12224f5c642059d1ec86f99944f37)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:27.418005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:27.474111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a12224f5c642059d1ec86f99944f37', 'request_id': 'req-5b2d0511-3430-42df-bd76-407973435918'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:27.481621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a12224f5c642059d1ec86f99944f37', 'request_id': 'req-5b2d0511-3430-42df-bd76-407973435918'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:27.495023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a12224f5c642059d1ec86f99944f37', 'request_id': 'req-5b2d0511-3430-42df-bd76-407973435918'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:27.499069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:27.525352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] The token cache is being invalidate because project 69a12224f5c642059d1ec86f99944f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:27.526149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:27.526949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5b2d0511-3430-42df-bd76-407973435918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:27.529392 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2319/4638] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:27 2022] DELETE /identity/v3/projects/69a12224f5c642059d1ec86f99944f37 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:27.534509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37927f-4a3f-4c55-b401-0526aa7838c7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:27.534798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37927f-4a3f-4c55-b401-0526aa7838c7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:27.535070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37927f-4a3f-4c55-b401-0526aa7838c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:27.564516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-df37927f-4a3f-4c55-b401-0526aa7838c7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:27.565930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-df37927f-4a3f-4c55-b401-0526aa7838c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:27.654219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:27.683279 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2320/4639] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:27.686065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a567ea7-377f-4fd7-8b84-5653aa2952dc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:27.686353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a567ea7-377f-4fd7-8b84-5653aa2952dc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:27.686627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a567ea7-377f-4fd7-8b84-5653aa2952dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:27.734898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4a567ea7-377f-4fd7-8b84-5653aa2952dc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:27.738542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a567ea7-377f-4fd7-8b84-5653aa2952dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:27.757216 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2320/4640] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:27.757686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37927f-4a3f-4c55-b401-0526aa7838c7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:27.774278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37927f-4a3f-4c55-b401-0526aa7838c7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:27.793865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:27.794680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:27.795056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:27.795421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:27.796295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:27.796707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:27.813932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:27.861551 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2321/4641] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:27.863636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a567ea7-377f-4fd7-8b84-5653aa2952dc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:27.877343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a567ea7-377f-4fd7-8b84-5653aa2952dc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:27.885868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:27.886734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:27.887106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:27.887989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:27.888816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:27.890868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:27.968797 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2321/4642] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:27 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:27.970129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:27.989099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03888d5f-de70-4dea-880a-4aabcfccfce6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.073959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.075575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.075575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.075812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.076996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.080941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.111757 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2322/4643] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:27 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:28.116668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.132789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eda01-05d8-408d-92ef-9ef3d91abe44 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.142991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.143821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.144105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.144457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.145128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.147178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.163625 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2322/4644] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:07:28.168145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.182306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b175e-5049-4934-82dc-bc7675519a6f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.189108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79a8d9-c84c-4798-9c88-bf946f00ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.190329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a8d9-c84c-4798-9c88-bf946f00ced4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.190329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a8d9-c84c-4798-9c88-bf946f00ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.190629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a8d9-c84c-4798-9c88-bf946f00ced4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.191552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79a8d9-c84c-4798-9c88-bf946f00ced4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.193523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79a8d9-c84c-4798-9c88-bf946f00ced4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.209204 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2323/4645] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:28.213970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.229980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7818321-b771-4c47-b360-76e58fea6d25 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.236019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.236915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.237243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.237555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] PATH_INFO: `/v3/projects/a8be55995adc410895db605583b33c71/users/f2f968ee559b4feeb9d5c20f1a400427/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.256656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8be55995adc410895db605583b33c71, user_id=f2f968ee559b4feeb9d5c20f1a400427, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.259877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.291577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471e849-ff79-45dc-8736-5f7d9a29b822 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.292359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471e849-ff79-45dc-8736-5f7d9a29b822 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.292532 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2323/4646] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:28 2022] PUT /identity/v3/projects/a8be55995adc410895db605583b33c71/users/f2f968ee559b4feeb9d5c20f1a400427/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:28.292640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471e849-ff79-45dc-8736-5f7d9a29b822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.322878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2471e849-ff79-45dc-8736-5f7d9a29b822 None None] MFA Rules not processed for user `f2f968ee559b4feeb9d5c20f1a400427`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:28.324248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2471e849-ff79-45dc-8736-5f7d9a29b822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:28.411959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2324/4647] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:28.422280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.424157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:28.526261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85a824-a607-46a1-a5af-afe5555d6714 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.539129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81f502-ec24-46e2-9f35-917ab0713187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.542203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81f502-ec24-46e2-9f35-917ab0713187 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.542526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81f502-ec24-46e2-9f35-917ab0713187 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.542796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81f502-ec24-46e2-9f35-917ab0713187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.565905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81f502-ec24-46e2-9f35-917ab0713187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.567368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81f502-ec24-46e2-9f35-917ab0713187 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.588070 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2324/4648] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:28.632767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471e849-ff79-45dc-8736-5f7d9a29b822 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.717677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:28.744100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471e849-ff79-45dc-8736-5f7d9a29b822 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.754851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.756305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.756661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.756980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] PATH_INFO: `/v3/users/f2f968ee559b4feeb9d5c20f1a400427` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.758247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f968ee559b4feeb9d5c20f1a400427)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.760529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.852895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f968ee559b4feeb9d5c20f1a400427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:28.879126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f968ee559b4feeb9d5c20f1a400427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:28.909748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f968ee559b4feeb9d5c20f1a400427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:28.911910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f968ee559b4feeb9d5c20f1a400427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:28.912664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f968ee559b4feeb9d5c20f1a400427'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:28.948761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2325/4649] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:28 2022] DELETE /identity/v3/users/f2f968ee559b4feeb9d5c20f1a400427 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:28.953934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.954380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.954788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:29.019844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:29.021129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:29.114646 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2325/4650] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:29.124975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:29.126491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:29.201827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c10a7e-ff67-4e56-b2f4-ddf8d50fb05b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:29.205567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b9000-f133-4b64-8373-05a8bf814688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:29.206243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9000-f133-4b64-8373-05a8bf814688 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:29.206585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9000-f133-4b64-8373-05a8bf814688 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:29.206917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b9000-f133-4b64-8373-05a8bf814688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:29.217962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b9000-f133-4b64-8373-05a8bf814688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:29.219644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b9000-f133-4b64-8373-05a8bf814688 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:29.233955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2326/4651] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:29.591749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:29.595864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:29.629424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b4d19-d694-4960-8404-a347ec93c62f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:29.645213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:29.646226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:29.646724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:29.647007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] PATH_INFO: `/v3/projects/a8be55995adc410895db605583b33c71` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:29.652024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8be55995adc410895db605583b33c71)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:29.654033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:29.733980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:29.768070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8be55995adc410895db605583b33c71', 'request_id': 'req-45a73885-4252-438a-b865-0e90f8c0b2da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:29.776838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8be55995adc410895db605583b33c71', 'request_id': 'req-45a73885-4252-438a-b865-0e90f8c0b2da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:29.791176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8be55995adc410895db605583b33c71', 'request_id': 'req-45a73885-4252-438a-b865-0e90f8c0b2da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:29.803465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] The token cache is being invalidate because project a8be55995adc410895db605583b33c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:29.803850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:29.804563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45a73885-4252-438a-b865-0e90f8c0b2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:29.806505 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2326/4652] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:29 2022] DELETE /identity/v3/projects/a8be55995adc410895db605583b33c71 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:29.811291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899e9c5-0597-43dd-9783-72356ca0b2ce None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:29.811901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899e9c5-0597-43dd-9783-72356ca0b2ce None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:29.812427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899e9c5-0597-43dd-9783-72356ca0b2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:29.835364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0899e9c5-0597-43dd-9783-72356ca0b2ce None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:29.837125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0899e9c5-0597-43dd-9783-72356ca0b2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:29.925794 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2327/4653] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:29.926178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8cb55-893c-4d22-be34-5895ad25de96 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:29.926493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8cb55-893c-4d22-be34-5895ad25de96 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:29.926769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8cb55-893c-4d22-be34-5895ad25de96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:29.962685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3bc8cb55-893c-4d22-be34-5895ad25de96 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:29.963910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3bc8cb55-893c-4d22-be34-5895ad25de96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:29.972534 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2327/4654] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:29.977234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899e9c5-0597-43dd-9783-72356ca0b2ce None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:29.992777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899e9c5-0597-43dd-9783-72356ca0b2ce None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:30.010944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:30.011668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.012286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.012286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.018817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:30.019195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:30.023309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:30.037505 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2328/4655] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:30.040571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8cb55-893c-4d22-be34-5895ad25de96 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:30.053846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8cb55-893c-4d22-be34-5895ad25de96 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:30.058863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:30.059482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.059882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.061673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.062486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:30.065053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:30.097689 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2328/4656] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:30 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:30.098934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:30.105213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:30.112150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f69e53-8d52-4c04-81ae-52c0fbea884c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:30.175796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:30.177162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.177627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.178071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.178897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:30.181029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:30.205499 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2329/4657] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:30 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:07:30.210782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:30.221728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317c1dd-f8f0-46f9-96b9-e79e0e442c71 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:30.226526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:30.227309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.227650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.227996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.228722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:30.230359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:30.240758 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2329/4658] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:30.244869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:30.255867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9d7c8-49db-443b-a00b-514aa07ed0d4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:30.261137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5949b2-61c8-4359-aa55-566e923f399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:30.262039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5949b2-61c8-4359-aa55-566e923f399e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.262039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5949b2-61c8-4359-aa55-566e923f399e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.262305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5949b2-61c8-4359-aa55-566e923f399e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.262974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5949b2-61c8-4359-aa55-566e923f399e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:30.264635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5949b2-61c8-4359-aa55-566e923f399e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:30.276037 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2330/4659] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:30.280491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:30.290977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93adf2-c1b1-48bc-bc3c-9e308ea6e4ac admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:30.296100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:30.296765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.297041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.297312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] PATH_INFO: `/v3/projects/4b7172e024ea40f897729cefb597b88a/users/4993acb6f51f4237a2edcb2b92a8bdb7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.313923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7172e024ea40f897729cefb597b88a, user_id=4993acb6f51f4237a2edcb2b92a8bdb7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:30.316592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:30.332397 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2330/4660] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:30 2022] PUT /identity/v3/projects/4b7172e024ea40f897729cefb597b88a/users/4993acb6f51f4237a2edcb2b92a8bdb7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:30.343360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.343639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.343910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.373685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] MFA Rules not processed for user `4993acb6f51f4237a2edcb2b92a8bdb7`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:30.375137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:30.517733 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2331/4661] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:30.537829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:30.540075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:30.674897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572057d1-e21c-400d-b510-26b56f32e170 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:30.679417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:30.680035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:30.680312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:30.680581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:30.702678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:30.704826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:30.718566 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2331/4662] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:35.506404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:35.508362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:35.653849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42db21-e31e-468f-a2cf-17f7be9a534f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:35.680011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:35.680676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:35.680952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:35.681221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] PATH_INFO: `/v3/users/02aa44fb4544430484ce6cf03e6d0aa3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:35.684091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02aa44fb4544430484ce6cf03e6d0aa3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:35.695123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:35.747826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:35.816623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02aa44fb4544430484ce6cf03e6d0aa3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:35.837994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02aa44fb4544430484ce6cf03e6d0aa3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:35.855695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02aa44fb4544430484ce6cf03e6d0aa3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:35.856348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02aa44fb4544430484ce6cf03e6d0aa3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:35.857358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2543b9c0-c6c4-4fce-ac8c-87d63ae8ba76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02aa44fb4544430484ce6cf03e6d0aa3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:35.886067 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2332/4663] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:35 2022] DELETE /identity/v3/users/02aa44fb4544430484ce6cf03e6d0aa3 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:36.667962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:36.684010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49fb54-bd21-4946-a916-420f3d149f98 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:36.824052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:36.837748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:36.839070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.853385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.854337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] PATH_INFO: `/v3/projects/d704de116c314d1e880d826915a43eae` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.857675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d704de116c314d1e880d826915a43eae)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:36.860764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:36.984494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd704de116c314d1e880d826915a43eae', 'request_id': 'req-b2045d9b-b9a8-46b9-812b-5ad867ae908c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:36.995991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd704de116c314d1e880d826915a43eae', 'request_id': 'req-b2045d9b-b9a8-46b9-812b-5ad867ae908c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:37.008604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd704de116c314d1e880d826915a43eae', 'request_id': 'req-b2045d9b-b9a8-46b9-812b-5ad867ae908c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:37.035648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] The token cache is being invalidate because project d704de116c314d1e880d826915a43eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:37.036422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:37.037363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b2045d9b-b9a8-46b9-812b-5ad867ae908c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:37.040156 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2332/4664] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:36 2022] DELETE /identity/v3/projects/d704de116c314d1e880d826915a43eae => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:37.045309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86da560-fd27-41f3-8913-cc16c34bd4f4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.045864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86da560-fd27-41f3-8913-cc16c34bd4f4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.046083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86da560-fd27-41f3-8913-cc16c34bd4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.076972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e86da560-fd27-41f3-8913-cc16c34bd4f4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:37.079251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e86da560-fd27-41f3-8913-cc16c34bd4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:37.172025 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2333/4665] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:37.175005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74dbe8-2e88-4d4b-9273-8c09ba6a1c15 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.175412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74dbe8-2e88-4d4b-9273-8c09ba6a1c15 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.175649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74dbe8-2e88-4d4b-9273-8c09ba6a1c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.216859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7c74dbe8-2e88-4d4b-9273-8c09ba6a1c15 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:37.218578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7c74dbe8-2e88-4d4b-9273-8c09ba6a1c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:37.233086 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2333/4666] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:37.234197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86da560-fd27-41f3-8913-cc16c34bd4f4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:37.249840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86da560-fd27-41f3-8913-cc16c34bd4f4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:37.267605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:37.268681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.269180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.269718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.270812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:37.271314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:37.272995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:37.288154 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2334/4667] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:07:37.296571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74dbe8-2e88-4d4b-9273-8c09ba6a1c15 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:37.309055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74dbe8-2e88-4d4b-9273-8c09ba6a1c15 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:37.318946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:37.319579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.319889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.320213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.322469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:37.324894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:37.373710 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2334/4668] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:37 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:37.374785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:37.387720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ded9b-a32b-44cb-84e4-02ec6c8a2b2e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:37.523119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:37.523931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.524493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.524769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.525439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:37.527491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:37.583670 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2335/4669] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:37 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:07:37.585763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:37.605393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846b7c6-aaa2-4d6f-9784-f022767cca96 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:37.624717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:37.625816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.628945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.632031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.633593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:37.653502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:37.701481 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2335/4670] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:37.702146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:37.718620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca49b9e-a09e-4807-bdc6-e917195d0f7d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:37.730914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52e3e2-e4c6-46a9-b1a6-b29e383a1dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:37.731577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52e3e2-e4c6-46a9-b1a6-b29e383a1dab admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.731823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52e3e2-e4c6-46a9-b1a6-b29e383a1dab admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.732098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52e3e2-e4c6-46a9-b1a6-b29e383a1dab admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.732764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52e3e2-e4c6-46a9-b1a6-b29e383a1dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:37.740384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52e3e2-e4c6-46a9-b1a6-b29e383a1dab admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:37.743023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:37.775415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2336/4671] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:07:37.780083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:37.809899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274b708-2822-4843-bad9-20a91946e54b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:37.822898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:37.824098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.824695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.825551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] PATH_INFO: `/v3/projects/c4a4d7fd12784db69cf699e7e58d4d6a/users/7a13dc0c95a64d499bd05f808565fccd/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.866193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a4d7fd12784db69cf699e7e58d4d6a, user_id=7a13dc0c95a64d499bd05f808565fccd, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:37.876301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:37.907457 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2336/4672] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:37 2022] PUT /identity/v3/projects/c4a4d7fd12784db69cf699e7e58d4d6a/users/7a13dc0c95a64d499bd05f808565fccd/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:37.913633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718341d-3a78-4e07-bbdb-34cfada86ba8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:37.913919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718341d-3a78-4e07-bbdb-34cfada86ba8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:37.914196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718341d-3a78-4e07-bbdb-34cfada86ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:37.967398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f718341d-3a78-4e07-bbdb-34cfada86ba8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:37.968671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f718341d-3a78-4e07-bbdb-34cfada86ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:38.142805 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2337/4673] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:38.205043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:38.206882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:38.356730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315d836-bb86-47f4-933e-d0b7d72e324c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:38.364630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:38.365494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:38.365946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:38.366382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:38.381395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:38.383673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:38.398028 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2337/4674] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:38.550896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca79b-0946-4699-8154-35d796c9d1c1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:38.551181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca79b-0946-4699-8154-35d796c9d1c1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:38.551934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca79b-0946-4699-8154-35d796c9d1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:38.580920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a9fca79b-0946-4699-8154-35d796c9d1c1 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:38.582533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9fca79b-0946-4699-8154-35d796c9d1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:38.664954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2338/4675] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:38.689850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:38.694500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:38.729012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:38.812250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617348-5245-4bfb-a7f3-579fb374735d service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:38.824436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:38.825408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:38.825838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:38.826410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:38.841300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:38.842921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:38.869693 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2338/4676] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:48.649903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=53 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:48.849715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=22 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:51.820773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d66b9-2640-4764-9a69-6fefcbc1abba None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:51.821702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d66b9-2640-4764-9a69-6fefcbc1abba None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:51.822217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d66b9-2640-4764-9a69-6fefcbc1abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:51.934792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cd4d66b9-2640-4764-9a69-6fefcbc1abba None None] MFA Rules not processed for user `7a13dc0c95a64d499bd05f808565fccd`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:51.936031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cd4d66b9-2640-4764-9a69-6fefcbc1abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:52.133659 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2339/4677] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:52.140040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:52.163388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb9c99-d5b5-4f5a-a4d7-dcccf52ad1f1 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:52.182686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:52.183352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:52.183625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:52.183891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:52.184555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:52.190130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:52.278651 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2339/4678] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:52 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:52.282843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d66b9-2640-4764-9a69-6fefcbc1abba None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:52.304744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d66b9-2640-4764-9a69-6fefcbc1abba None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:52.443862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:52.444544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:52.444774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:52.445044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:52.445898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:52.447968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:52.514307 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2340/4679] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:52 2022] POST /identity/v3/users => generated 301 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:52.516607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:52.537445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731684c-7aaf-4ca1-aa88-d70709ba8e4c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:52.564018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:52.564663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:52.564945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:52.565198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:52.565878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:52.567502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:52.603102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2340/4680] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:52.607582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:52.619925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3965e-7338-49cd-8ea4-4c4d9738468c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:52.638581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:52.639518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:52.639962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:52.640394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] PATH_INFO: `/v3/projects/3271ad542baf4788824fb264cbc5d0d8/users/0b0991772b84482bbb1a6ae35629558d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:52.680868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3271ad542baf4788824fb264cbc5d0d8, user_id=0b0991772b84482bbb1a6ae35629558d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:52.683847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:52.725895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2341/4681] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:52 2022] PUT /identity/v3/projects/3271ad542baf4788824fb264cbc5d0d8/users/0b0991772b84482bbb1a6ae35629558d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:52.729699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:52.746528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d03e1-faf4-48a4-ac44-e2eae0b46800 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:52.954047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:52.954792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:52.955177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:52.956020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:52.959977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:52.961860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:53.001282 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2341/4682] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:53.006028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:53.026431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3389dc7-4a81-47bb-a1f4-2006a5c59e25 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:53.046077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311fc01-2f34-4dbc-b7d9-833ea560dd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:53.046870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311fc01-2f34-4dbc-b7d9-833ea560dd6b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:53.047291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311fc01-2f34-4dbc-b7d9-833ea560dd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:53.047635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311fc01-2f34-4dbc-b7d9-833ea560dd6b admin admin] PATH_INFO: `/v3/projects/3271ad542baf4788824fb264cbc5d0d8/users/0b0991772b84482bbb1a6ae35629558d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:53.051099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311fc01-2f34-4dbc-b7d9-833ea560dd6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3271ad542baf4788824fb264cbc5d0d8, user_id=0b0991772b84482bbb1a6ae35629558d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:53.053981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311fc01-2f34-4dbc-b7d9-833ea560dd6b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:53.101243 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2342/4683] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:53 2022] PUT /identity/v3/projects/3271ad542baf4788824fb264cbc5d0d8/users/0b0991772b84482bbb1a6ae35629558d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:54.424820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:54.426818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:54.448506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda6f49-954b-45ae-84f7-25fbcc3a16f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:54.695356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:54.696026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:54.696302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:54.696705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] PATH_INFO: `/v3/users/4993acb6f51f4237a2edcb2b92a8bdb7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:54.698058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4993acb6f51f4237a2edcb2b92a8bdb7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:54.702952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:54.832579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4993acb6f51f4237a2edcb2b92a8bdb7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:54.835910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4993acb6f51f4237a2edcb2b92a8bdb7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:54.885164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4993acb6f51f4237a2edcb2b92a8bdb7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:54.888589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4993acb6f51f4237a2edcb2b92a8bdb7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:54.944955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:54.963673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4993acb6f51f4237a2edcb2b92a8bdb7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:54.996959 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2342/4684] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:54 2022] DELETE /identity/v3/users/4993acb6f51f4237a2edcb2b92a8bdb7 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:54.999908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:55.000430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:55.000509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:55.047866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:55.049361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:55.118788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:55.195338 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2343/4685] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:55.202425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:55.205961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:55.387198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d6162-2862-4680-b182-4b7a9bec45b0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:55.396960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf9bd4-27f2-437d-a155-6bd70498c979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:55.397838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9bd4-27f2-437d-a155-6bd70498c979 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:55.398216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9bd4-27f2-437d-a155-6bd70498c979 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:55.398585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf9bd4-27f2-437d-a155-6bd70498c979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:55.422022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf9bd4-27f2-437d-a155-6bd70498c979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:55.427956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf9bd4-27f2-437d-a155-6bd70498c979 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:55.457174 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2343/4686] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:55.947114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:55.948587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:55.982249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701941b-56ca-4f73-b86a-af26a25281e6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:55.996178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:55.997914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:55.999416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:55.999940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] PATH_INFO: `/v3/projects/4b7172e024ea40f897729cefb597b88a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:56.008308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7172e024ea40f897729cefb597b88a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:56.027908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:56.220324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7172e024ea40f897729cefb597b88a', 'request_id': 'req-2bed7c05-8b3d-4eed-adcb-318ad675edcc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:56.243515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7172e024ea40f897729cefb597b88a', 'request_id': 'req-2bed7c05-8b3d-4eed-adcb-318ad675edcc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:56.251016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7172e024ea40f897729cefb597b88a', 'request_id': 'req-2bed7c05-8b3d-4eed-adcb-318ad675edcc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:56.277636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] The token cache is being invalidate because project 4b7172e024ea40f897729cefb597b88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:56.278226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:56.279096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2bed7c05-8b3d-4eed-adcb-318ad675edcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:56.287148 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2344/4687] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:55 2022] DELETE /identity/v3/projects/4b7172e024ea40f897729cefb597b88a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:56.289016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e273b-11d0-429a-b078-ea30184eec7c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:56.289601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e273b-11d0-429a-b078-ea30184eec7c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:56.290035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e273b-11d0-429a-b078-ea30184eec7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:56.331929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2d8e273b-11d0-429a-b078-ea30184eec7c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:56.333478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d8e273b-11d0-429a-b078-ea30184eec7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:56.501260 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2344/4688] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:56.507347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50304f9-d4ec-400b-afe0-c68f6a058173 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:56.507869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50304f9-d4ec-400b-afe0-c68f6a058173 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:56.508295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50304f9-d4ec-400b-afe0-c68f6a058173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:56.578213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d50304f9-d4ec-400b-afe0-c68f6a058173 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:56.579815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d50304f9-d4ec-400b-afe0-c68f6a058173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:56.607532 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2345/4689] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:56.612023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e273b-11d0-429a-b078-ea30184eec7c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:56.630889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e273b-11d0-429a-b078-ea30184eec7c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:56.648714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:56.649669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:56.650121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:56.652502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:56.653403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:56.653882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:56.657241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:56.701018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2345/4690] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:56.706595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50304f9-d4ec-400b-afe0-c68f6a058173 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:56.729428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50304f9-d4ec-400b-afe0-c68f6a058173 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:56.738845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:56.739487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:56.739913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:56.740201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:56.740853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:56.742950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:56.834589 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2346/4691] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:56 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:56.838666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:56.855785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af745d-b75b-42b2-8b50-2bb287d68a26 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:57.086437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:57.087077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:57.087355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:57.087627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:57.088401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:57.090440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:57.161051 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2346/4692] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:56 2022] POST /identity/v3/users => generated 285 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:57.165863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:57.181339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017da1d-33f7-47b9-a3e5-a4d4ba5eb9f9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:57.215289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:57.219302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:57.220039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:57.227932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:57.230256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:57.233398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:57.280313 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2347/4693] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:57.284141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:57.290136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:57.300306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523c0fd-a512-4d34-8b57-d42df5d3bfcc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:57.311817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b304a75-9e17-4574-a2a9-f1060173dd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:57.312079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b304a75-9e17-4574-a2a9-f1060173dd2b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:57.318124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b304a75-9e17-4574-a2a9-f1060173dd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:57.319705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b304a75-9e17-4574-a2a9-f1060173dd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:57.319832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b304a75-9e17-4574-a2a9-f1060173dd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:57.326282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b304a75-9e17-4574-a2a9-f1060173dd2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:57.367827 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2347/4694] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:57.372579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:57.385309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3ac27-ff5a-4140-8529-38e3ea2e4d8e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:57.415560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:57.416258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:57.416536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:57.416810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] PATH_INFO: `/v3/projects/9a2e9d7846c24a91addc9f43215b13d0/users/386e12cd145942648098c6569ccf79a7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:57.492209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2e9d7846c24a91addc9f43215b13d0, user_id=386e12cd145942648098c6569ccf79a7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:57.515396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:57.569285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2348/4695] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:57 2022] PUT /identity/v3/projects/9a2e9d7846c24a91addc9f43215b13d0/users/386e12cd145942648098c6569ccf79a7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:57.582422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18118202-5e2c-4bc1-8f30-29d5594b969e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:57.584634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18118202-5e2c-4bc1-8f30-29d5594b969e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:57.585061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18118202-5e2c-4bc1-8f30-29d5594b969e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:57.654361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-18118202-5e2c-4bc1-8f30-29d5594b969e None None] MFA Rules not processed for user `386e12cd145942648098c6569ccf79a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:57.655855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-18118202-5e2c-4bc1-8f30-29d5594b969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:57.854434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2348/4696] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:07:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:57.921716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:57.924024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:58.180320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db58b64-e204-4e9b-a33a-021e1f1755aa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:58.190075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:58.193543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:58.193828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:58.197627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:58.227357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:58.228186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:58.274665 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2349/4697] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:04.264672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ce424-0a62-4aa8-8c43-ce6e84448841 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:04.265208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ce424-0a62-4aa8-8c43-ce6e84448841 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:04.265208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ce424-0a62-4aa8-8c43-ce6e84448841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:04.325948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-161ce424-0a62-4aa8-8c43-ce6e84448841 None None] MFA Rules not processed for user `0b0991772b84482bbb1a6ae35629558d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:04.327187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-161ce424-0a62-4aa8-8c43-ce6e84448841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:04.495595 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2349/4698] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:04.509668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:04.511235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:04.520427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:04.606042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363e7d3-bde3-4114-987a-3c84953e5f85 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:04.614583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:04.615414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:04.616445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:04.616763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:04.622600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:04.788925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:04.790753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:04.802171 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2350/4699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:06.599237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:06.600704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:06.601121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:06.633516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] MFA Rules not processed for user `2ef547e071fd43ec96c745c26eca119f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:06.635677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:06.787736 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2350/4700] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:08.162069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:08.164557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:08.197887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae61b4-5cfc-4798-b74f-5795125ada63 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:08.210694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:08.211329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:08.211607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:08.212033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:08.215336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:08.258024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:08.259435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:08.302931 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2351/4701] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:13.771852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:13.773326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:13.798246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:13.954352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f59f-8a6b-4cbb-a47c-1393bd33326d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:13.979883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:13.984040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:13.984560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:13.984665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] PATH_INFO: `/v3/users/24c7a82126124ebb8baf5c656ff9849d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:13.988335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c7a82126124ebb8baf5c656ff9849d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:13.992145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:14.125003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c7a82126124ebb8baf5c656ff9849d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:14.126942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c7a82126124ebb8baf5c656ff9849d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:14.145357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c7a82126124ebb8baf5c656ff9849d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:14.147080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c7a82126124ebb8baf5c656ff9849d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:14.170075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c7a82126124ebb8baf5c656ff9849d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:14.186551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2351/4702] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:08:13 2022] DELETE /identity/v3/users/24c7a82126124ebb8baf5c656ff9849d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:14.843190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:14.880036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572bb2a-712d-48d9-9b70-79cc129938ef service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:15.108724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:15.109854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:15.110360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:15.110870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] PATH_INFO: `/v3/projects/c3a83c9c29f1483b9ee9c48e94dc17ce` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:15.112382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a83c9c29f1483b9ee9c48e94dc17ce)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:15.125187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:15.352913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:15.357412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a83c9c29f1483b9ee9c48e94dc17ce', 'request_id': 'req-7f965f9b-9496-4163-877c-a5eb71c8b690'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.377143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a83c9c29f1483b9ee9c48e94dc17ce', 'request_id': 'req-7f965f9b-9496-4163-877c-a5eb71c8b690'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.389550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a83c9c29f1483b9ee9c48e94dc17ce', 'request_id': 'req-7f965f9b-9496-4163-877c-a5eb71c8b690'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.403638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] The token cache is being invalidate because project c3a83c9c29f1483b9ee9c48e94dc17ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:08:15.404071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.404951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.407362 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2352/4703] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:08:14 2022] DELETE /identity/v3/projects/c3a83c9c29f1483b9ee9c48e94dc17ce => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:15.411685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:15.414680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:15.431036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8dcb9-53a3-48c6-b8b4-1f22b232a84f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:15.576251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:15.581836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:15.583523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:15.584277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] PATH_INFO: `/v3/users/fce8fdffbc354641b50a0d46ea6be26f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:15.588134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce8fdffbc354641b50a0d46ea6be26f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:15.596456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:15.731719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:15.751321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce8fdffbc354641b50a0d46ea6be26f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.752196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce8fdffbc354641b50a0d46ea6be26f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.772555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce8fdffbc354641b50a0d46ea6be26f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.773738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce8fdffbc354641b50a0d46ea6be26f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.801559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ada5ee75-9e29-40bd-89a0-caca1bdf6960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce8fdffbc354641b50a0d46ea6be26f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:15.821632 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2352/4704] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:08:15 2022] DELETE /identity/v3/users/fce8fdffbc354641b50a0d46ea6be26f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:17.006330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:17.007985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:17.032789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f965f9b-9496-4163-877c-a5eb71c8b690 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:17.140369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:17.141800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:17.142490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:17.144040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] PATH_INFO: `/v3/projects/f1fca09044e249e497bc266060c3c3ab` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:17.146355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1fca09044e249e497bc266060c3c3ab)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:17.152856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:17.283268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1fca09044e249e497bc266060c3c3ab', 'request_id': 'req-2d345fb5-d15b-46e7-af21-dde97ebfe585'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:17.306527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1fca09044e249e497bc266060c3c3ab', 'request_id': 'req-2d345fb5-d15b-46e7-af21-dde97ebfe585'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:17.314660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1fca09044e249e497bc266060c3c3ab', 'request_id': 'req-2d345fb5-d15b-46e7-af21-dde97ebfe585'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:17.338064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] The token cache is being invalidate because project f1fca09044e249e497bc266060c3c3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:08:17.338520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:17.339281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d345fb5-d15b-46e7-af21-dde97ebfe585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:17.341421 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2353/4705] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:08:17 2022] DELETE /identity/v3/projects/f1fca09044e249e497bc266060c3c3ab => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:17.346147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58445d-a853-4236-8005-893906d3f4ee None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:17.346724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58445d-a853-4236-8005-893906d3f4ee None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:17.346859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58445d-a853-4236-8005-893906d3f4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:17.372681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2a58445d-a853-4236-8005-893906d3f4ee None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:17.382951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a58445d-a853-4236-8005-893906d3f4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:17.481110 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2353/4706] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:08:17.487425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a40ad-d64d-4f2a-b60b-c8794ee39ff4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:17.487993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a40ad-d64d-4f2a-b60b-c8794ee39ff4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:17.488364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a40ad-d64d-4f2a-b60b-c8794ee39ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:17.540987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-782a40ad-d64d-4f2a-b60b-c8794ee39ff4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:17.545134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-782a40ad-d64d-4f2a-b60b-c8794ee39ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:17.597055 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2354/4707] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:17.598407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58445d-a853-4236-8005-893906d3f4ee None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:17.634237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58445d-a853-4236-8005-893906d3f4ee None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:17.652127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:17.664312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:17.664632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:17.664878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:17.665580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:08:17.665885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:17.674145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:17.716212 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2354/4708] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:08:17.721213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a40ad-d64d-4f2a-b60b-c8794ee39ff4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:17.737583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a40ad-d64d-4f2a-b60b-c8794ee39ff4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:17.750339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:17.750984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:17.751265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:17.751870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:17.755615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:17.762862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:17.841251 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2355/4709] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:08:17 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:08:17.845243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:17.881304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507deed1-eee3-4e79-87b9-f3520b5d4762 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:18.013805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:18.014958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:18.015404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:18.016245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:18.017291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:18.021530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:18.108600 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2355/4710] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:08:17 2022] POST /identity/v3/users => generated 286 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:08:18.112659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:18.130635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545b5a2-2572-4b50-a119-ed062a934da9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:18.148660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:18.150722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:18.152630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:18.153154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:18.154577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:18.157229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:18.198603 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2356/4711] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:18.202609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:18.218300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56378afb-a41f-4ccb-8064-eea326ab27ab admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:18.234729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b1a03-4c4f-45b1-a363-ec28257c2370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:18.241810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1a03-4c4f-45b1-a363-ec28257c2370 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:18.243413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1a03-4c4f-45b1-a363-ec28257c2370 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:18.245181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b1a03-4c4f-45b1-a363-ec28257c2370 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:18.248459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b1a03-4c4f-45b1-a363-ec28257c2370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:18.250850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b1a03-4c4f-45b1-a363-ec28257c2370 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:18.274141 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2356/4712] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:18.278346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:18.292359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46a51d-216d-49b6-9c4b-cc89293b98eb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:18.316109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:18.318260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:18.320308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:18.322268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] PATH_INFO: `/v3/projects/7b9b4528d62c4e37958b6c5ba0dc000a/users/da1264855d824485880b5b477c7bc4f9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:18.380676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9b4528d62c4e37958b6c5ba0dc000a, user_id=da1264855d824485880b5b477c7bc4f9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:18.383806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:18.483291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83d6d-61a8-4749-85c1-b1ed5a493af9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:18.483880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83d6d-61a8-4749-85c1-b1ed5a493af9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:18.483880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83d6d-61a8-4749-85c1-b1ed5a493af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:18.498029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2357/4713] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:08:18 2022] PUT /identity/v3/projects/7b9b4528d62c4e37958b6c5ba0dc000a/users/da1264855d824485880b5b477c7bc4f9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:18.561101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f1e83d6d-61a8-4749-85c1-b1ed5a493af9 None None] MFA Rules not processed for user `da1264855d824485880b5b477c7bc4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:18.562452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1e83d6d-61a8-4749-85c1-b1ed5a493af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:18.577812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:18.768686 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2357/4714] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:18.874787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:18.878232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:18.887760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:19.114084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0ccbd-4e8b-48c0-b826-5043ac8816e3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:19.124528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fd134-f058-4369-8ad8-38a4828c2e73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:19.125426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fd134-f058-4369-8ad8-38a4828c2e73 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:19.128295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fd134-f058-4369-8ad8-38a4828c2e73 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:19.128575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fd134-f058-4369-8ad8-38a4828c2e73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:19.156692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fd134-f058-4369-8ad8-38a4828c2e73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:19.158791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fd134-f058-4369-8ad8-38a4828c2e73 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:19.213109 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2358/4715] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:20.481690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e83d6d-61a8-4749-85c1-b1ed5a493af9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:20.647921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e83d6d-61a8-4749-85c1-b1ed5a493af9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:20.660489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:20.661418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:20.661866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:20.663360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] PATH_INFO: `/v3/users/da1264855d824485880b5b477c7bc4f9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:20.666373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1264855d824485880b5b477c7bc4f9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:20.677823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:20.801824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1264855d824485880b5b477c7bc4f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:20.803174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1264855d824485880b5b477c7bc4f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:20.834485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1264855d824485880b5b477c7bc4f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:20.837091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1264855d824485880b5b477c7bc4f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:20.866702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1264855d824485880b5b477c7bc4f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:20.899948 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2358/4716] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:08:20 2022] DELETE /identity/v3/users/da1264855d824485880b5b477c7bc4f9 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:20.904173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8eca9-e24d-43b9-951f-30e700956316 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:20.904757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8eca9-e24d-43b9-951f-30e700956316 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:20.904757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8eca9-e24d-43b9-951f-30e700956316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:20.943742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-66f8eca9-e24d-43b9-951f-30e700956316 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:20.946753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-66f8eca9-e24d-43b9-951f-30e700956316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:21.058451 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2359/4717] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:21.101683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:21.103838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:21.182049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:21.305735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeca775-e079-452f-9057-d5a499719ce1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:21.320074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:21.323710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:21.324053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:21.330318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:21.364167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:21.368440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:21.394236 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2359/4718] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:23.123517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:23.124088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:23.124088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:23.161214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:23.162813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:23.185860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:23.187478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:23.204171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30868f10-055a-4d00-8e4d-b1c2f682adfb service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:23.210952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:23.212213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:23.212699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:23.213271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] PATH_INFO: `/v3/projects/7b9b4528d62c4e37958b6c5ba0dc000a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:23.215717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9b4528d62c4e37958b6c5ba0dc000a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:23.218269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:23.294074 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2360/4719] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:23.317535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:23.321049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:23.328875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9b4528d62c4e37958b6c5ba0dc000a', 'request_id': 'req-f7afef46-0b89-4b27-9339-4123d0275b1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:23.349857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9b4528d62c4e37958b6c5ba0dc000a', 'request_id': 'req-f7afef46-0b89-4b27-9339-4123d0275b1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:23.368432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9b4528d62c4e37958b6c5ba0dc000a', 'request_id': 'req-f7afef46-0b89-4b27-9339-4123d0275b1c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:23.386014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] The token cache is being invalidate because project 7b9b4528d62c4e37958b6c5ba0dc000a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:08:23.387986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:23.389605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f7afef46-0b89-4b27-9339-4123d0275b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:23.398205 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2360/4720] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:08:23 2022] DELETE /identity/v3/projects/7b9b4528d62c4e37958b6c5ba0dc000a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:23.401117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3cbbd-f26e-4e3e-8efc-d055dbad1500 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:23.401671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3cbbd-f26e-4e3e-8efc-d055dbad1500 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:23.401938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:23.404199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3cbbd-f26e-4e3e-8efc-d055dbad1500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:23.476366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3ac3cbbd-f26e-4e3e-8efc-d055dbad1500 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:23.478630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3ac3cbbd-f26e-4e3e-8efc-d055dbad1500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:23.520568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8d90c-8cf7-4a4a-b1bf-9a276a91511f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:23.592197 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2361/4721] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:08:23.602079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74310e07-ba1f-46ee-9185-15a4df8b25a8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:23.602391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74310e07-ba1f-46ee-9185-15a4df8b25a8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:23.602619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74310e07-ba1f-46ee-9185-15a4df8b25a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:23.611436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:23.612420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:23.612866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:23.613592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:23.616113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:23.671374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-74310e07-ba1f-46ee-9185-15a4df8b25a8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:23.673498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-74310e07-ba1f-46ee-9185-15a4df8b25a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:23.711334 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2362/4722] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:23.712391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74310e07-ba1f-46ee-9185-15a4df8b25a8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:23.723189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74310e07-ba1f-46ee-9185-15a4df8b25a8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:23.736132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:23.736786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:23.737092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:23.737382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:23.739280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:23.744930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:23.748633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:08:23.748890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:23.752142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:23.771571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2361/4723] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:08:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:08:23.789105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2363/4724] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:08:23.790779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:23.808287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbddaa2-d258-4488-84c5-0c2561084940 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:23.816294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:23.816924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:23.817248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:23.817609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:23.818254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:23.820246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:23.876907 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2362/4725] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:08:23 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:08:23.878958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:23.898656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c1a2a-04d1-46a7-b70e-46c0b18038fd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:23.985603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:24.035937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:24.036699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:24.037043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:24.037383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:24.038252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:24.040743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:24.092228 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2364/4726] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:08:23 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:08:24.095038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:24.121147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fde61-59f7-4980-86cc-554c8beab55a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:24.134819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:24.137678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:24.138033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:24.139585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:24.141231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:24.145231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:24.173606 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2363/4727] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:24.177517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:24.202652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8cd41-c6f4-472c-bb68-0f780987d6d4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:24.220287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869645c5-e016-4df2-9545-9af7493551de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:24.221743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869645c5-e016-4df2-9545-9af7493551de admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:24.222261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869645c5-e016-4df2-9545-9af7493551de admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:24.222676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869645c5-e016-4df2-9545-9af7493551de admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:24.226347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869645c5-e016-4df2-9545-9af7493551de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:24.228821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869645c5-e016-4df2-9545-9af7493551de admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:24.256501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2365/4728] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:24.260369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:24.288444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070438c1-04d0-4d94-a961-c235a2379486 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:24.295422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:24.296572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:24.297103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:24.297599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] PATH_INFO: `/v3/projects/15b15d63f37e4e93b17a2396cf22f500/users/e50d034fcbdf484896e8c06d98b9d9eb/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:24.330255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b15d63f37e4e93b17a2396cf22f500, user_id=e50d034fcbdf484896e8c06d98b9d9eb, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:24.334253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:24.382262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2364/4729] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:08:24 2022] PUT /identity/v3/projects/15b15d63f37e4e93b17a2396cf22f500/users/e50d034fcbdf484896e8c06d98b9d9eb/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:24.388540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:24.388830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:24.389574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:24.460601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:24.466134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:24.705599 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2366/4730] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:24.744006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:24.745979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:24.871904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:24.948830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf25ea4-0581-4be1-8fab-56e11f04b85f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:24.956618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:24.957490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:24.957859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:24.958185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:24.973982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:24.976563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:25.014399 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2365/4731] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:25.845989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:25.847405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:26.122754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80f445-1747-4212-a402-576ebf94d2ac None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:26.135903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac1817-d420-4f3f-9d9a-98165194375f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:26.136782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac1817-d420-4f3f-9d9a-98165194375f service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:26.137636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac1817-d420-4f3f-9d9a-98165194375f service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:26.138119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac1817-d420-4f3f-9d9a-98165194375f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:26.142160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c0ac1817-d420-4f3f-9d9a-98165194375f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:26.174247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac1817-d420-4f3f-9d9a-98165194375f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:26.182102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac1817-d420-4f3f-9d9a-98165194375f service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:26.216791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2367/4732] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:08:27.469655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:27.471369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:27.489870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964e2f-f3a0-4efc-8ad3-26ab92180f9a service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:27.506588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:27.507925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:27.507925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:27.507925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] PATH_INFO: `/v3/users/33784305478f46ae9fdc4d4b82ebc6f5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:27.521969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=33784305478f46ae9fdc4d4b82ebc6f5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:27.528489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:27.684754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33784305478f46ae9fdc4d4b82ebc6f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:27.716732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33784305478f46ae9fdc4d4b82ebc6f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:27.733395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33784305478f46ae9fdc4d4b82ebc6f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:27.734038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33784305478f46ae9fdc4d4b82ebc6f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:27.734984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26bf0d7b-86af-45fb-80e4-d998a39ff8ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33784305478f46ae9fdc4d4b82ebc6f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:27.812513 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2366/4733] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:08:27 2022] DELETE /identity/v3/users/33784305478f46ae9fdc4d4b82ebc6f5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:29.306970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:29.307450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:29.307580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:29.337659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] MFA Rules not processed for user `7a13dc0c95a64d499bd05f808565fccd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:29.339506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:29.353772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:29.462694 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2368/4734] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:29.468294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303957af-64da-4014-b3ca-e623567dabcb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:29.468817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303957af-64da-4014-b3ca-e623567dabcb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:29.468895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303957af-64da-4014-b3ca-e623567dabcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:29.532927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-303957af-64da-4014-b3ca-e623567dabcb None None] MFA Rules not processed for user `7a13dc0c95a64d499bd05f808565fccd`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:29.536519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-303957af-64da-4014-b3ca-e623567dabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:29.565495 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2367/4735] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:29.660756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:29.660756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:29.821957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f385e0-51ab-4e77-9b24-e4f93d54b465 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:29.850019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:29.851679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:29.851954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:29.852231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] PATH_INFO: `/v3/projects/3e857c532a264ea78e24fd8fe1a44fbc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:29.858465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e857c532a264ea78e24fd8fe1a44fbc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:29.862745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:29.944958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e857c532a264ea78e24fd8fe1a44fbc', 'request_id': 'req-e4763651-93e6-42b5-93a2-330123c182fa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:29.956668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e857c532a264ea78e24fd8fe1a44fbc', 'request_id': 'req-e4763651-93e6-42b5-93a2-330123c182fa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:29.967413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e857c532a264ea78e24fd8fe1a44fbc', 'request_id': 'req-e4763651-93e6-42b5-93a2-330123c182fa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:29.985071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] The token cache is being invalidate because project 3e857c532a264ea78e24fd8fe1a44fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:08:29.985421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:29.986105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e4763651-93e6-42b5-93a2-330123c182fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:29.995459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee495d1-37de-4047-bb2e-216591ddc545 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:29.996116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee495d1-37de-4047-bb2e-216591ddc545 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:29.996641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee495d1-37de-4047-bb2e-216591ddc545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.005942 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2369/4736] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:08:29 2022] DELETE /identity/v3/projects/3e857c532a264ea78e24fd8fe1a44fbc => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:30.082345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4ee495d1-37de-4047-bb2e-216591ddc545 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:30.083968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4ee495d1-37de-4047-bb2e-216591ddc545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:30.177977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229eed8-5a78-451f-b82c-24a7e87bb9db None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.178256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229eed8-5a78-451f-b82c-24a7e87bb9db None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.178524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229eed8-5a78-451f-b82c-24a7e87bb9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.180553 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2368/4737] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:30.202729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c229eed8-5a78-451f-b82c-24a7e87bb9db None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:30.204441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c229eed8-5a78-451f-b82c-24a7e87bb9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:30.215673 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2370/4738] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:30.219986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee495d1-37de-4047-bb2e-216591ddc545 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:30.237185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee495d1-37de-4047-bb2e-216591ddc545 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:30.244172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:30.245098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.245623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.246066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.246965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:08:30.247497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:30.250547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:30.273199 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2369/4739] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:08:30.277289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229eed8-5a78-451f-b82c-24a7e87bb9db None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:30.290696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229eed8-5a78-451f-b82c-24a7e87bb9db None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:30.298401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:30.299273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.299542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.299807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.300447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:30.302633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:30.351232 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2371/4740] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:08:30 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:08:30.359358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:30.373241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555d0e9-955c-4364-930b-c4e3fad3373a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:30.391715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:30.459801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:30.460877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.461352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.461868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.462697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:30.465248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:30.504345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:30.505892 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2370/4741] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:08:30 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:08:30.521242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2032ac-1d46-4b90-912f-e9b6fdd69da2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:30.526707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:30.527485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.527764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.528034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.528682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:30.531121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:30.546151 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2372/4742] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:30.550164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:30.567249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeeadad-2818-461f-91e4-a0b46ba6eb7a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:30.580753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b374a3-ac53-4891-938a-aad3cd4bc3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:30.581909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b374a3-ac53-4891-938a-aad3cd4bc3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.582280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b374a3-ac53-4891-938a-aad3cd4bc3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.582608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b374a3-ac53-4891-938a-aad3cd4bc3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.583853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b374a3-ac53-4891-938a-aad3cd4bc3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:30.586311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b374a3-ac53-4891-938a-aad3cd4bc3d3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:30.600056 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2371/4743] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:30.604354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:30.616410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf7c18-45e1-4a4f-8501-6b82777ffb06 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:30.622611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:30.623311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.623612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.623889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] PATH_INFO: `/v3/projects/42e1e6b096bb41419c594dcde97e16e7/users/6d03e73a3bed48f6bf61a702f0f41658/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.641050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:30.645513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e1e6b096bb41419c594dcde97e16e7, user_id=6d03e73a3bed48f6bf61a702f0f41658, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:30.648580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:30.709385 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2373/4744] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:08:30 2022] PUT /identity/v3/projects/42e1e6b096bb41419c594dcde97e16e7/users/6d03e73a3bed48f6bf61a702f0f41658/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:30.721735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.722024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.722301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.756255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] MFA Rules not processed for user `6d03e73a3bed48f6bf61a702f0f41658`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:30.757428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:30.843233 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2372/4745] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:30.851680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:30.853774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:30.952897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c390ae4-be9c-4d0b-be9f-d40bd3fc7a5d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:30.965774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe459561-d59f-4053-8c51-2559fc20c59e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:30.966426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe459561-d59f-4053-8c51-2559fc20c59e service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:30.966707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe459561-d59f-4053-8c51-2559fc20c59e service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:30.966980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe459561-d59f-4053-8c51-2559fc20c59e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:30.996316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe459561-d59f-4053-8c51-2559fc20c59e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:30.997158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe459561-d59f-4053-8c51-2559fc20c59e service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:31.008072 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2374/4746] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:08:32.336282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:32.337989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:32.500684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b221a9-72e1-4992-a5b7-9298aaf9ab18 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:32.507708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:32.507708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:32.507708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:32.508130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:32.510068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:32.524383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:32.526412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:32.577020 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2373/4747] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:32.633263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12094653-833c-492c-9967-46adfbbc8b0b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:32.634422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12094653-833c-492c-9967-46adfbbc8b0b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:32.634843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12094653-833c-492c-9967-46adfbbc8b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:32.660342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-12094653-833c-492c-9967-46adfbbc8b0b None None] MFA Rules not processed for user `6d03e73a3bed48f6bf61a702f0f41658`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:32.662067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-12094653-833c-492c-9967-46adfbbc8b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:32.674897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2375/4748] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:35.359540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:35.361024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:35.377001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:35.437399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b012da-7b30-45e5-b86c-f21300820c01 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:35.443298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fc98d-c420-4359-b529-85301301416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:35.444737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fc98d-c420-4359-b529-85301301416a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:35.445237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fc98d-c420-4359-b529-85301301416a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:35.445770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fc98d-c420-4359-b529-85301301416a admin admin] PATH_INFO: `/v3/users/599c7072545f4280988f6bc50e3ccc5f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:35.450062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fc98d-c420-4359-b529-85301301416a admin admin] RBAC: Authorizing `identity:delete_user(user_id=599c7072545f4280988f6bc50e3ccc5f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:35.452424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fc98d-c420-4359-b529-85301301416a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:35.523854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '599c7072545f4280988f6bc50e3ccc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:35.539376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '599c7072545f4280988f6bc50e3ccc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:35.550376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '599c7072545f4280988f6bc50e3ccc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:35.551567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '599c7072545f4280988f6bc50e3ccc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:35.552632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '599c7072545f4280988f6bc50e3ccc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:35.568236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2374/4749] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:08:35 2022] DELETE /identity/v3/users/599c7072545f4280988f6bc50e3ccc5f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:36.184844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12094653-833c-492c-9967-46adfbbc8b0b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:36.201792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12094653-833c-492c-9967-46adfbbc8b0b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:36.327237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:36.332066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:36.332733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:36.333178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] PATH_INFO: `/v3/projects/d527ac7a0b864094a5c74dea23ec445b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:36.336860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=d527ac7a0b864094a5c74dea23ec445b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:36.339097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:36.477510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:36.480242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:36.491315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd527ac7a0b864094a5c74dea23ec445b', 'request_id': 'req-059193c2-c2c8-4127-b656-c04cd5c50ace'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:36.504704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd527ac7a0b864094a5c74dea23ec445b', 'request_id': 'req-059193c2-c2c8-4127-b656-c04cd5c50ace'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:36.523147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd527ac7a0b864094a5c74dea23ec445b', 'request_id': 'req-059193c2-c2c8-4127-b656-c04cd5c50ace'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:36.550805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] The token cache is being invalidate because project d527ac7a0b864094a5c74dea23ec445b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:08:36.551384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:36.561865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-059193c2-c2c8-4127-b656-c04cd5c50ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:08:36.565979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2376/4750] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:08:36 2022] DELETE /identity/v3/projects/d527ac7a0b864094a5c74dea23ec445b => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:36.572045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b21f8-905a-4e3d-a9dd-d3306a5388e8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:36.576059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b21f8-905a-4e3d-a9dd-d3306a5388e8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:36.576505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b21f8-905a-4e3d-a9dd-d3306a5388e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:36.620736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e55b21f8-905a-4e3d-a9dd-d3306a5388e8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:36.625100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e55b21f8-905a-4e3d-a9dd-d3306a5388e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:36.672332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:36.697151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fc98d-c420-4359-b529-85301301416a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:36.750169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:36.776976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2377/4751] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:36.785514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30399b-e826-40e9-be6b-6910fa1fb33a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:36.786628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30399b-e826-40e9-be6b-6910fa1fb33a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:36.787377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30399b-e826-40e9-be6b-6910fa1fb33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:36.811343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:36.811927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:36.812548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:36.812548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:36.815541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:36.823074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3f30399b-e826-40e9-be6b-6910fa1fb33a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:36.824155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3f30399b-e826-40e9-be6b-6910fa1fb33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:36.849186 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2378/4752] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:36.852774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30399b-e826-40e9-be6b-6910fa1fb33a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:36.852936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:36.854635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:36.869228 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2375/4753] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:36.873549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30399b-e826-40e9-be6b-6910fa1fb33a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:36.888140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:36.890393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:36.891067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:36.892638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:36.893745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:08:36.895314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:36.899315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:36.942096 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2379/4754] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:08:36.943508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:36.956961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a15320-34db-4eeb-b7b4-7b06eabc9c4e service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:36.965899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:36.967482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:36.967939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:36.970246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:36.971130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:36.982018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:37.075493 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2376/4755] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:08:36 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:08:37.076113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:37.090552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73bf37-9908-4a01-a21f-e98981292716 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:37.200435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:37.201800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:37.202246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:37.202700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:37.203505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:37.206451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:37.287662 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2380/4756] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:08:37 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:08:37.290397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:37.315944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d05a20-4f96-4361-a85d-3c77c908f7cd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:37.332091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:37.332389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:37.332665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:37.332941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:37.334051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:37.335678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:37.386417 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2377/4757] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:37.387892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:37.404620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7a30e-3268-433c-a105-5d46e881d7f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:37.416873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169a9f-4183-49d7-bf84-78cd65129d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:37.417546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169a9f-4183-49d7-bf84-78cd65129d8e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:37.417823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169a9f-4183-49d7-bf84-78cd65129d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:37.418090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169a9f-4183-49d7-bf84-78cd65129d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:37.418737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3169a9f-4183-49d7-bf84-78cd65129d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:37.421702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3169a9f-4183-49d7-bf84-78cd65129d8e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:37.450094 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2381/4758] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:08:37.451323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:37.464410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb9a90-499f-404f-92c8-976878d9c671 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:37.481123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:37.482485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:37.482950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:37.483720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] PATH_INFO: `/v3/projects/0b6f25c3540546bf9bffd5610f4f86c8/users/784233ee78764e7dbb774d3cb50165e3/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:37.507744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6f25c3540546bf9bffd5610f4f86c8, user_id=784233ee78764e7dbb774d3cb50165e3, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:37.522756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:37.540540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2378/4759] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:08:37 2022] PUT /identity/v3/projects/0b6f25c3540546bf9bffd5610f4f86c8/users/784233ee78764e7dbb774d3cb50165e3/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:08:37.545442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:37.545744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:37.546179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:37.598804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:37.604610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:37.780277 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2382/4760] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:37.790708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:37.792583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:37.900211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1de3e0-0fa4-4c94-a972-4c92a6aefa07 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:37.907747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6950a5-c368-47a5-8cad-f3b84ce6329d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:37.908880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6950a5-c368-47a5-8cad-f3b84ce6329d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:37.909579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6950a5-c368-47a5-8cad-f3b84ce6329d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:37.910062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6950a5-c368-47a5-8cad-f3b84ce6329d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:37.931814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6950a5-c368-47a5-8cad-f3b84ce6329d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:37.933717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6950a5-c368-47a5-8cad-f3b84ce6329d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:37.946134 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2379/4761] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:39.007121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:39.008751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:39.031561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a7724-8ac8-4973-a2da-b40fb49695db None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:39.041884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239fcc9-f7b6-4e32-a22a-63dd57cf8182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:39.042551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239fcc9-f7b6-4e32-a22a-63dd57cf8182 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:39.042861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239fcc9-f7b6-4e32-a22a-63dd57cf8182 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:39.043138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239fcc9-f7b6-4e32-a22a-63dd57cf8182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:39.052551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d239fcc9-f7b6-4e32-a22a-63dd57cf8182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:39.083596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:39.244942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239fcc9-f7b6-4e32-a22a-63dd57cf8182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:39.246442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239fcc9-f7b6-4e32-a22a-63dd57cf8182 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:39.265587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2383/4762] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:39.944292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:39.944868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:39.944868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:39.967275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] MFA Rules not processed for user `2ef547e071fd43ec96c745c26eca119f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:39.968532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:40.036048 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2380/4763] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:41.200495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba155db2-3421-4b59-9d78-2ddb73d78d10 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:41.201001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba155db2-3421-4b59-9d78-2ddb73d78d10 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:41.201335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba155db2-3421-4b59-9d78-2ddb73d78d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:41.278183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ba155db2-3421-4b59-9d78-2ddb73d78d10 None None] MFA Rules not processed for user `e50d034fcbdf484896e8c06d98b9d9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:41.279484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ba155db2-3421-4b59-9d78-2ddb73d78d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:41.553113 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2384/4764] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:08:41.858314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:41.859919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:41.905216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc752cbe-9f28-418a-b22c-25a689ef95c6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:41.933835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d3fb8-4af1-401d-b730-db82b9c81286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:41.934654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3fb8-4af1-401d-b730-db82b9c81286 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:41.935486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3fb8-4af1-401d-b730-db82b9c81286 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:41.935773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d3fb8-4af1-401d-b730-db82b9c81286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:41.949425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:41.962613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3fb8-4af1-401d-b730-db82b9c81286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:41.967879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d3fb8-4af1-401d-b730-db82b9c81286 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:42.023341 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2381/4765] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:51.538650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=58 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:51.992991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:52.047713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84249ace-7718-4551-8328-7f02754ef2c9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:52.048918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84249ace-7718-4551-8328-7f02754ef2c9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:52.049200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84249ace-7718-4551-8328-7f02754ef2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:52.114601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-84249ace-7718-4551-8328-7f02754ef2c9 None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:52.115898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-84249ace-7718-4551-8328-7f02754ef2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:52.235355 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2385/4766] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:52.247929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f84a4-0253-43df-b1e2-a7bc39967940 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:52.248217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f84a4-0253-43df-b1e2-a7bc39967940 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:52.248492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f84a4-0253-43df-b1e2-a7bc39967940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:52.274038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1d5f84a4-0253-43df-b1e2-a7bc39967940 None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:52.275865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1d5f84a4-0253-43df-b1e2-a7bc39967940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:52.284794 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2382/4767] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:53.200569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c839-900a-4a7a-b547-94cd1a057bad None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:53.201444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c839-900a-4a7a-b547-94cd1a057bad None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:53.204521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c839-900a-4a7a-b547-94cd1a057bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:53.266541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dfb3c839-900a-4a7a-b547-94cd1a057bad None None] MFA Rules not processed for user `795813bb461e42ad8c2a758e9b8430d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:53.268088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dfb3c839-900a-4a7a-b547-94cd1a057bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:53.366380 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2386/4768] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:08:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:53.670466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f84a4-0253-43df-b1e2-a7bc39967940 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:53.716499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f84a4-0253-43df-b1e2-a7bc39967940 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:53.734621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:53.737493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:53.738607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:53.739300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:53.756203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:53.759110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:53.775979 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2383/4769] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:01.999039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:01.999581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:01.999803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:02.019650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] MFA Rules not processed for user `c544e8e6f7a741c985c92ed7437397cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:02.021280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:02.086424 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2387/4770] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:02.729574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:02.732698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:02.823729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51771c27-ecc2-4e0b-8c3b-b21ba9a7a76c service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:02.830693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f7d9e-154b-44b5-8e0e-0d169b589a6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:02.831073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f7d9e-154b-44b5-8e0e-0d169b589a6a service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:02.831284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f7d9e-154b-44b5-8e0e-0d169b589a6a service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:02.831541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f7d9e-154b-44b5-8e0e-0d169b589a6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:02.833309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-711f7d9e-154b-44b5-8e0e-0d169b589a6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:02.847523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f7d9e-154b-44b5-8e0e-0d169b589a6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:02.849293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f7d9e-154b-44b5-8e0e-0d169b589a6a service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:02.862100 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2384/4771] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:09.884515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:09.887432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:09.927394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fd63a-5c43-4370-b083-ccbec44140b7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:09.940521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:09.941878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:09.944218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:09.944836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:09.951215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:09.990155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:09.992936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:10.022762 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2388/4772] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:12.582395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8650250-5532-4e15-ae8f-e50a8485bda7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:12.583336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8650250-5532-4e15-ae8f-e50a8485bda7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:12.583812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8650250-5532-4e15-ae8f-e50a8485bda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:12.610454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f8650250-5532-4e15-ae8f-e50a8485bda7 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:12.612371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f8650250-5532-4e15-ae8f-e50a8485bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:12.714226 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2385/4773] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:12.745649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:12.747384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:12.809632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:12.927704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b8a49-62b1-49ea-950e-8f887b12eeab service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:12.935219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:12.936189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:12.936668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:12.937097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:12.950249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:12.953405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:12.967780 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2389/4774] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:09:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:09:17.036682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76322d-1138-40df-98a0-e1202df483cc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:17.037703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76322d-1138-40df-98a0-e1202df483cc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:17.038291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76322d-1138-40df-98a0-e1202df483cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:17.078911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cf76322d-1138-40df-98a0-e1202df483cc None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:17.080297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cf76322d-1138-40df-98a0-e1202df483cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:17.106460 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2386/4775] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:22.148306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:22.168184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f322852-8307-4b28-b802-18d2aedb0c54 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:22.188516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:22.189741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:22.190282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:22.190788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:22.194333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:22.212208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:22.214610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:22.233304 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2390/4776] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:24.294105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d167b-85f6-4255-8813-4a9fb5f51d72 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:24.294654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d167b-85f6-4255-8813-4a9fb5f51d72 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:24.294773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d167b-85f6-4255-8813-4a9fb5f51d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:24.304149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=77 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:24.335516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1b3d167b-85f6-4255-8813-4a9fb5f51d72 None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:24.337065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1b3d167b-85f6-4255-8813-4a9fb5f51d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:24.354674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2387/4777] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:09:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:32.226473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=26 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:34.314432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:42.088792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:42.090364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:42.115238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4ef4-c2fd-4798-baa7-56338d6bc468 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:42.130346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:42.131745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:42.132252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:42.132721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] PATH_INFO: `/v3/users/e50d034fcbdf484896e8c06d98b9d9eb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:42.134218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50d034fcbdf484896e8c06d98b9d9eb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:42.137618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:42.238194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50d034fcbdf484896e8c06d98b9d9eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:42.239062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50d034fcbdf484896e8c06d98b9d9eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:42.251137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50d034fcbdf484896e8c06d98b9d9eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:42.252218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50d034fcbdf484896e8c06d98b9d9eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:42.284680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9ec99126-e1e3-4457-8335-a8f1b48e6fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50d034fcbdf484896e8c06d98b9d9eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:42.300709 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2391/4778] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:09:42 2022] DELETE /identity/v3/users/e50d034fcbdf484896e8c06d98b9d9eb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:09:42.790783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d167b-85f6-4255-8813-4a9fb5f51d72 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:42.808920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d167b-85f6-4255-8813-4a9fb5f51d72 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:42.907755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:42.908415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:42.908686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:42.909148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] PATH_INFO: `/v3/projects/15b15d63f37e4e93b17a2396cf22f500` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:42.910492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b15d63f37e4e93b17a2396cf22f500)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:42.912585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:42.974241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b15d63f37e4e93b17a2396cf22f500', 'request_id': 'req-1ed1175b-162a-4f25-8315-21973ac51729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:42.983705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b15d63f37e4e93b17a2396cf22f500', 'request_id': 'req-1ed1175b-162a-4f25-8315-21973ac51729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:42.990432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b15d63f37e4e93b17a2396cf22f500', 'request_id': 'req-1ed1175b-162a-4f25-8315-21973ac51729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:43.005213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] The token cache is being invalidate because project 15b15d63f37e4e93b17a2396cf22f500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:09:43.005558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:43.006258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1ed1175b-162a-4f25-8315-21973ac51729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:09:43.008271 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2388/4779] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:09:42 2022] DELETE /identity/v3/projects/15b15d63f37e4e93b17a2396cf22f500 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:09:43.013553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c78c5c-e93d-49c9-9196-a296026407b7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.013883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c78c5c-e93d-49c9-9196-a296026407b7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.014205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c78c5c-e93d-49c9-9196-a296026407b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.037447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-39c78c5c-e93d-49c9-9196-a296026407b7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:43.038774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-39c78c5c-e93d-49c9-9196-a296026407b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:43.160393 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2392/4780] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:09:43.168210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28bd9c-0290-4797-a2bd-239f366cabe1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.168514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28bd9c-0290-4797-a2bd-239f366cabe1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.168791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28bd9c-0290-4797-a2bd-239f366cabe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.194777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ca28bd9c-0290-4797-a2bd-239f366cabe1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:43.197178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ca28bd9c-0290-4797-a2bd-239f366cabe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:43.216182 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2389/4781] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:43.217000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c78c5c-e93d-49c9-9196-a296026407b7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.233516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c78c5c-e93d-49c9-9196-a296026407b7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:43.242269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:43.243347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.243956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.244385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.251173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:09:43.252102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:43.253993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:43.276733 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2393/4782] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:09:43.281336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28bd9c-0290-4797-a2bd-239f366cabe1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.294249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28bd9c-0290-4797-a2bd-239f366cabe1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:43.318166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:43.319052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.319474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.319948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.321319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:43.323579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:43.409624 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2390/4783] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:09:43 2022] POST /identity/v3/projects => generated 401 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:09:43.413860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.427276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348f8ca-0185-49ac-aaa2-288fba16303f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:43.502108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:43.503400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.504879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.505447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.506405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:43.509708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:43.520910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:43.549744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.553951 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2394/4784] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:09:43 2022] POST /identity/v3/users => generated 301 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:09:43.566895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503445b-3541-4b5f-a7f0-21773bb57afa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:43.574189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:43.575021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.575382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.575799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.576856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:43.578791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:43.596788 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2391/4785] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:09:43.598250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.610915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3b85f-30be-4ca1-9f7a-1e52e7306487 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:43.629630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2569b1-31ed-462a-b610-99c4e8f782ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:43.629630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2569b1-31ed-462a-b610-99c4e8f782ad admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.629630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2569b1-31ed-462a-b610-99c4e8f782ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.629630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2569b1-31ed-462a-b610-99c4e8f782ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.630030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2569b1-31ed-462a-b610-99c4e8f782ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:43.630030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2569b1-31ed-462a-b610-99c4e8f782ad admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:43.664854 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2395/4786] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:09:43.666149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.679923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd6da-3550-41b1-9534-c227179fd442 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:43.685233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:43.686079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.686421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.686704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] PATH_INFO: `/v3/projects/f0d230830b124d1ea4068261cb71702c/users/8c18a98e163f4789879dce15ed4402be/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.714006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d230830b124d1ea4068261cb71702c, user_id=8c18a98e163f4789879dce15ed4402be, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:43.717482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:43.750106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2392/4787] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:09:43 2022] PUT /identity/v3/projects/f0d230830b124d1ea4068261cb71702c/users/8c18a98e163f4789879dce15ed4402be/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:09:43.755326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b96d0-1e43-4f0c-8056-c3ac46fd763d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.755610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b96d0-1e43-4f0c-8056-c3ac46fd763d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.755883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b96d0-1e43-4f0c-8056-c3ac46fd763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.782858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-075b96d0-1e43-4f0c-8056-c3ac46fd763d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:43.784433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-075b96d0-1e43-4f0c-8056-c3ac46fd763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:43.916515 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2396/4788] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:09:43.927568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.931164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:44.031356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:44.132725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba5caa-af83-4748-a42f-1c243d8a055e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:44.142208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcacdc4-c9d0-47e4-aed2-485861f43778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:44.143166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcacdc4-c9d0-47e4-aed2-485861f43778 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:44.144251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcacdc4-c9d0-47e4-aed2-485861f43778 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:44.144677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcacdc4-c9d0-47e4-aed2-485861f43778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:44.167560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcacdc4-c9d0-47e4-aed2-485861f43778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:44.169214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcacdc4-c9d0-47e4-aed2-485861f43778 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:44.190321 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2393/4789] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:53.903635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=3,keystone:SELECT=34 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:54.182713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=14 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:59.963847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:59.964438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:59.964438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:00.000370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:00.005109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:00.196358 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2397/4790] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:01.492829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:01.493349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:01.493733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:01.585265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] MFA Rules not processed for user `8c18a98e163f4789879dce15ed4402be`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:01.587434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:01.809840 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2394/4791] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:02.766081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:02.769320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:03.255661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d694b-a11f-48f8-bb35-3b5e750fe2f9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:03.269920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43751e73-572b-42ba-8561-012f8fbd6068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:03.271362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43751e73-572b-42ba-8561-012f8fbd6068 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:03.274893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43751e73-572b-42ba-8561-012f8fbd6068 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:03.275190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43751e73-572b-42ba-8561-012f8fbd6068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:03.306033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43751e73-572b-42ba-8561-012f8fbd6068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:03.309025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43751e73-572b-42ba-8561-012f8fbd6068 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:03.358435 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2398/4792] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:02 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:06.449346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:06.452812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:06.537749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdc9b7-7c6f-42eb-bb90-dfefc1be8d02 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:06.554885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:06.556487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:06.562512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:06.564179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] PATH_INFO: `/v3/users/386e12cd145942648098c6569ccf79a7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:06.569575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=386e12cd145942648098c6569ccf79a7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:06.583347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:06.800813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '386e12cd145942648098c6569ccf79a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:06.843476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '386e12cd145942648098c6569ccf79a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:06.862142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '386e12cd145942648098c6569ccf79a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:06.863125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '386e12cd145942648098c6569ccf79a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:06.864023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '386e12cd145942648098c6569ccf79a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:06.923609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2395/4793] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:10:06 2022] DELETE /identity/v3/users/386e12cd145942648098c6569ccf79a7 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:06.928076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:06.928973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:06.929397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:06.955349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:06.956823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:07.063149 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2399/4794] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:07.081285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:07.087809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:07.223507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e38a7-8a0b-4b94-9a0a-8778fb3ad765 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:07.228522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e01305-f68e-4eef-b762-be3c485c6f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:07.229401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e01305-f68e-4eef-b762-be3c485c6f67 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:07.232282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e01305-f68e-4eef-b762-be3c485c6f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:07.232282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e01305-f68e-4eef-b762-be3c485c6f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:07.257700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e01305-f68e-4eef-b762-be3c485c6f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:07.261175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e01305-f68e-4eef-b762-be3c485c6f67 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:07.276140 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2396/4795] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:07.605366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:07.607129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:07.632426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2d69e-56eb-4508-bc77-c66bcafdbf1e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:07.642660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:07.643638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:07.644071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:07.644508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] PATH_INFO: `/v3/projects/9a2e9d7846c24a91addc9f43215b13d0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:07.645953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a2e9d7846c24a91addc9f43215b13d0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:07.648158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:07.714345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:07.791450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a2e9d7846c24a91addc9f43215b13d0', 'request_id': 'req-730dba84-5082-4095-8d1f-c4a8616e09e2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:07.805666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a2e9d7846c24a91addc9f43215b13d0', 'request_id': 'req-730dba84-5082-4095-8d1f-c4a8616e09e2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:07.825867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a2e9d7846c24a91addc9f43215b13d0', 'request_id': 'req-730dba84-5082-4095-8d1f-c4a8616e09e2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:07.886015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] The token cache is being invalidate because project 9a2e9d7846c24a91addc9f43215b13d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:10:07.886324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:07.892306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-730dba84-5082-4095-8d1f-c4a8616e09e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:07.919420 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2400/4796] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:10:07 2022] DELETE /identity/v3/projects/9a2e9d7846c24a91addc9f43215b13d0 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:07.926506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad7fb-c3b1-42d2-9895-9dd675c58b2f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:07.927006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad7fb-c3b1-42d2-9895-9dd675c58b2f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:07.927302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad7fb-c3b1-42d2-9895-9dd675c58b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:07.939576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:07.962730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7b0ad7fb-c3b1-42d2-9895-9dd675c58b2f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:07.966371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7b0ad7fb-c3b1-42d2-9895-9dd675c58b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:08.111038 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2397/4797] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:08.113123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18bada-59ac-4ceb-ae64-32a523400995 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.117089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18bada-59ac-4ceb-ae64-32a523400995 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.117089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18bada-59ac-4ceb-ae64-32a523400995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.158246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3c18bada-59ac-4ceb-ae64-32a523400995 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:08.159488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3c18bada-59ac-4ceb-ae64-32a523400995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:08.172215 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2401/4798] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:08.176145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad7fb-c3b1-42d2-9895-9dd675c58b2f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:08.209274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad7fb-c3b1-42d2-9895-9dd675c58b2f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:08.231204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:08.232010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.232422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.232986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.233671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:10:08.234077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:08.244317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:08.277374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18bada-59ac-4ceb-ae64-32a523400995 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:08.279209 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2398/4799] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:10:08.292551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18bada-59ac-4ceb-ae64-32a523400995 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:08.312715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:08.313844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.314289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.314793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.315675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:08.317963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:08.381279 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2402/4800] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:10:08 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:10:08.394112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:08.414547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e4c0-934a-4a57-ae80-c3530ec60dc0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:08.516152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:08.518310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.518675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.519055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.519884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:08.522871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:08.557491 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2399/4801] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:10:08 2022] POST /identity/v3/users => generated 282 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:08.559520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:08.576857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3550-0800-4f47-b97a-04b57f0cf1ea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:08.582641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:08.583433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.583822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.584181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.585047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:08.587581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:08.605867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2403/4802] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:08.610163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:08.622054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b7756-ef01-4209-af24-74880ad10b18 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:08.642734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa335c1d-1d40-4ffe-9691-635727ea3bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:08.643929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa335c1d-1d40-4ffe-9691-635727ea3bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.645710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa335c1d-1d40-4ffe-9691-635727ea3bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.646384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa335c1d-1d40-4ffe-9691-635727ea3bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.647281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa335c1d-1d40-4ffe-9691-635727ea3bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:08.652815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa335c1d-1d40-4ffe-9691-635727ea3bd0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:08.681054 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2400/4803] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:08.682299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:08.700291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8753ab4-8991-4af0-b0dc-c5e5935acd59 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:08.716710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:08.717714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.718142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.718563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] PATH_INFO: `/v3/projects/27a2296cdbb84eada80ed0322c8c680a/users/f3dd6e7db0af433d9746af0ec6ae6e08/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.758280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a2296cdbb84eada80ed0322c8c680a, user_id=f3dd6e7db0af433d9746af0ec6ae6e08, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:08.761198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:08.823317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2404/4804] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:10:08 2022] PUT /identity/v3/projects/27a2296cdbb84eada80ed0322c8c680a/users/f3dd6e7db0af433d9746af0ec6ae6e08/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:08.832850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f11ef-07d3-4f7e-b249-c0683f0cf17d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:08.833125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f11ef-07d3-4f7e-b249-c0683f0cf17d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:08.833396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f11ef-07d3-4f7e-b249-c0683f0cf17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:08.894738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-392f11ef-07d3-4f7e-b249-c0683f0cf17d None None] MFA Rules not processed for user `f3dd6e7db0af433d9746af0ec6ae6e08`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:08.896627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-392f11ef-07d3-4f7e-b249-c0683f0cf17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:09.013829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:09.023449 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2401/4805] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:09.044084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:09.046339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:09.128688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0ca0-fcd2-4343-8fe8-68f8b2c7c108 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:09.133666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:09.134292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:09.134572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:09.134977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:09.146773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:09.148156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:09.192723 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2405/4806] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:14.828504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:14.829063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:14.829063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:14.858288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:14.861828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:14.973626 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2402/4807] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:10:15.065562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:15.067764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:15.126565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:15.161718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c5fd6-698e-4ea2-bdfa-a2eeaf577def service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:15.169635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:15.170668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:15.171114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:15.171634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:15.182786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:15.185930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:15.217706 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2406/4808] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:10:15.436308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:15.439517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:15.519663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cc107-0d61-4eed-870a-59f2e4fbef52 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:15.528887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:15.529839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:15.530274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:15.530732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] PATH_INFO: `/v3/users/8c18a98e163f4789879dce15ed4402be` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:15.532077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c18a98e163f4789879dce15ed4402be)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:15.534322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:15.628312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c18a98e163f4789879dce15ed4402be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:15.659198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c18a98e163f4789879dce15ed4402be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:15.680983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c18a98e163f4789879dce15ed4402be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:15.681829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c18a98e163f4789879dce15ed4402be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:15.682462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f427ec04-3172-465d-9bfa-dbbdc5ab223b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c18a98e163f4789879dce15ed4402be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:15.708088 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2403/4809] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:10:15 2022] DELETE /identity/v3/users/8c18a98e163f4789879dce15ed4402be => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:16.143596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:16.172391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a22f2-30e9-49b4-a525-9fe6790a4a79 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:16.310322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:16.311085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:16.311358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:16.311670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] PATH_INFO: `/v3/projects/f0d230830b124d1ea4068261cb71702c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:16.314342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d230830b124d1ea4068261cb71702c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:16.316304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:16.474081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d230830b124d1ea4068261cb71702c', 'request_id': 'req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:16.491278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d230830b124d1ea4068261cb71702c', 'request_id': 'req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:16.509965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d230830b124d1ea4068261cb71702c', 'request_id': 'req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:16.539406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] The token cache is being invalidate because project f0d230830b124d1ea4068261cb71702c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:10:16.539954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:16.549199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22ea6e41-1dc2-4cc3-bd90-a8d3f63071ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:16.557450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2407/4810] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:10:16 2022] DELETE /identity/v3/projects/f0d230830b124d1ea4068261cb71702c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:16.563430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ec6de-d883-4833-a63b-344cb0c24a1f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:16.564017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ec6de-d883-4833-a63b-344cb0c24a1f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:16.564472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ec6de-d883-4833-a63b-344cb0c24a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:16.590949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-443ec6de-d883-4833-a63b-344cb0c24a1f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:16.593376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-443ec6de-d883-4833-a63b-344cb0c24a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:16.624612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:16.715979 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2404/4811] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:10:16.717995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b408-29c2-48cc-8230-024d03479c1a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:16.718293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b408-29c2-48cc-8230-024d03479c1a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:16.718581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b408-29c2-48cc-8230-024d03479c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:16.742474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0cb0b408-29c2-48cc-8230-024d03479c1a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:16.743704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0cb0b408-29c2-48cc-8230-024d03479c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:16.756131 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2408/4812] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:16.760672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ec6de-d883-4833-a63b-344cb0c24a1f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:16.772073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ec6de-d883-4833-a63b-344cb0c24a1f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:16.778530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:16.779610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:16.780165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:16.780934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:16.782008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:10:16.782506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:16.784449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:16.811148 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2405/4813] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:10:16.816489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0b408-29c2-48cc-8230-024d03479c1a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:16.835487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0b408-29c2-48cc-8230-024d03479c1a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:16.843460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:16.844924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:16.845244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:16.845605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:16.846415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:16.849210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:16.907075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:16.909671 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2409/4814] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:10:16 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:16.925754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7702f1-f0cd-47f1-ac9d-07e7cec7d342 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:17.032241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:17.033596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:17.034035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:17.037799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:17.038910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:17.041180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:17.078001 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2406/4815] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:10:16 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:17.080124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:17.096718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229156a-9208-4457-bea1-476a02f7fabe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:17.103120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:17.104074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:17.104584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:17.104947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:17.106273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:17.108261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:17.124128 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2410/4816] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:17.125505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:17.139254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf467a7-46ce-430e-9b03-9b7f0ab9fc0a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:17.146000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5046b-2eef-4cdc-badf-7dc61bdb021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:17.146878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5046b-2eef-4cdc-badf-7dc61bdb021c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:17.147414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5046b-2eef-4cdc-badf-7dc61bdb021c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:17.147798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5046b-2eef-4cdc-badf-7dc61bdb021c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:17.148605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5046b-2eef-4cdc-badf-7dc61bdb021c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:17.150887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5046b-2eef-4cdc-badf-7dc61bdb021c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:17.168635 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2407/4817] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:17.169970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:17.194705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269b11-701b-4438-925f-6fe3b489b0d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:17.203677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa22efe-95c2-44ec-bff6-b14682f9023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:17.204625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa22efe-95c2-44ec-bff6-b14682f9023b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:17.205106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa22efe-95c2-44ec-bff6-b14682f9023b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:17.206258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa22efe-95c2-44ec-bff6-b14682f9023b admin admin] PATH_INFO: `/v3/projects/4ceec62fde59414ebb34572d8f8fbcfe/users/1fbd80f0e5bf46b78a79fd821e9ff89e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:17.227216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa22efe-95c2-44ec-bff6-b14682f9023b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ceec62fde59414ebb34572d8f8fbcfe, user_id=1fbd80f0e5bf46b78a79fd821e9ff89e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:17.230252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa22efe-95c2-44ec-bff6-b14682f9023b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:17.245794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:17.266031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2411/4818] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:10:17 2022] PUT /identity/v3/projects/4ceec62fde59414ebb34572d8f8fbcfe/users/1fbd80f0e5bf46b78a79fd821e9ff89e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:10:17.266729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:17.267086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:17.267292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:17.303196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] MFA Rules not processed for user `1fbd80f0e5bf46b78a79fd821e9ff89e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:17.304088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:17.400810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:17.401483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:17.402111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:17.412146 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2408/4819] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:17.431517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] MFA Rules not processed for user `1fbd80f0e5bf46b78a79fd821e9ff89e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:17.434292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:17.447895 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2412/4820] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:17.710814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:17.714070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:17.757200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:17.864609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3bf0e-efb5-49cb-9a5e-b69857a96613 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:17.912229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:17.913105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:17.914153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:17.914700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:17.936662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:17.941956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:17.966763 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2409/4821] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:18.166386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:18.167856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:18.333313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:18.334878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:18.368998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c95faa-476b-4789-84ea-195c29cab352 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:18.385028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72ec13-37eb-43c7-b9de-78ccc2278fe7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:18.386012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ec13-37eb-43c7-b9de-78ccc2278fe7 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:18.386392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ec13-37eb-43c7-b9de-78ccc2278fe7 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:18.386757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72ec13-37eb-43c7-b9de-78ccc2278fe7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:18.392606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f72ec13-37eb-43c7-b9de-78ccc2278fe7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:18.412109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72ec13-37eb-43c7-b9de-78ccc2278fe7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:18.414556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72ec13-37eb-43c7-b9de-78ccc2278fe7 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:18.427542 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2413/4822] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:18.445068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283369b3-f5a8-472d-9119-9fd424b92f6e service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:18.450449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:18.451149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:18.451622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:18.451993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] PATH_INFO: `/v3/users/6d03e73a3bed48f6bf61a702f0f41658` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:18.453712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d03e73a3bed48f6bf61a702f0f41658)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:18.455893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:18.576838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d03e73a3bed48f6bf61a702f0f41658'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:18.592526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d03e73a3bed48f6bf61a702f0f41658'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:18.621744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d03e73a3bed48f6bf61a702f0f41658'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:18.622602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d03e73a3bed48f6bf61a702f0f41658'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:18.625329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d03e73a3bed48f6bf61a702f0f41658'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:18.656363 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2410/4823] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:10:18 2022] DELETE /identity/v3/users/6d03e73a3bed48f6bf61a702f0f41658 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:18.661617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:18.661930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:18.662693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:18.767220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:18.771417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:18.931981 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2414/4824] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:18.952637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:18.954382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:19.048741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:19.144410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7006a6-0785-4392-afb4-a5f034496346 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:19.156422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ea594-c1ec-4ab2-afc8-7a1da8defad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:19.157187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ea594-c1ec-4ab2-afc8-7a1da8defad6 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:19.157545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ea594-c1ec-4ab2-afc8-7a1da8defad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:19.157920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ea594-c1ec-4ab2-afc8-7a1da8defad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:19.197488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ea594-c1ec-4ab2-afc8-7a1da8defad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:19.201721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ea594-c1ec-4ab2-afc8-7a1da8defad6 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:19.245566 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2411/4825] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:19.663129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:19.665433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:19.695028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab448378-d97c-4b13-800a-fed87930c36b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:19.713154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:19.714892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:19.722178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:19.723687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] PATH_INFO: `/v3/projects/42e1e6b096bb41419c594dcde97e16e7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:19.728480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e1e6b096bb41419c594dcde97e16e7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:19.733206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:19.832545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e1e6b096bb41419c594dcde97e16e7', 'request_id': 'req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:19.842723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e1e6b096bb41419c594dcde97e16e7', 'request_id': 'req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:19.854061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e1e6b096bb41419c594dcde97e16e7', 'request_id': 'req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:19.887194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] The token cache is being invalidate because project 42e1e6b096bb41419c594dcde97e16e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:10:19.887559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:19.891814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8d4a3177-b8cf-4ed8-b563-527214d5ebc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:19.900945 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2415/4826] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:10:19 2022] DELETE /identity/v3/projects/42e1e6b096bb41419c594dcde97e16e7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:19.902559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5979fe-ba96-4081-81c2-4089636031a6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:19.902962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5979fe-ba96-4081-81c2-4089636031a6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:19.903364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5979fe-ba96-4081-81c2-4089636031a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:19.972335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ce5979fe-ba96-4081-81c2-4089636031a6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:19.982737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce5979fe-ba96-4081-81c2-4089636031a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:20.147070 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2412/4827] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:20.157043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b9e3-2040-4bfa-a584-66d5b27d1a85 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.157328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b9e3-2040-4bfa-a584-66d5b27d1a85 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.161709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0b9e3-2040-4bfa-a584-66d5b27d1a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.176610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:20.219843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-39a0b9e3-2040-4bfa-a584-66d5b27d1a85 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:20.222046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-39a0b9e3-2040-4bfa-a584-66d5b27d1a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:20.244535 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2416/4828] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:20.248863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5979fe-ba96-4081-81c2-4089636031a6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:20.269623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5979fe-ba96-4081-81c2-4089636031a6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:20.279965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:20.280891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.281332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.281838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.286457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:10:20.286949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:20.290045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:20.312121 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2413/4829] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:10:20.312689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0b9e3-2040-4bfa-a584-66d5b27d1a85 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:20.324032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0b9e3-2040-4bfa-a584-66d5b27d1a85 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:20.331358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:20.332422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.332854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.334574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.335448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:20.337926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:20.428804 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2417/4830] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:10:20 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:10:20.433259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:20.447193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c435e07-75d7-4594-b2fc-ca0fba3aaec7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:20.551307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:20.552096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.552442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.553325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.554017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:20.558271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:20.593334 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2414/4831] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:10:20 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:20.593969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:20.606033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566e318-4be4-4ad7-85e1-0825c144c3b9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:20.611241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:20.611996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.612379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.612878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.613631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:20.615308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:20.627316 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2418/4832] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:10:20.632139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:20.643946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db807d-1ad3-4bae-9b4b-87f9fc73db29 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:20.655117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596bdd3-77e9-45ad-8618-7f011d5bb3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:20.655812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596bdd3-77e9-45ad-8618-7f011d5bb3cf admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.656167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596bdd3-77e9-45ad-8618-7f011d5bb3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.656494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596bdd3-77e9-45ad-8618-7f011d5bb3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.657286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596bdd3-77e9-45ad-8618-7f011d5bb3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:20.659922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596bdd3-77e9-45ad-8618-7f011d5bb3cf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:20.675574 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2415/4833] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:20.680173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:20.697949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026617-05f3-4262-8f75-eddccc71a73f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:20.702850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:20.703594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.703944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.704295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] PATH_INFO: `/v3/projects/0a2922eb9e0245c19efd8ff93adf68a9/users/224b008680cf4312a5a4bec862e86960/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.721127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2922eb9e0245c19efd8ff93adf68a9, user_id=224b008680cf4312a5a4bec862e86960, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:20.723935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:20.739676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2419/4834] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:10:20 2022] PUT /identity/v3/projects/0a2922eb9e0245c19efd8ff93adf68a9/users/224b008680cf4312a5a4bec862e86960/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:20.753000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.753765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.753765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:20.788003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] MFA Rules not processed for user `224b008680cf4312a5a4bec862e86960`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:20.791501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:20.817051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:20.892405 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2416/4835] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:20.893392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:20.895534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:20.985922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2dc3a-a073-45f5-9ce3-e7d789bbf863 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:20.990761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292bd24-f3e8-48f8-8828-1ff876207b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:20.991912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292bd24-f3e8-48f8-8828-1ff876207b6a service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:20.992263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292bd24-f3e8-48f8-8828-1ff876207b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:20.992641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292bd24-f3e8-48f8-8828-1ff876207b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:21.006262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292bd24-f3e8-48f8-8828-1ff876207b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:21.008590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292bd24-f3e8-48f8-8828-1ff876207b6a service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:21.033669 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2420/4836] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:10:25.151396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:25.153100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:25.333877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d98d03-a49f-4149-a49f-c6859493b4b0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:25.346773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:25.347428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:25.347703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:25.347970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:25.349953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:25.553055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:25.557566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:25.583504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2417/4837] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:27.486119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119c1cf-399d-425c-922f-cc69cb250478 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:27.487005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119c1cf-399d-425c-922f-cc69cb250478 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:27.487418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119c1cf-399d-425c-922f-cc69cb250478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:27.534736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7119c1cf-399d-425c-922f-cc69cb250478 None None] MFA Rules not processed for user `c544e8e6f7a741c985c92ed7437397cd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:27.536369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7119c1cf-399d-425c-922f-cc69cb250478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:27.611593 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2421/4838] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:30.664913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:30.667076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:30.698472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d11f7-f7f7-47ca-af18-d279d73dfd27 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:30.704837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:30.705840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:30.706328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:30.718331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] PATH_INFO: `/v3/users/795813bb461e42ad8c2a758e9b8430d6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:30.721417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=795813bb461e42ad8c2a758e9b8430d6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:30.724746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:30.832028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795813bb461e42ad8c2a758e9b8430d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:30.883005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795813bb461e42ad8c2a758e9b8430d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:30.888084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:30.899193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795813bb461e42ad8c2a758e9b8430d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:30.899911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795813bb461e42ad8c2a758e9b8430d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:30.901137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e2cfc93c-d726-4d44-b839-4488b260b871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795813bb461e42ad8c2a758e9b8430d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:30.941366 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2418/4839] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:10:30 2022] DELETE /identity/v3/users/795813bb461e42ad8c2a758e9b8430d6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:31.940822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119c1cf-399d-425c-922f-cc69cb250478 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:31.974564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119c1cf-399d-425c-922f-cc69cb250478 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:32.071644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:32.096752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:32.097916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:32.098021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:32.098429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] PATH_INFO: `/v3/users/224b008680cf4312a5a4bec862e86960` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:32.100030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224b008680cf4312a5a4bec862e86960)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:32.102166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:32.279954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224b008680cf4312a5a4bec862e86960'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.280693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224b008680cf4312a5a4bec862e86960'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.299445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224b008680cf4312a5a4bec862e86960'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.300264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224b008680cf4312a5a4bec862e86960'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.326232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224b008680cf4312a5a4bec862e86960'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.355168 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2422/4840] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:10:31 2022] DELETE /identity/v3/users/224b008680cf4312a5a4bec862e86960 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:32.363858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:32.363858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:32.363858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:32.402248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:32.405075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:32.553403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:32.555875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:32.601556 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2419/4841] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:32.648555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:32.651586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:32.708375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579c3a-2909-46ee-b0a6-4037ffacbd22 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:32.716181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:32.717588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:32.718162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:32.719431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] PATH_INFO: `/v3/projects/f8ed24e4bf6643f7b1474fcec78c4995` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:32.722098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ed24e4bf6643f7b1474fcec78c4995)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:32.727014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:32.801315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246219aa-0d0d-4d04-9bfd-98e4a4176aac None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:32.812064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be0966-f580-4b3f-9cc7-e35a8cb544c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:32.812709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be0966-f580-4b3f-9cc7-e35a8cb544c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:32.812989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be0966-f580-4b3f-9cc7-e35a8cb544c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:32.813261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be0966-f580-4b3f-9cc7-e35a8cb544c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:32.836875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be0966-f580-4b3f-9cc7-e35a8cb544c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:32.838524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be0966-f580-4b3f-9cc7-e35a8cb544c8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:32.850029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ed24e4bf6643f7b1474fcec78c4995', 'request_id': 'req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.859885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ed24e4bf6643f7b1474fcec78c4995', 'request_id': 'req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.861368 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2420/4842] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:32.871124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ed24e4bf6643f7b1474fcec78c4995', 'request_id': 'req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.882944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:32.893382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] The token cache is being invalidate because project f8ed24e4bf6643f7b1474fcec78c4995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:10:32.894164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.894800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:32.901696 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2423/4843] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:10:32 2022] DELETE /identity/v3/projects/f8ed24e4bf6643f7b1474fcec78c4995 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:10:32.903791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986a85-ee8b-4941-b2f2-9c0e5cbf722c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:32.904093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986a85-ee8b-4941-b2f2-9c0e5cbf722c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:32.904516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0986a85-ee8b-4941-b2f2-9c0e5cbf722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:32.943309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d0986a85-ee8b-4941-b2f2-9c0e5cbf722c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:32.944548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d0986a85-ee8b-4941-b2f2-9c0e5cbf722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:32.974719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:32.976865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:33.072883 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2421/4844] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:33.081416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd3dec-dd27-4f2d-b666-e1efaca36308 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.081751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd3dec-dd27-4f2d-b666-e1efaca36308 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.082018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd3dec-dd27-4f2d-b666-e1efaca36308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.086132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747b2d9-cb14-48e9-9d6b-041e1efaa3d3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:33.097339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:33.098003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.098290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.098587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.101818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:33.123572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-38cd3dec-dd27-4f2d-b666-e1efaca36308 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:33.139965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-38cd3dec-dd27-4f2d-b666-e1efaca36308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:33.155897 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2422/4845] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:33.159918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd3dec-dd27-4f2d-b666-e1efaca36308 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:33.171389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd3dec-dd27-4f2d-b666-e1efaca36308 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:33.179962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:33.180839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.181182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.181958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.183797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:10:33.184301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:33.189918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:33.216698 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2423/4846] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:10:33.218561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:33.256780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883854-fab8-4304-a9ba-a888d361a7fa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:33.264882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:33.266376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.266506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.268898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.268898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:33.270027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:33.281139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:33.282086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:33.283892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:33.316721 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2424/4847] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:33.335181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2424/4848] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:10:33 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:33.339118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:33.360327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fd1d5-2206-44b5-9ba8-f86eff242e51 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:33.486035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:33.486307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.486587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.486860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.487511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:33.489619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:33.523355 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2425/4849] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:10:33 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:33.525278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:33.542816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d629f-966e-4233-8b33-a4d184344de2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:33.553879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:33.555382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.555831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.556339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.557354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:33.562131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:33.587165 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2425/4850] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:33.588175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:33.602209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebcadf-b372-4858-81c7-9bc59ccf31b0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:33.610512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a770d07-b6f9-4014-a4c3-4a641dcfa9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:33.610996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770d07-b6f9-4014-a4c3-4a641dcfa9df admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.611395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770d07-b6f9-4014-a4c3-4a641dcfa9df admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.611693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770d07-b6f9-4014-a4c3-4a641dcfa9df admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.612480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770d07-b6f9-4014-a4c3-4a641dcfa9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:33.614361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770d07-b6f9-4014-a4c3-4a641dcfa9df admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:33.649647 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2426/4851] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:10:33.653025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:33.667048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6851e9-6a5c-4d28-8d6c-28cdccb896ad admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:33.673104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14f2b4-2934-498f-b22c-65cc979d1a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:33.673953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f2b4-2934-498f-b22c-65cc979d1a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.674456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f2b4-2934-498f-b22c-65cc979d1a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.674964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f2b4-2934-498f-b22c-65cc979d1a0b admin admin] PATH_INFO: `/v3/projects/f7a3c50d58d347d0a4467cd4e61b06e0/users/e8b19d9480444a44a48ac3c5efe7fe17/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.696695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14f2b4-2934-498f-b22c-65cc979d1a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a3c50d58d347d0a4467cd4e61b06e0, user_id=e8b19d9480444a44a48ac3c5efe7fe17, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:33.699743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14f2b4-2934-498f-b22c-65cc979d1a0b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:33.717334 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2426/4852] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:10:33 2022] PUT /identity/v3/projects/f7a3c50d58d347d0a4467cd4e61b06e0/users/e8b19d9480444a44a48ac3c5efe7fe17/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:33.728760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.729271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.729607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.769699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] MFA Rules not processed for user `e8b19d9480444a44a48ac3c5efe7fe17`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:33.769699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:33.774001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:33.862832 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2427/4853] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:10:33.888025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:33.888435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:33.888707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:33.940087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] MFA Rules not processed for user `e8b19d9480444a44a48ac3c5efe7fe17`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:33.941328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:33.961857 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2427/4854] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:33.975761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:33.977263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:34.136007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ab2a6-297a-47d9-9d4b-15c28c51b5d5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:34.143645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:34.144709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:34.145158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:34.145612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:34.158816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:34.161002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:34.172843 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2428/4855] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:34.230954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:34.232737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:34.262202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:34.263844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:34.398973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed3f22-03ca-4aaf-83cf-c2733d4e1bff service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:34.405055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f7fde-a681-43be-bf1a-d75f4ee5918c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:34.407020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7fde-a681-43be-bf1a-d75f4ee5918c service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:34.407391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7fde-a681-43be-bf1a-d75f4ee5918c service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:34.407753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7fde-a681-43be-bf1a-d75f4ee5918c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:34.412528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-771f7fde-a681-43be-bf1a-d75f4ee5918c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:34.431576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f7fde-a681-43be-bf1a-d75f4ee5918c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:34.433381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f7fde-a681-43be-bf1a-d75f4ee5918c service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:34.436618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd780871-4f81-4e80-a1ad-1e3fdc1ff5a2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:34.442238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:34.442874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:34.443313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:34.443589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] PATH_INFO: `/v3/projects/0a2922eb9e0245c19efd8ff93adf68a9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:34.444737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2922eb9e0245c19efd8ff93adf68a9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:34.451799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:34.477960 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2429/4856] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:10:34.481081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7f87b-8d30-48ec-b0c3-b2d86522b5d0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:34.481380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7f87b-8d30-48ec-b0c3-b2d86522b5d0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:34.481700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7f87b-8d30-48ec-b0c3-b2d86522b5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:34.522554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2922eb9e0245c19efd8ff93adf68a9', 'request_id': 'req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:34.526285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-97d7f87b-8d30-48ec-b0c3-b2d86522b5d0 None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:34.527731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97d7f87b-8d30-48ec-b0c3-b2d86522b5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:34.535676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2922eb9e0245c19efd8ff93adf68a9', 'request_id': 'req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:34.544034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2922eb9e0245c19efd8ff93adf68a9', 'request_id': 'req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:34.560640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] The token cache is being invalidate because project 0a2922eb9e0245c19efd8ff93adf68a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:10:34.561715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:34.562831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9f621c6e-9ba8-4f08-a087-0c68fefd47cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:34.564481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:34.566828 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2428/4857] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:10:34 2022] DELETE /identity/v3/projects/0a2922eb9e0245c19efd8ff93adf68a9 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:34.572128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c6d13-e43d-469d-b24d-9a010b3a1cba None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:34.572904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c6d13-e43d-469d-b24d-9a010b3a1cba None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:34.573948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c6d13-e43d-469d-b24d-9a010b3a1cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:34.598149 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2430/4858] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:34.607636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-202c6d13-e43d-469d-b24d-9a010b3a1cba None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:34.609216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-202c6d13-e43d-469d-b24d-9a010b3a1cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:34.710336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:34.822264 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2429/4859] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:34.829335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240bd32-d5b2-4173-be34-1679c9006760 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:34.829678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240bd32-d5b2-4173-be34-1679c9006760 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:34.830257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240bd32-d5b2-4173-be34-1679c9006760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:34.893894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2240bd32-d5b2-4173-be34-1679c9006760 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:34.895579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2240bd32-d5b2-4173-be34-1679c9006760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:34.918961 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2431/4860] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:34.922030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c6d13-e43d-469d-b24d-9a010b3a1cba None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:34.944085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c6d13-e43d-469d-b24d-9a010b3a1cba None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:34.966037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:34.968332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:34.968620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:34.968888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:34.978034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:10:34.979077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:34.987271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:35.031060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2430/4861] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:10:35.032709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240bd32-d5b2-4173-be34-1679c9006760 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:35.043921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240bd32-d5b2-4173-be34-1679c9006760 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:35.055633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:35.056411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:35.056720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:35.057039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:35.057858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:35.064342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:35.135963 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2432/4862] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:10:35 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:35.139266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:35.155425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67ba6-277a-419d-b6c7-46503b4633be admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:35.358420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:35.372853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:35.374591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:35.378794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:35.379630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:35.382668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:35.448183 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2431/4863] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:10:35 2022] POST /identity/v3/users => generated 286 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:35.459449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:35.501626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c31ffa-ee65-4d52-84ee-a940b5d41550 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:35.513971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:35.514747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:35.515092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:35.515368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:35.516095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:35.517774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:35.546977 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2433/4864] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:35.551106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:35.568047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf938a-f200-439b-b9a3-a62653f4db19 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:35.593201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d082e9b-b2c3-423e-a85d-14e605ba27f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:35.598393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d082e9b-b2c3-423e-a85d-14e605ba27f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:35.599574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d082e9b-b2c3-423e-a85d-14e605ba27f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:35.600348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d082e9b-b2c3-423e-a85d-14e605ba27f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:35.603534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d082e9b-b2c3-423e-a85d-14e605ba27f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:35.608559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d082e9b-b2c3-423e-a85d-14e605ba27f5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:35.660956 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2432/4865] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:35.666760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:35.700581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cff26-d639-4bef-90d7-7ad1b8e6dac5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:35.719108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:35.724645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:35.724940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:35.728709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] PATH_INFO: `/v3/projects/93597f97e50a47eb9e718a168a11dad1/users/1c225ad7f31942838eb1391a988b59ab/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:35.812906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93597f97e50a47eb9e718a168a11dad1, user_id=1c225ad7f31942838eb1391a988b59ab, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:35.819083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:35.863468 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2434/4866] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:10:35 2022] PUT /identity/v3/projects/93597f97e50a47eb9e718a168a11dad1/users/1c225ad7f31942838eb1391a988b59ab/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:10:35.864181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad797e-b2fd-48f7-827a-43f142b17d51 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:35.864471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad797e-b2fd-48f7-827a-43f142b17d51 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:35.864770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad797e-b2fd-48f7-827a-43f142b17d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:35.918245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e6ad797e-b2fd-48f7-827a-43f142b17d51 None None] MFA Rules not processed for user `1c225ad7f31942838eb1391a988b59ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:35.919666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e6ad797e-b2fd-48f7-827a-43f142b17d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:36.027701 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2433/4867] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:36.040284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:36.043861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:36.148214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ec90f-31bc-4ac2-ac8c-9eade07c8759 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:36.155526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:36.156672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:36.156999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:36.157538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:36.199579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:36.206181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:36.240444 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2435/4868] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:41.583217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:41.583828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:41.583828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:41.664352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:41.686377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:41.690256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:41.851317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2434/4869] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:10:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:41.860198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:41.862009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:42.008856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced404d-25f2-4757-88c5-e63df4bafc2a service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:42.021814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:42.022810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:42.023248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:42.023815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:42.039861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:42.045707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:42.073727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:42.100756 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2436/4870] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:10:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:10:49.387761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:49.389860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:49.516273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99673e-2d6f-4729-91a4-3e50f78d8880 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:49.526212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:49.532258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:49.532892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:49.534719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:49.542805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:49.822432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:49.839112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:49.879901 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2435/4871] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:50.998893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:51.000917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:51.020339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff775-dcda-4818-93fb-70b75008f43f service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:51.029339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c772eb-a05e-42e9-9b5d-8db074429730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:51.030364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c772eb-a05e-42e9-9b5d-8db074429730 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:51.030880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c772eb-a05e-42e9-9b5d-8db074429730 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:51.031409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c772eb-a05e-42e9-9b5d-8db074429730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:51.035370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-56c772eb-a05e-42e9-9b5d-8db074429730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:51.048215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c772eb-a05e-42e9-9b5d-8db074429730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:51.050322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c772eb-a05e-42e9-9b5d-8db074429730 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:51.095406 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2437/4872] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:52.280402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:52.296882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5e60-5606-4fdc-af32-9e486348b791 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:52.312676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:52.313665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:52.314805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:52.315159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] PATH_INFO: `/v3/users/1c225ad7f31942838eb1391a988b59ab` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:52.316696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c225ad7f31942838eb1391a988b59ab)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:52.319697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:52.446683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c225ad7f31942838eb1391a988b59ab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:52.448187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:52.469903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c225ad7f31942838eb1391a988b59ab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:52.485079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c225ad7f31942838eb1391a988b59ab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:52.487789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c225ad7f31942838eb1391a988b59ab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:52.488714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c225ad7f31942838eb1391a988b59ab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:52.524656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2436/4873] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:10:52 2022] DELETE /identity/v3/users/1c225ad7f31942838eb1391a988b59ab => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:52.527262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:52.528400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:52.529089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:52.567088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:52.569868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:52.677066 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2438/4874] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:10:52.696742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:52.698300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:52.803610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f084fa-bd54-420b-aa47-e02245e84585 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:52.808986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86accab2-e63c-43a1-9a2e-31f61c78ef6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:52.810300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86accab2-e63c-43a1-9a2e-31f61c78ef6b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:52.810845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86accab2-e63c-43a1-9a2e-31f61c78ef6b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:52.812118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86accab2-e63c-43a1-9a2e-31f61c78ef6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:52.826372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86accab2-e63c-43a1-9a2e-31f61c78ef6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:52.828548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86accab2-e63c-43a1-9a2e-31f61c78ef6b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:52.853793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2437/4875] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:54.385781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:54.386392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:54.406412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd671e2-cbcb-46ef-8182-8629a87b0ed9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:54.416449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:54.416947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:54.417237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:54.417612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] PATH_INFO: `/v3/projects/93597f97e50a47eb9e718a168a11dad1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:54.420154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=93597f97e50a47eb9e718a168a11dad1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:54.422168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:54.531991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93597f97e50a47eb9e718a168a11dad1', 'request_id': 'req-65527f03-3a66-4ee6-b2eb-234faae63d4b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:54.539981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93597f97e50a47eb9e718a168a11dad1', 'request_id': 'req-65527f03-3a66-4ee6-b2eb-234faae63d4b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:54.553849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93597f97e50a47eb9e718a168a11dad1', 'request_id': 'req-65527f03-3a66-4ee6-b2eb-234faae63d4b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:54.583275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] The token cache is being invalidate because project 93597f97e50a47eb9e718a168a11dad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:10:54.583275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:54.588837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65527f03-3a66-4ee6-b2eb-234faae63d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:10:54.607547 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2439/4876] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:10:54 2022] DELETE /identity/v3/projects/93597f97e50a47eb9e718a168a11dad1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:54.613640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c200-6109-4f33-8117-9da0e68d8f36 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:54.614205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c200-6109-4f33-8117-9da0e68d8f36 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:54.614205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2c200-6109-4f33-8117-9da0e68d8f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:54.688741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d1f2c200-6109-4f33-8117-9da0e68d8f36 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:54.690218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1f2c200-6109-4f33-8117-9da0e68d8f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:54.786297 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2438/4877] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:54.795348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227146a1-e3cd-4969-ac73-ea9c5ede7a57 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:54.795586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227146a1-e3cd-4969-ac73-ea9c5ede7a57 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:54.796034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227146a1-e3cd-4969-ac73-ea9c5ede7a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:54.832853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-227146a1-e3cd-4969-ac73-ea9c5ede7a57 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:54.834516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-227146a1-e3cd-4969-ac73-ea9c5ede7a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:54.848430 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2440/4878] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:54.852603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2c200-6109-4f33-8117-9da0e68d8f36 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:54.863408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2c200-6109-4f33-8117-9da0e68d8f36 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:54.869074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:54.869738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:54.870016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:54.870284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:54.870959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:10:54.871254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:54.872959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:54.893972 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2439/4879] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:10:54.896353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227146a1-e3cd-4969-ac73-ea9c5ede7a57 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:54.911868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227146a1-e3cd-4969-ac73-ea9c5ede7a57 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:54.921443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:54.922393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:54.922685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:54.923115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:54.924285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:54.928413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:55.022879 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2441/4880] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:10:54 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:10:55.027201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:55.039401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f85dd-443b-4878-a7b3-1c09886b9f88 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:55.203029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:55.220555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:55.221586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:55.223213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:55.223213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:55.224077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:55.232250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:55.292004 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2440/4881] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:10:55 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:10:55.296454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:55.315980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b383b1-02b8-4085-9bc0-aee87a0da1e5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:55.326828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:55.328016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:55.328671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:55.329112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:55.330023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:55.332525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:55.348185 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2442/4882] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:55.353155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:55.368577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3559b-188b-496a-a851-a228ba5ac835 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:55.385783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08884f33-e6eb-499e-a98d-e485a7a7d799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:55.386746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08884f33-e6eb-499e-a98d-e485a7a7d799 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:55.387139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08884f33-e6eb-499e-a98d-e485a7a7d799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:55.387510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08884f33-e6eb-499e-a98d-e485a7a7d799 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:55.388814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08884f33-e6eb-499e-a98d-e485a7a7d799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:55.390680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08884f33-e6eb-499e-a98d-e485a7a7d799 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:55.413514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2441/4883] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:10:55.417957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:55.434558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b14e10-fe27-4125-a9cb-2f6e3bbc3731 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:55.444334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:55.445612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:55.446100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:55.446572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] PATH_INFO: `/v3/projects/9818fed65901412b8da672532cc5bc52/users/7e12ad7a93ae4a9a95b2a54c1a2b9845/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:55.476529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9818fed65901412b8da672532cc5bc52, user_id=7e12ad7a93ae4a9a95b2a54c1a2b9845, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:55.479571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:55.536286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2443/4884] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:10:55 2022] PUT /identity/v3/projects/9818fed65901412b8da672532cc5bc52/users/7e12ad7a93ae4a9a95b2a54c1a2b9845/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:10:55.547738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f27e37-665b-4ca3-8a5a-377a497b69e6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:55.548403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f27e37-665b-4ca3-8a5a-377a497b69e6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:55.548856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f27e37-665b-4ca3-8a5a-377a497b69e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:55.630849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-96f27e37-665b-4ca3-8a5a-377a497b69e6 None None] MFA Rules not processed for user `7e12ad7a93ae4a9a95b2a54c1a2b9845`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:55.632088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-96f27e37-665b-4ca3-8a5a-377a497b69e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:55.801764 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2442/4885] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:10:55.818716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:55.860230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:55.887050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:56.009100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a4f51-b1fb-4913-aae7-20b6ce2c141f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:56.019689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4572716-1a50-4128-9ab6-e431a84a4062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:56.020704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4572716-1a50-4128-9ab6-e431a84a4062 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:56.020983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4572716-1a50-4128-9ab6-e431a84a4062 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:56.021282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4572716-1a50-4128-9ab6-e431a84a4062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:56.041230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4572716-1a50-4128-9ab6-e431a84a4062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:56.042742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4572716-1a50-4128-9ab6-e431a84a4062 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:56.066474 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2444/4886] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:57.569438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:57.570452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:57.571175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:57.616796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:57.617869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:57.752684 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2443/4887] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:57.930574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:57.931595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:57.932379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:57.971058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] MFA Rules not processed for user `7e12ad7a93ae4a9a95b2a54c1a2b9845`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:57.975852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:57.996227 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2445/4888] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:10:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:59.174604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:59.180915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:59.437157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20b387-cb1d-41fa-9073-a4fd3db929ff None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:59.458293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:59.459244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:59.459667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:59.460150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:59.462920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:59.511365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:59.532178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:59.551904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:59.597718 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2444/4889] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:11:00.920386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:00.922233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:01.084653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462b6fe-f0f1-430a-989e-2cd0db0932c4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:01.103493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:01.104856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:01.105374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:01.105895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] PATH_INFO: `/v3/users/2ef547e071fd43ec96c745c26eca119f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:01.107538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef547e071fd43ec96c745c26eca119f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:01.109843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:01.218503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef547e071fd43ec96c745c26eca119f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.219431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef547e071fd43ec96c745c26eca119f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.239810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef547e071fd43ec96c745c26eca119f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.241701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef547e071fd43ec96c745c26eca119f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.261280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2036f8ac-3dce-4810-aa59-a6d199eefbd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef547e071fd43ec96c745c26eca119f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.279735 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2446/4890] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:00 2022] DELETE /identity/v3/users/2ef547e071fd43ec96c745c26eca119f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:01.642136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:01.667989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8973db-9302-4298-8902-b2c37deea3ba service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:01.781198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:01.782067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:01.782535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:01.783059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] PATH_INFO: `/v3/projects/3e48e3c2f7f74636a75679fdf10ef33a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:01.784527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e48e3c2f7f74636a75679fdf10ef33a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:01.786651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:01.943525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e48e3c2f7f74636a75679fdf10ef33a', 'request_id': 'req-1f21fb70-05a5-44f2-89fc-d8b96db41c39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.967668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e48e3c2f7f74636a75679fdf10ef33a', 'request_id': 'req-1f21fb70-05a5-44f2-89fc-d8b96db41c39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.981709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e48e3c2f7f74636a75679fdf10ef33a', 'request_id': 'req-1f21fb70-05a5-44f2-89fc-d8b96db41c39'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.997249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] The token cache is being invalidate because project 3e48e3c2f7f74636a75679fdf10ef33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:01.998288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:01.999132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1f21fb70-05a5-44f2-89fc-d8b96db41c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:02.009913 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2445/4891] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:01 2022] DELETE /identity/v3/projects/3e48e3c2f7f74636a75679fdf10ef33a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:02.010191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05469697-c2d3-4ada-ac34-80de8623ab49 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.010657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05469697-c2d3-4ada-ac34-80de8623ab49 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.010952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05469697-c2d3-4ada-ac34-80de8623ab49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:02.061395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-05469697-c2d3-4ada-ac34-80de8623ab49 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:02.065071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-05469697-c2d3-4ada-ac34-80de8623ab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:02.108327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:02.236412 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2447/4892] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:02.245708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a2fee-25e2-4c0c-920c-ca74b4735f4f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.246290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a2fee-25e2-4c0c-920c-ca74b4735f4f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.246835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a2fee-25e2-4c0c-920c-ca74b4735f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:02.276447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bf2a2fee-25e2-4c0c-920c-ca74b4735f4f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:02.278109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bf2a2fee-25e2-4c0c-920c-ca74b4735f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:02.292345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05469697-c2d3-4ada-ac34-80de8623ab49 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:02.294758 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2446/4893] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:02.358652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05469697-c2d3-4ada-ac34-80de8623ab49 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:02.395374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:02.406253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.408493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.409550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:02.412377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:02.416522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:02.424460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:02.512628 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2448/4894] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:02.521855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a2fee-25e2-4c0c-920c-ca74b4735f4f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:02.548798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a2fee-25e2-4c0c-920c-ca74b4735f4f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:02.555251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:02.567011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.567011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.567011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:02.567011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:02.567011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:02.647738 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2447/4895] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:02 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:02.652709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:02.666420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636d3af-ac60-442d-b53b-c343e8094211 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:02.788529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:02.790064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.790405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.791403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:02.795474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:02.797926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:02.849833 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2449/4896] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:02 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:02.851666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:02.870478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc9a63-27c2-4e29-9b84-d8ba036c00ac admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:02.878546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:02.879307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.879706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.880048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:02.881006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:02.883333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:02.902580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:02.925866 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2448/4897] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:02.932919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bd360-3e21-4602-a9a5-d86272a178ab admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:02.946248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a790b-1d0b-45cc-af0f-8c537c6d9064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:02.946892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a790b-1d0b-45cc-af0f-8c537c6d9064 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.947172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a790b-1d0b-45cc-af0f-8c537c6d9064 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.952072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a790b-1d0b-45cc-af0f-8c537c6d9064 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:02.952895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a790b-1d0b-45cc-af0f-8c537c6d9064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:02.954812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a790b-1d0b-45cc-af0f-8c537c6d9064 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:02.975032 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2450/4898] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:02.975549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:02.989103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee1305-8cb3-4839-86b2-8b22a8af5882 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:02.994479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:02.995637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:02.996140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:02.996747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] PATH_INFO: `/v3/projects/68c1204ce7704bf7b519e2f28025af83/users/c5823f7af890462c9d489b826c5e62b7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:03.022250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=68c1204ce7704bf7b519e2f28025af83, user_id=c5823f7af890462c9d489b826c5e62b7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:03.025942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:03.064801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724997-235d-4919-af82-7738879ad8c2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:03.065933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724997-235d-4919-af82-7738879ad8c2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:03.066431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724997-235d-4919-af82-7738879ad8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:03.070013 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2449/4899] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:02 2022] PUT /identity/v3/projects/68c1204ce7704bf7b519e2f28025af83/users/c5823f7af890462c9d489b826c5e62b7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:03.111746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1e724997-235d-4919-af82-7738879ad8c2 None None] MFA Rules not processed for user `c5823f7af890462c9d489b826c5e62b7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:03.114956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1e724997-235d-4919-af82-7738879ad8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:03.305543 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2451/4900] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:03.312447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:03.410648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:03.480012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278c137-3eec-4d4f-87c6-e945b8c762bd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:03.503869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:03.505922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:03.506366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:03.506809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:03.508107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:03.514284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:03.619020 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2450/4901] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:03 2022] POST /identity/v3/projects => generated 383 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:03.619486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724997-235d-4919-af82-7738879ad8c2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:03.634936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724997-235d-4919-af82-7738879ad8c2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:03.659101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:03.829563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:03.830275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:03.831834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:03.832190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:03.834056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:03.839693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:03.873152 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2452/4902] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:03 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:03.878109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:03.896287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a62c4-df81-4f08-b4c4-54eefb5792f9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:03.912091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:03.912727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:03.913570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:03.914098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:03.915598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:03.922553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:03.952459 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2451/4903] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:03.953720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:03.972285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca160ba-72ea-459b-b55e-594c8b8788a9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:03.988592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:03.990317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:03.992805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:03.993437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] PATH_INFO: `/v3/projects/71e16681780b4537bc6751884befeb8e/users/e304425f226a41708b8ac00cba334e88/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.033252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e16681780b4537bc6751884befeb8e, user_id=e304425f226a41708b8ac00cba334e88, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.049258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.094316 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2453/4904] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:03 2022] PUT /identity/v3/projects/71e16681780b4537bc6751884befeb8e/users/e304425f226a41708b8ac00cba334e88/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:04.098700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:04.116477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844f5f7-0dec-486a-ae3a-5bac46aeaefd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:04.166868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:04.169357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:04.252330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:04.253261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.259161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.259820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.260897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.262906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.303798 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2452/4905] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:04.306705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:04.310041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fdf80-8e75-44cb-bffb-bf7f71d3a33a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:04.319438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d4f23-2851-41c1-8f58-5dc1fea59f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:04.320446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d4f23-2851-41c1-8f58-5dc1fea59f28 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.320878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d4f23-2851-41c1-8f58-5dc1fea59f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.321530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d4f23-2851-41c1-8f58-5dc1fea59f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.323483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e89d4f23-2851-41c1-8f58-5dc1fea59f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:04.323917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b012585-66f2-4fc0-9066-f85196e9b8ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:04.333036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:04.334818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.335354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.335810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] PATH_INFO: `/v3/projects/71e16681780b4537bc6751884befeb8e/users/e304425f226a41708b8ac00cba334e88/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.337193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d4f23-2851-41c1-8f58-5dc1fea59f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.338723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d4f23-2851-41c1-8f58-5dc1fea59f28 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.339756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e16681780b4537bc6751884befeb8e, user_id=e304425f226a41708b8ac00cba334e88, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.343819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.349520 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2454/4906] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:11:04.382791 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2453/4907] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:04 2022] PUT /identity/v3/projects/71e16681780b4537bc6751884befeb8e/users/e304425f226a41708b8ac00cba334e88/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:04.386528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87475c8-65b0-4d00-b5b9-d03d9a24a335 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.387046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87475c8-65b0-4d00-b5b9-d03d9a24a335 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.387475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87475c8-65b0-4d00-b5b9-d03d9a24a335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.442163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a87475c8-65b0-4d00-b5b9-d03d9a24a335 None None] MFA Rules not processed for user `e304425f226a41708b8ac00cba334e88`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:04.444185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a87475c8-65b0-4d00-b5b9-d03d9a24a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:04.541391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2455/4908] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:04.546660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:04.566290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebdfce-fbe6-43fd-8d82-8b64563d0611 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:04.576752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:04.577757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.578123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.578396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.579086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.582187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invalid input for field 'name': '' is too short Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: On instance['name']: Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: On instance['name']: Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: '' Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise error Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application '' Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:04.626866 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:11:04.629354 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application '' Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:04.632554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2454/4909] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:11:04 2022] POST /identity/v3/domains => generated 283 bytes in 82 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 17 21:11:04.633616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87475c8-65b0-4d00-b5b9-d03d9a24a335 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:04.649023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87475c8-65b0-4d00-b5b9-d03d9a24a335 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:04.661669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:04.662309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.663208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.663208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.664081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.666046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: On instance['name']: Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: On instance['name']: Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise error Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.682780 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 17 21:11:04.686188 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 17 21:11:04.687828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.689256 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2456/4910] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:11:04 2022] POST /identity/v3/domains => generated 412 bytes in 56 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 17 21:11:04.714572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:04.728436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893feac-816e-4388-9349-e8dde9256c91 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:04.734327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:04.735539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.735539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.735817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.736609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.739148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.767351 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2455/4911] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:11:04 2022] POST /identity/v3/domains => generated 276 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:04.768856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:04.784983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae4f9b-6485-4043-bfe1-1d29c773ee36 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:04.791067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:04.919026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:04.919541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:04.919913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:04.920270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains/1e402440619f4e6bb86d51748e9a83a5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:04.938442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=1e402440619f4e6bb86d51748e9a83a5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:04.940829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Sep 17 21:11:04.974583 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 17 21:11:04.977121 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:04.978472 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2457/4912] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:11:04 2022] DELETE /identity/v3/domains/1e402440619f4e6bb86d51748e9a83a5 => generated 120 bytes in 211 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:11:04.982720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:05.003723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d37839-2546-4c1c-afbb-1d65d1beeb08 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:05.017374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:05.023605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:05.024024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:05.024328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains/1e402440619f4e6bb86d51748e9a83a5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:05.027460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=1e402440619f4e6bb86d51748e9a83a5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:05.030497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:05.074066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.074885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.119064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.120118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.125512 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2456/4913] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:11:04 2022] PATCH /identity/v3/domains/1e402440619f4e6bb86d51748e9a83a5 => generated 277 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:05.130443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:05.132474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:05.151044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a5626-35ca-43ee-9867-7abcb0a12778 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:05.243222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:05.244473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:05.245227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:05.245814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains/1e402440619f4e6bb86d51748e9a83a5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:05.274149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=1e402440619f4e6bb86d51748e9a83a5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:05.276228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:05.320873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.329905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.487980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5', 'request_id': 'req-82dc5689-8f70-4a22-8f31-38a7ff367c14'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.498631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5', 'request_id': 'req-82dc5689-8f70-4a22-8f31-38a7ff367c14'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.516017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5', 'request_id': 'req-82dc5689-8f70-4a22-8f31-38a7ff367c14'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.517701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:05.535807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] The token cache is being invalidate because project 1e402440619f4e6bb86d51748e9a83a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:05.536006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.537427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.557026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5', 'request_id': 'req-82dc5689-8f70-4a22-8f31-38a7ff367c14'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.563035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5', 'request_id': 'req-82dc5689-8f70-4a22-8f31-38a7ff367c14'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.564978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '1e402440619f4e6bb86d51748e9a83a5', 'request_id': 'req-82dc5689-8f70-4a22-8f31-38a7ff367c14'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:05.593063 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2458/4914] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:11:05 2022] DELETE /identity/v3/domains/1e402440619f4e6bb86d51748e9a83a5 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:05.595686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:05.615204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbc3a2-7d4c-4994-b9a6-ac203ddd4e11 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:05.661497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:05.836929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:05.837997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:05.838438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:05.838891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains/ff1546845c9947e4ae6a6c0304c61e32` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:05.850011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Could not find domain: ff1546845c9947e4ae6a6c0304c61e32.: keystone.exception.DomainNotFound: Could not find domain: ff1546845c9947e4ae6a6c0304c61e32. Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ff1546845c9947e4ae6a6c0304c61e32. Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:05.856444 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:11:05.857598 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:11:05.859056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: ff1546845c9947e4ae6a6c0304c61e32. Sep 17 21:11:05.862379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:05.862379 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2457/4915] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:11:05 2022] DELETE /identity/v3/domains/ff1546845c9947e4ae6a6c0304c61e32 => generated 112 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:11:05.863412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:05.865135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:05.883868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc5689-8f70-4a22-8f31-38a7ff367c14 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:05.892071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77630b-ce55-4b41-ac5e-4a66c18a54ef tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:05.893452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77630b-ce55-4b41-ac5e-4a66c18a54ef tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:05.893452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77630b-ce55-4b41-ac5e-4a66c18a54ef tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:05.894744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77630b-ce55-4b41-ac5e-4a66c18a54ef tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:05.897738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77630b-ce55-4b41-ac5e-4a66c18a54ef tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:05.902205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77630b-ce55-4b41-ac5e-4a66c18a54ef tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:05.929728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:05.933628 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2459/4916] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:11:05 2022] POST /identity/v3/domains => generated 252 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:05.948431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78231-6dea-4e86-8e06-2eabe9936f8f tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:06.115349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:06.117022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:06.117307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:06.118959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:06.119814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:06.128936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:06.141570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:06.142614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:06.143057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:06.149157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.sql.core [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-598233104' for key 'project.ixu_project_name_domain_id'") Sep 17 21:11:06.149157 ncdv-master-1 devstack@keystone.service[99166]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 21:11:06.149157 ncdv-master-1 devstack@keystone.service[99166]: [parameters: {'id': '3c38ffa558da40a69f6a2e6eec5a2e68', 'name': 'tempest-domain-dup-598233104', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 17 21:11:06.149157 ncdv-master-1 devstack@keystone.service[99166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 21:11:06.170587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:06.172511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-598233104.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-598233104. Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 21:11:06.204780 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-598233104' for key 'project.ixu_project_name_domain_id'") Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 17 21:11:06.205992 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application yield resource Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise self.value Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application session.commit() Sep 17 21:11:06.207407 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.flush() Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application compat.raise_( Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 21:11:06.209593 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 21:11:06.211138 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 21:11:06.212427 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-598233104' for key 'project.ixu_project_name_domain_id'") Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [parameters: {'id': '3c38ffa558da40a69f6a2e6eec5a2e68', 'name': 'tempest-domain-dup-598233104', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:06.213785 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-598233104. Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:06.215141 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-598233104. Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:06.218093 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2458/4917] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:11:05 2022] POST /identity/v3/domains => generated 211 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 21:11:06.219963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:06.247753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c94f-f199-4e2f-a508-eea6c486f499 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:06.259867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:06.263479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:06.265110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:06.265552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains/52e50d68d7a04ca1a00c7eba865b12f7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:06.285035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=52e50d68d7a04ca1a00c7eba865b12f7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:06.287687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:06.305384 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2460/4918] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:06.306327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:06.307735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:06.325440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:06.326908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:06.418041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:06.420951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:06.425517 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2459/4919] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:11:06 2022] PATCH /identity/v3/domains/52e50d68d7a04ca1a00c7eba865b12f7 => generated 253 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:06.432539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:06.434416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:06.442139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a68ed0-efbd-4a48-ac92-4716ac056d0c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:06.548003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:06.574797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:06.575356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:06.575628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:06.575902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:06.578102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:06.602039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac9b6c-f064-4b2a-b69a-6b76b0210bc5 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:06.607392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:06.610792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:06.619315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:06.619885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:06.620161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:06.620435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:06.627034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:06.641240 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2461/4920] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:06.646138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:06.646335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:06.649041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:06.759600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:06.762526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:06.784807 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2460/4921] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:11:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:11:06.787747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff7cd-f77d-4b5e-afe3-03c29c5138a8 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:06.793346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:06.794381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:06.794381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:06.794746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains/52e50d68d7a04ca1a00c7eba865b12f7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:06.810687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=52e50d68d7a04ca1a00c7eba865b12f7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:06.812628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:06.847940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:06.852532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:06.965771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7', 'request_id': 'req-575a925b-43fc-4160-9eaa-a7160fecad27'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:06.984268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7', 'request_id': 'req-575a925b-43fc-4160-9eaa-a7160fecad27'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.007279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7', 'request_id': 'req-575a925b-43fc-4160-9eaa-a7160fecad27'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.028969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] The token cache is being invalidate because project 52e50d68d7a04ca1a00c7eba865b12f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:07.029264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.030160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.051026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7', 'request_id': 'req-575a925b-43fc-4160-9eaa-a7160fecad27'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.058011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7', 'request_id': 'req-575a925b-43fc-4160-9eaa-a7160fecad27'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.058827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '52e50d68d7a04ca1a00c7eba865b12f7', 'request_id': 'req-575a925b-43fc-4160-9eaa-a7160fecad27'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.080135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2462/4922] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:11:06 2022] DELETE /identity/v3/domains/52e50d68d7a04ca1a00c7eba865b12f7 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:07.085359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:07.101644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feca958-330d-411b-8a4c-206470578fc8 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:07.184257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cb846-1908-4cfb-b2f7-994c9d52eadb tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e304425f226a41708b8ac00cba334e88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e16681780b4537bc6751884befeb8e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:07.185094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cb846-1908-4cfb-b2f7-994c9d52eadb tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:07.185277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cb846-1908-4cfb-b2f7-994c9d52eadb tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:07.186015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cb846-1908-4cfb-b2f7-994c9d52eadb tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] PATH_INFO: `/v3/domains/1e402440619f4e6bb86d51748e9a83a5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:07.201324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341cb846-1908-4cfb-b2f7-994c9d52eadb tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-341cb846-1908-4cfb-b2f7-994c9d52eadb tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Could not find domain: 1e402440619f4e6bb86d51748e9a83a5.: keystone.exception.DomainNotFound: Could not find domain: 1e402440619f4e6bb86d51748e9a83a5. Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 1e402440619f4e6bb86d51748e9a83a5. Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:07.210379 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:11:07.211581 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:11:07.215176 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 1e402440619f4e6bb86d51748e9a83a5. Sep 17 21:11:07.217613 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:07.217613 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2461/4923] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:11:07 2022] PATCH /identity/v3/domains/1e402440619f4e6bb86d51748e9a83a5 => generated 112 bytes in 127 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:11:07.217877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:07.217877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:07.282211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:07.353454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a925b-43fc-4160-9eaa-a7160fecad27 tempest-DomainsNegativeTestJSON-1306668795 tempest-DomainsNegativeTestJSON-1306668795-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:07.358981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:07.359899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:07.360343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:07.360778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] PATH_INFO: `/v3/users/c5823f7af890462c9d489b826c5e62b7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:07.362565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5823f7af890462c9d489b826c5e62b7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:07.364832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:07.464169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5823f7af890462c9d489b826c5e62b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.465125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5823f7af890462c9d489b826c5e62b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.482820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5823f7af890462c9d489b826c5e62b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.484112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5823f7af890462c9d489b826c5e62b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.514455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5823f7af890462c9d489b826c5e62b7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:07.537738 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2463/4924] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:07 2022] DELETE /identity/v3/users/c5823f7af890462c9d489b826c5e62b7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:07.539842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:07.540219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:07.540576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:07.570566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:07.571913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:07.658508 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2462/4925] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:07.682306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:07.684310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:07.793727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96287687-36f9-48c4-9a68-e99d49bb69cc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:07.802570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:07.803277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:07.803561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:07.803850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:07.821236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:07.822810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:07.834676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2464/4926] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:08.344452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:08.346959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:08.363873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab0d20-2fb4-4b56-86b0-423378b1d8cc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:08.385351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:08.386725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:08.387008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:08.390615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] PATH_INFO: `/v3/projects/68c1204ce7704bf7b519e2f28025af83` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:08.393199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68c1204ce7704bf7b519e2f28025af83)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:08.395621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:08.523208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:08.533962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68c1204ce7704bf7b519e2f28025af83', 'request_id': 'req-ef5a446a-e429-4cd8-8049-3dd8dca54511'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.552047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68c1204ce7704bf7b519e2f28025af83', 'request_id': 'req-ef5a446a-e429-4cd8-8049-3dd8dca54511'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.564933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68c1204ce7704bf7b519e2f28025af83', 'request_id': 'req-ef5a446a-e429-4cd8-8049-3dd8dca54511'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.581790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] The token cache is being invalidate because project 68c1204ce7704bf7b519e2f28025af83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:08.582440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.583485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.588056 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2463/4927] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:08 2022] DELETE /identity/v3/projects/68c1204ce7704bf7b519e2f28025af83 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:08.592444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:08.607503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf3cf4-9ca5-402a-8a28-677f9d50d251 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:08.712783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:08.741682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:08.742891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:08.743189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:08.743495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] PATH_INFO: `/v3/users/e304425f226a41708b8ac00cba334e88` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:08.744943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e304425f226a41708b8ac00cba334e88)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:08.748252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:08.854237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e304425f226a41708b8ac00cba334e88'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.855271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e304425f226a41708b8ac00cba334e88'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.874631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e304425f226a41708b8ac00cba334e88'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.875766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e304425f226a41708b8ac00cba334e88'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.897774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e9f11ef3-d209-4812-9ac7-00ca38483e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e304425f226a41708b8ac00cba334e88'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:08.929864 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2465/4928] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:08 2022] DELETE /identity/v3/users/e304425f226a41708b8ac00cba334e88 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:09.760308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:09.762038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:09.786344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a446a-e429-4cd8-8049-3dd8dca54511 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:09.873692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:09.874695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:09.875362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:09.875630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] PATH_INFO: `/v3/projects/71e16681780b4537bc6751884befeb8e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:09.877941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e16681780b4537bc6751884befeb8e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:09.881029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:09.948313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e16681780b4537bc6751884befeb8e', 'request_id': 'req-22641a1f-8bb3-4288-9adc-945eb5355290'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:09.957901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e16681780b4537bc6751884befeb8e', 'request_id': 'req-22641a1f-8bb3-4288-9adc-945eb5355290'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:09.965000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e16681780b4537bc6751884befeb8e', 'request_id': 'req-22641a1f-8bb3-4288-9adc-945eb5355290'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:09.980722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] The token cache is being invalidate because project 71e16681780b4537bc6751884befeb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:09.981022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:09.982163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22641a1f-8bb3-4288-9adc-945eb5355290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:09.984637 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2464/4929] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:09 2022] DELETE /identity/v3/projects/71e16681780b4537bc6751884befeb8e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:09.989986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d13e2-943d-411e-a63b-d83add5811a0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:09.990598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d13e2-943d-411e-a63b-d83add5811a0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:09.990598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d13e2-943d-411e-a63b-d83add5811a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.026609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d23d13e2-943d-411e-a63b-d83add5811a0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:10.027849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d23d13e2-943d-411e-a63b-d83add5811a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:10.102348 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2466/4930] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:10.111067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a109d-d252-44ef-a43d-004a1c3cf62d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.111570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a109d-d252-44ef-a43d-004a1c3cf62d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.111904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a109d-d252-44ef-a43d-004a1c3cf62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.135174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b23a109d-d252-44ef-a43d-004a1c3cf62d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:10.136401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b23a109d-d252-44ef-a43d-004a1c3cf62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:10.145343 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2465/4931] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:10.149788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d13e2-943d-411e-a63b-d83add5811a0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.161153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d13e2-943d-411e-a63b-d83add5811a0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:10.168744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:10.169564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.169859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.170477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.171045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:10.171417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:10.173816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:10.192927 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2467/4932] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:10.197899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a109d-d252-44ef-a43d-004a1c3cf62d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.209227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a109d-d252-44ef-a43d-004a1c3cf62d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:10.216385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:10.217100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.217511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.218213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.218574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:10.220732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:10.262755 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2466/4933] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:10 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:10.264019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.277301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51611bc0-3a40-4d71-9d71-c6ed04f96a72 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:10.343278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:10.359296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:10.360685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.360685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.360685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.361426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:10.363550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:10.391502 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2468/4934] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:10 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:10.395605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.409229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5146bb-5009-4d30-80cb-11e44140f84d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:10.414345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:10.414493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.414760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.415122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.415792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:10.417814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:10.438609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2467/4935] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:10.442665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.455873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74e01c-2fad-4186-8bc0-381469c26afe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:10.463743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef6b94-f73e-4e7f-bc00-626170d62bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:10.464378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6b94-f73e-4e7f-bc00-626170d62bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.464676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6b94-f73e-4e7f-bc00-626170d62bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.465245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6b94-f73e-4e7f-bc00-626170d62bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.466074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef6b94-f73e-4e7f-bc00-626170d62bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:10.468300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef6b94-f73e-4e7f-bc00-626170d62bd7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:10.494894 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2469/4936] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:10.501169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.519298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6e2ad-888b-4e55-90f6-ac6bf4879599 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:10.525945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:10.528126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.528295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.528881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] PATH_INFO: `/v3/projects/6f519786afa249f4a345a443be12748e/users/82f78ec256ce4425b36a250149bfcdf8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.563784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f519786afa249f4a345a443be12748e, user_id=82f78ec256ce4425b36a250149bfcdf8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:10.568331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:10.584313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:10.604828 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2468/4937] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:10 2022] PUT /identity/v3/projects/6f519786afa249f4a345a443be12748e/users/82f78ec256ce4425b36a250149bfcdf8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:10.618934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89927014-f2d0-4c84-b65d-95b42e634c9e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.619622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89927014-f2d0-4c84-b65d-95b42e634c9e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.620076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89927014-f2d0-4c84-b65d-95b42e634c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.681430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-89927014-f2d0-4c84-b65d-95b42e634c9e None None] MFA Rules not processed for user `82f78ec256ce4425b36a250149bfcdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:10.682747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-89927014-f2d0-4c84-b65d-95b42e634c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:10.794229 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2470/4938] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:10.799806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.913764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4f83e-6b79-4fb0-a1b7-5bec0dc8101b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:10.920462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:10.921231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:10.921531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:10.921869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:10.923173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:10.926840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:10.976415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2469/4939] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:10 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:10.980889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89927014-f2d0-4c84-b65d-95b42e634c9e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:10.996740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89927014-f2d0-4c84-b65d-95b42e634c9e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:11.105976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:11.106633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.106992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.108101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.108101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:11.112457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:11.167642 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2471/4940] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:10 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:11.172336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:11.188138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac29c8-134e-41de-ab72-c7b3b5005d90 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:11.202960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:11.203628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.203932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.204201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.204882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:11.207044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:11.223335 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2470/4941] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:11.228801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:11.243009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee496c22-545e-46fe-97c0-9336626cefb7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:11.250599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:11.251423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.251920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.252376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] PATH_INFO: `/v3/projects/332cb6ed786341949801b77de5c338ab/users/ea16fbb006304ff598c9cb851bb6a232/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.274424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332cb6ed786341949801b77de5c338ab, user_id=ea16fbb006304ff598c9cb851bb6a232, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:11.277707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:11.297825 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2472/4942] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:11 2022] PUT /identity/v3/projects/332cb6ed786341949801b77de5c338ab/users/ea16fbb006304ff598c9cb851bb6a232/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:11.302343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:11.321784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a7f1c-73ed-4ffe-ab54-e47fdb602110 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:11.417672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be066c28-7377-4506-86d7-5d9607af89b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:11.418399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be066c28-7377-4506-86d7-5d9607af89b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.418909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be066c28-7377-4506-86d7-5d9607af89b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.419848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be066c28-7377-4506-86d7-5d9607af89b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.420117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be066c28-7377-4506-86d7-5d9607af89b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:11.425013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be066c28-7377-4506-86d7-5d9607af89b6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:11.439420 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2471/4943] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:11.443491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:11.460645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1c85e-8094-424d-969f-5874eca73b40 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:11.472599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:11.477239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.477691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.478134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] PATH_INFO: `/v3/projects/332cb6ed786341949801b77de5c338ab/users/ea16fbb006304ff598c9cb851bb6a232/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.482472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332cb6ed786341949801b77de5c338ab, user_id=ea16fbb006304ff598c9cb851bb6a232, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:11.486117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:11.508239 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2473/4944] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:11 2022] PUT /identity/v3/projects/332cb6ed786341949801b77de5c338ab/users/ea16fbb006304ff598c9cb851bb6a232/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:11.518854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d0d8e-e2fb-4a0e-9cd9-764f883adef8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.519227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d0d8e-e2fb-4a0e-9cd9-764f883adef8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.519505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d0d8e-e2fb-4a0e-9cd9-764f883adef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.572868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-677d0d8e-e2fb-4a0e-9cd9-764f883adef8 None None] MFA Rules not processed for user `ea16fbb006304ff598c9cb851bb6a232`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:11.574175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-677d0d8e-e2fb-4a0e-9cd9-764f883adef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:11.751419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:11.771778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:11.781238 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2472/4945] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:11.798726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa07a5-2aed-4a3b-beb1-8ad4763aa328 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:11.805421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:11.806203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.806511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.806756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.807512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:11.808743 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:11.808743 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:11.808743 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:11.809816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:11.830878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:11.843594 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2474/4946] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:11 2022] POST /identity/v3/services => generated 287 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:11.847447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d0d8e-e2fb-4a0e-9cd9-764f883adef8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:11.861201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d0d8e-e2fb-4a0e-9cd9-764f883adef8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:11.892435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:11.893895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:11.894404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:11.895595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:11.896024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:11.898467 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:11.898467 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:11.898467 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:11.900275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:11.975002 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1932022256. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:11.975002 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:11.975002 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:12.055306 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2473/4947] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:11 2022] POST /identity/v3/endpoints => generated 361 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:12.058194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.088099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f9596-e486-4e61-8ea7-a4db39f66e2e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.128146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.130248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.130657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.131021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-1932022256` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.131748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1932022256)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.133026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.135812 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2475/4948] 10.222.0.26 () {58 vars in 1206 bytes} [Sat Sep 17 21:11:12 2022] GET /identity/v3/regions/tempest-region-1932022256 => generated 180 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:12.142233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.156549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23b973-2d42-40e1-88c9-d52aca9e53d9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.162101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.164895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.165168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.165438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.168645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.169934 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:12.169934 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:12.169934 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:12.171002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.230656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2474/4949] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:12 2022] POST /identity/v3/services => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:12.236671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.271065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a53877-6b67-4111-a9e9-5cb7414ad107 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.349262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.350301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.350771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.351237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.355392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.362004 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:12.362004 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:12.362004 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:12.363628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.438998 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-2005273156. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:12.438998 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:12.438998 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:12.536885 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2476/4950] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:12 2022] POST /identity/v3/endpoints => generated 363 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:12.541532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.555782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c47469-d839-4438-8fbc-350f823b8af2 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.579497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.581003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.581588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.582069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-2005273156` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.583068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-2005273156)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.585079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.598021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.613674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2475/4951] 10.222.0.26 () {58 vars in 1206 bytes} [Sat Sep 17 21:11:12 2022] GET /identity/v3/regions/tempest-region-2005273156 => generated 180 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:12.647755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cd913-7acf-4587-982c-d7cd6b4d52d1 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.657862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.659896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.660621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.661611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.662928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.670549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.692933 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1908077222. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:12.692933 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:12.692933 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:12.764622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.765621 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2477/4952] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:12 2022] POST /identity/v3/endpoints => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:12.778545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cecb1-12b7-4b47-90ad-c8b888947f5d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.812699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.820917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.821217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.821560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-1908077222` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.822234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1908077222)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.823438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.832653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.833911 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2476/4953] 10.222.0.26 () {58 vars in 1206 bytes} [Sat Sep 17 21:11:12 2022] GET /identity/v3/regions/tempest-region-1908077222 => generated 180 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:12.862628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfa9f4-231f-4ccb-b885-a1ab709bb888 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.878172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.882139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.882498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.883353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.884145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.887387 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:12.887387 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:12.887387 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:12.889374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.924544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2478/4954] 10.222.0.26 () {58 vars in 1134 bytes} [Sat Sep 17 21:11:12 2022] GET /identity/v3/endpoints => generated 4399 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:12.926236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.941597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5166167-bae4-440f-b267-a0c09b54a559 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:12.949576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:12.951370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:12.951899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:12.952423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints/205ccbd3be654d19a3703ae3fa248fa9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:12.977223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=205ccbd3be654d19a3703ae3fa248fa9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:12.978606 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:12.978606 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:12.978606 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:12.980137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:12.988923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:12.999382 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2477/4955] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:12 2022] GET /identity/v3/endpoints/205ccbd3be654d19a3703ae3fa248fa9 => generated 361 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:13.005501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3a598-ee76-4dbb-a744-6b6c801c2ae0 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.015624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.016544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.017331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.017886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints/205ccbd3be654d19a3703ae3fa248fa9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.019966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=205ccbd3be654d19a3703ae3fa248fa9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.021943 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:13.021943 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:13.021943 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:13.023469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.061315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '205ccbd3be654d19a3703ae3fa248fa9', 'request_id': 'req-aa341971-474e-4567-a0ac-77d65195943c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:13.097875 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2479/4956] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:11:12 2022] DELETE /identity/v3/endpoints/205ccbd3be654d19a3703ae3fa248fa9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:13.104286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.117101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38620b-b37b-4b62-9c37-5b5ffd0943b8 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.142544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.143313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.143951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.144358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.145006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.146277 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:13.146277 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:13.146277 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:13.148891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.166858 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2478/4957] 10.222.0.26 () {58 vars in 1134 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/endpoints => generated 4051 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:13.168507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.183033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341971-474e-4567-a0ac-77d65195943c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.193209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.193886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.194221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.194669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints/205ccbd3be654d19a3703ae3fa248fa9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.202967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Could not find endpoint: 205ccbd3be654d19a3703ae3fa248fa9.: keystone.exception.EndpointNotFound: Could not find endpoint: 205ccbd3be654d19a3703ae3fa248fa9. Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._iter().one() Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._only_one_row( Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:13.238129 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 205ccbd3be654d19a3703ae3fa248fa9. Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:13.239914 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:11:13.245110 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 205ccbd3be654d19a3703ae3fa248fa9. Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:13.246637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2480/4958] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:11:13 2022] DELETE /identity/v3/endpoints/205ccbd3be654d19a3703ae3fa248fa9 => generated 114 bytes in 72 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:11:13.249658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.265107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5656c2-b93e-46dd-8186-4c1fa7feb607 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.270839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.272308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.272981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.273622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-1908077222` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.275788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1908077222)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.277675 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:13.277675 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:13.277675 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:13.281390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.305096 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1908077222. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:13.305096 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:13.305096 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:13.315046 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2479/4959] 10.222.0.26 () {58 vars in 1209 bytes} [Sat Sep 17 21:11:13 2022] DELETE /identity/v3/regions/tempest-region-1908077222 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:13.323402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.335824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763ca69-142a-4365-a324-a92cca9d6919 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.357513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.358157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.358426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.359323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.360283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.363140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.377637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2481/4960] 10.222.0.26 () {58 vars in 1134 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/endpoints => generated 4051 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:13.383550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.400957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b217f7d-84d0-42ee-ab1c-cbe1e2f3f33d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.410537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.411219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.411505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.411780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.412608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Adding query filter params (service_id=73dc9ee10345418781efca956f6b8f28) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:13.413054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.415389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.425266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2480/4961] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/endpoints?service_id=73dc9ee10345418781efca956f6b8f28 => generated 503 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:13.429232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.461858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28530a-2aac-407f-87e9-96be2e259ed9 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.469414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.470097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.470425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.470705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.471408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Adding query filter params (service_id=849e4b379d374f85a2520c889d57ad8d) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:13.471793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.476342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.495774 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2482/4962] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/endpoints?service_id=849e4b379d374f85a2520c889d57ad8d => generated 505 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:13.500378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.516985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a855130-401b-44dc-b8aa-d375e65cfb60 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.529214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d32c42-e10f-46a3-9ddb-92a4db886c05 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.529952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d32c42-e10f-46a3-9ddb-92a4db886c05 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.530449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d32c42-e10f-46a3-9ddb-92a4db886c05 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.530926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d32c42-e10f-46a3-9ddb-92a4db886c05 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.532213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d32c42-e10f-46a3-9ddb-92a4db886c05 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:13.532721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d32c42-e10f-46a3-9ddb-92a4db886c05 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.536491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d32c42-e10f-46a3-9ddb-92a4db886c05 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.559758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.560662 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2481/4963] 10.222.0.26 () {58 vars in 1167 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/endpoints?interface=public => generated 3406 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:13.568616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.569548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.570382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.573045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133480c-bd2e-46f1-8f3d-2c1d3ef87e77 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.587527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.589933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.590941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.591521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.592824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:13.593669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.596204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.606177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:13.608734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:13.613805 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2483/4964] 10.222.0.26 () {58 vars in 1171 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/endpoints?interface=internal => generated 480 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:13.619753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.645522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b1e0-a1a0-4248-842e-8d580fc74912 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.664349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.668566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.669188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.669188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.669925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.673677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.724486 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2484/4965] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:13 2022] POST /identity/v3/services => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:13.727236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.764479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:13.776456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bcd48-4622-4ea0-a47c-b6669084a77e tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.785867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.786506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.786862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.787588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.788081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.790184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.795160 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2482/4966] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:11:13 2022] POST /identity/v3/auth/tokens => generated 3960 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:13.808430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.809434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:13.824016 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-355544128. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:13.824016 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:13.824016 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:13.842986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:13.870090 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2485/4967] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:13 2022] POST /identity/v3/endpoints => generated 359 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:13.874523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.909692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755eaa-ff9a-4bc6-8ca4-3edc06898e0a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.922970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.925954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.926323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.928622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-355544128` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:13.931421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-355544128)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:13.936338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:13.951756 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2486/4968] 10.222.0.26 () {58 vars in 1203 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/regions/tempest-region-355544128 => generated 178 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:13.956962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:13.985413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea960e-9f71-450c-a694-c9c0d2f74de5 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.989711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb549e8-8301-4b6e-8301-3525aae4f773 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:13.994507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:13.995393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:13.995948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:13.996304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints/8dead1184fb4424ab6d096ab64a55caf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.003966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=8dead1184fb4424ab6d096ab64a55caf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.006298 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:14.006298 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:14.006298 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:14.008196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.017910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.018829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.019447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.019928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.030724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.032430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.036971 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1842583125. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:14.036971 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:14.036971 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:14.057706 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2483/4969] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:11:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:11:14.124855 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2487/4970] 10.222.0.26 () {60 vars in 1256 bytes} [Sat Sep 17 21:11:13 2022] PATCH /identity/v3/endpoints/8dead1184fb4424ab6d096ab64a55caf => generated 363 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:14.126295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.147058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cf434-284b-4532-9b84-e87042306ac9 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.175978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.176986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.177497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.178039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-1842583125` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.178782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1842583125)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.180169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.183213 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2484/4971] 10.222.0.26 () {58 vars in 1206 bytes} [Sat Sep 17 21:11:14 2022] GET /identity/v3/regions/tempest-region-1842583125 => generated 180 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:14.192393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.208215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d3e01-5450-4603-85b0-c318a2e2233d tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.216546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.217547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.217986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.218420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints/8dead1184fb4424ab6d096ab64a55caf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.232612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=8dead1184fb4424ab6d096ab64a55caf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.245019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.265280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '8dead1184fb4424ab6d096ab64a55caf', 'request_id': 'req-74af6cba-c2e2-4ac1-a438-67982bab4fe7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:14.292728 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2488/4972] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:11:14 2022] DELETE /identity/v3/endpoints/8dead1184fb4424ab6d096ab64a55caf => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:14.294075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.308937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c4e40-53de-42a0-a2d1-cfc278cb8352 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.335797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.336352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.336992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.337558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-1842583125` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.338410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1842583125)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.340953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.382705 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1842583125. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:14.382705 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:14.382705 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:14.417072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.417866 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2485/4973] 10.222.0.26 () {58 vars in 1209 bytes} [Sat Sep 17 21:11:14 2022] DELETE /identity/v3/regions/tempest-region-1842583125 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:14.437187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af6cba-c2e2-4ac1-a438-67982bab4fe7 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.473164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.474329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.474855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.475399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-355544128` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.476325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-355544128)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.480270 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:14.480270 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:14.480270 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:14.481756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.535947 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2489/4974] 10.222.0.26 () {58 vars in 1206 bytes} [Sat Sep 17 21:11:14 2022] DELETE /identity/v3/regions/tempest-region-355544128 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:14.541511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.564911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf9c47-36cf-45d5-a9ce-bb93b4d7bc96 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.589151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.590271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.590859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.591265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/services/65f03ea2d6de491cba304bc15b764705` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.602016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_service(service_id=65f03ea2d6de491cba304bc15b764705)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.606012 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:14.606012 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:14.606012 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:14.608315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.677106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2486/4975] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:14 2022] DELETE /identity/v3/services/65f03ea2d6de491cba304bc15b764705 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:14.690844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.714343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa53d01-b806-48ad-933d-135daeb6ca21 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.763414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.764432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.764432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.764632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints/93f8ee99387247eda41c1933de7b41e2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.772410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=93f8ee99387247eda41c1933de7b41e2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.774085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.793386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '93f8ee99387247eda41c1933de7b41e2', 'request_id': 'req-629197a1-f87e-4887-8a38-62d35cb02b99'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:14.811400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2490/4976] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:11:14 2022] DELETE /identity/v3/endpoints/93f8ee99387247eda41c1933de7b41e2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:14.815392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.832354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38769bcd-4349-4734-b5e2-261a3f237b7c tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.855787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.856693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.856983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.857681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-2005273156` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.858403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-2005273156)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.860481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:14.878276 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-2005273156. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:14.878276 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:14.878276 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:14.894610 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2487/4977] 10.222.0.26 () {58 vars in 1209 bytes} [Sat Sep 17 21:11:14 2022] DELETE /identity/v3/regions/tempest-region-2005273156 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:14.894792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:14.908968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629197a1-f87e-4887-8a38-62d35cb02b99 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:14.939380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:14.940486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:14.943253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:14.943566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/services/849e4b379d374f85a2520c889d57ad8d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:14.944193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_service(service_id=849e4b379d374f85a2520c889d57ad8d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:14.945218 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:14.945218 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:14.945218 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:14.950706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:15.002464 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2491/4978] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:14 2022] DELETE /identity/v3/services/849e4b379d374f85a2520c889d57ad8d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:15.008461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:15.026338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d33191-85e0-44aa-b7a9-5607ce7570de tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:15.050662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:15.051547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:15.051950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:15.052291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/endpoints/e1eb5839e780415e8b4d6367b3c282b9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:15.058220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=e1eb5839e780415e8b4d6367b3c282b9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:15.059581 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:15.059581 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:15.059581 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:15.060686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:15.062556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:15.073970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'e1eb5839e780415e8b4d6367b3c282b9', 'request_id': 'req-9b288a80-da9e-41b1-90e0-8ea262fded3a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:15.095503 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2488/4979] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:11:15 2022] DELETE /identity/v3/endpoints/e1eb5839e780415e8b4d6367b3c282b9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:15.096670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:15.117960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4b1f1-78d3-4fc4-aa34-49382a8d640f tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:15.147276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:15.149104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:15.149716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:15.150142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/regions/tempest-region-1932022256` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:15.151047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1932022256)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:15.153574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:15.179480 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1932022256. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:15.179480 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:15.179480 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:15.182810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:15.190831 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2492/4980] 10.222.0.26 () {58 vars in 1209 bytes} [Sat Sep 17 21:11:15 2022] DELETE /identity/v3/regions/tempest-region-1932022256 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:15.195589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:15.221678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288a80-da9e-41b1-90e0-8ea262fded3a tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:15.244384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14137a-ec0f-4473-8c3c-c61ed965ef74 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea16fbb006304ff598c9cb851bb6a232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '332cb6ed786341949801b77de5c338ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:15.245671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14137a-ec0f-4473-8c3c-c61ed965ef74 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:15.245998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14137a-ec0f-4473-8c3c-c61ed965ef74 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:15.246489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14137a-ec0f-4473-8c3c-c61ed965ef74 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] PATH_INFO: `/v3/services/73dc9ee10345418781efca956f6b8f28` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:15.247391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14137a-ec0f-4473-8c3c-c61ed965ef74 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorizing `identity:delete_service(service_id=73dc9ee10345418781efca956f6b8f28)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:15.249827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14137a-ec0f-4473-8c3c-c61ed965ef74 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:15.290307 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2489/4981] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:15 2022] DELETE /identity/v3/services/73dc9ee10345418781efca956f6b8f28 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:15.294509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:15.447785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b157b-1714-4be0-b768-32f783d52597 tempest-EndPointsTestJSON-2008026614 tempest-EndPointsTestJSON-2008026614-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:15.466494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:15.467502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:15.467868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:15.468230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] PATH_INFO: `/v3/users/82f78ec256ce4425b36a250149bfcdf8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:15.469664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f78ec256ce4425b36a250149bfcdf8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:15.472287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:15.537389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f78ec256ce4425b36a250149bfcdf8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:15.538430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f78ec256ce4425b36a250149bfcdf8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:15.550328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f78ec256ce4425b36a250149bfcdf8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:15.551931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f78ec256ce4425b36a250149bfcdf8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:15.581936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f78ec256ce4425b36a250149bfcdf8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:15.605231 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2493/4982] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:15 2022] DELETE /identity/v3/users/82f78ec256ce4425b36a250149bfcdf8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:15.610434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:15.610892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:15.611374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:15.640197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:15.641918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:15.765290 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2490/4983] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:15.778481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:15.780312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:15.935824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac6147-82de-4d90-bba5-94389ca7a787 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:15.951296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:15.952149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:15.956323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:15.956809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:15.998920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:16.007982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:16.010503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:16.035848 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2494/4984] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:16.552154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:16.554502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:16.579310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabccd7-0d8e-48bd-9d40-cbb01704199a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:16.586421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:16.587889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:16.589168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:16.589657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] PATH_INFO: `/v3/projects/6f519786afa249f4a345a443be12748e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:16.591735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f519786afa249f4a345a443be12748e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:16.594795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:16.704039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f519786afa249f4a345a443be12748e', 'request_id': 'req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.716859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f519786afa249f4a345a443be12748e', 'request_id': 'req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.726164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f519786afa249f4a345a443be12748e', 'request_id': 'req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.740639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] The token cache is being invalidate because project 6f519786afa249f4a345a443be12748e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:16.740934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.741709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.743587 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2491/4985] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:16 2022] DELETE /identity/v3/projects/6f519786afa249f4a345a443be12748e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:16.747921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:16.761744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbc210-dd26-4ff7-a15b-e9bbbba33fb1 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:16.873451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:16.874322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:16.874602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:16.875204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] PATH_INFO: `/v3/users/ea16fbb006304ff598c9cb851bb6a232` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:16.877426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea16fbb006304ff598c9cb851bb6a232)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:16.879352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:16.970555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea16fbb006304ff598c9cb851bb6a232'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.971351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea16fbb006304ff598c9cb851bb6a232'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.996661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea16fbb006304ff598c9cb851bb6a232'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:16.997854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea16fbb006304ff598c9cb851bb6a232'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:17.029232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea16fbb006304ff598c9cb851bb6a232'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:17.045837 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2495/4986] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:16 2022] DELETE /identity/v3/users/ea16fbb006304ff598c9cb851bb6a232 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:17.966881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:17.974589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:18.009442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f86f1-a0fb-4ec5-93fd-17e21b21bc9b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:18.101613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:18.145631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:18.146454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:18.146875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:18.147365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] PATH_INFO: `/v3/projects/332cb6ed786341949801b77de5c338ab` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:18.156513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=332cb6ed786341949801b77de5c338ab)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:18.158978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:18.275121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:18.277551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:18.312916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332cb6ed786341949801b77de5c338ab', 'request_id': 'req-4803d081-5868-43f6-a7f5-3a2d925d5fe0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:18.348826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332cb6ed786341949801b77de5c338ab', 'request_id': 'req-4803d081-5868-43f6-a7f5-3a2d925d5fe0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:18.359470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332cb6ed786341949801b77de5c338ab', 'request_id': 'req-4803d081-5868-43f6-a7f5-3a2d925d5fe0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:18.398760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] The token cache is being invalidate because project 332cb6ed786341949801b77de5c338ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:18.400538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:18.404315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4803d081-5868-43f6-a7f5-3a2d925d5fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:18.410426 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2492/4987] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:17 2022] DELETE /identity/v3/projects/332cb6ed786341949801b77de5c338ab => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:18.415599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab98b-af21-4503-8a75-25ab5408a217 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:18.416156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab98b-af21-4503-8a75-25ab5408a217 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:18.416451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab98b-af21-4503-8a75-25ab5408a217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:18.465919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-754ab98b-af21-4503-8a75-25ab5408a217 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:18.469766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-754ab98b-af21-4503-8a75-25ab5408a217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:18.594112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:18.603663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93baaf-f0b4-4f81-94b2-9296d9bde0d7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:18.647129 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2493/4988] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:18.659181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0821f-c64e-4d36-b033-1ef8425772d1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:18.659181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0821f-c64e-4d36-b033-1ef8425772d1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:18.659181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0821f-c64e-4d36-b033-1ef8425772d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:18.729669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f5e0821f-c64e-4d36-b033-1ef8425772d1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:18.731566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f5e0821f-c64e-4d36-b033-1ef8425772d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:18.747517 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2494/4989] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:18.751029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0821f-c64e-4d36-b033-1ef8425772d1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:18.782861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0821f-c64e-4d36-b033-1ef8425772d1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:18.805584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:18.810399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:18.810886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:18.811865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:18.814236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:18.814799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:18.818820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:18.819466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:18.819608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:18.819731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:18.820060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:18.831426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:18.855990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2495/4990] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:18.860661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:18.888795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:18.892618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:18.897952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f401fab-f863-4f0c-a169-784e5979d293 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:18.922899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:18.923641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:18.923936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:18.924231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:18.928335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:18.930437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:18.968515 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2496/4991] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:19.029344 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2496/4992] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:18 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:19.035075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:19.054915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869d316-1cc3-4058-b8e8-8a4490868091 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:19.291141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:19.292655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:19.293118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:19.293694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:19.294670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:19.296909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:19.348539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2497/4993] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:19 2022] POST /identity/v3/users => generated 294 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:19.353736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:19.365322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554b5bb-6a25-4c13-8973-491cf648b18c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:19.388167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:19.388843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:19.391626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:19.391976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:19.392637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:19.398378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:19.420483 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2497/4994] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:19.425909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:19.461639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de975d40-1288-4b4c-af34-caf8081ec1bd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:19.474183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83ad41-677c-4fbd-94cf-3859e249a37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:19.476190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83ad41-677c-4fbd-94cf-3859e249a37f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:19.476515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83ad41-677c-4fbd-94cf-3859e249a37f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:19.478099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83ad41-677c-4fbd-94cf-3859e249a37f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:19.478862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83ad41-677c-4fbd-94cf-3859e249a37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:19.517641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83ad41-677c-4fbd-94cf-3859e249a37f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:19.547791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2498/4995] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:19.558004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:19.576665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af4bcf-55d1-49c8-8cde-7f147d38b664 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:19.590247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:19.591026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:19.591310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:19.591604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] PATH_INFO: `/v3/projects/7f1dc1c379bd4cf19c10dc3fe7feed8a/users/1dd1355261ef40b28e465d1193c82a78/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:19.630941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:19.638469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1dc1c379bd4cf19c10dc3fe7feed8a, user_id=1dd1355261ef40b28e465d1193c82a78, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:19.641407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:19.667420 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2498/4996] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:19 2022] PUT /identity/v3/projects/7f1dc1c379bd4cf19c10dc3fe7feed8a/users/1dd1355261ef40b28e465d1193c82a78/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:19.766240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7984a-1e95-4693-90c3-22121c9936e3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:19.766664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7984a-1e95-4693-90c3-22121c9936e3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:19.766664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7984a-1e95-4693-90c3-22121c9936e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:19.840303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dff7984a-1e95-4693-90c3-22121c9936e3 None None] MFA Rules not processed for user `1dd1355261ef40b28e465d1193c82a78`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:19.842538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dff7984a-1e95-4693-90c3-22121c9936e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:19.984672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:19.997225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:20.008719 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2499/4997] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:20.147074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad20fd-d3ee-4cad-9f92-ec1aa25daff8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:20.168113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:20.168813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:20.169094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:20.174763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:20.175507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:20.177499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:20.235292 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2499/4998] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:19 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:20.239802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7984a-1e95-4693-90c3-22121c9936e3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:20.285802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7984a-1e95-4693-90c3-22121c9936e3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:20.513088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:20.513911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:20.514336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:20.515642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:20.516525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:20.520365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:20.584548 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2500/4999] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:20 2022] POST /identity/v3/users => generated 299 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:20.588577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:20.617903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40693d8-9467-4c25-9150-5944a5d4481e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:20.625920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:20.627327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:20.627893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:20.628777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:20.629672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:20.632558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:20.654914 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2500/5000] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:20.659127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:20.674728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b11542-6e64-4675-87ce-01570907c64b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:20.694357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:20.696882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:20.697515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:20.697946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] PATH_INFO: `/v3/projects/5b7d12bfa3dd4d1f83faf64ec55ade65/users/17e5391e8cf04e38b917d33171f19a72/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:20.730601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7d12bfa3dd4d1f83faf64ec55ade65, user_id=17e5391e8cf04e38b917d33171f19a72, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:20.757840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:20.829974 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2501/5001] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:20 2022] PUT /identity/v3/projects/5b7d12bfa3dd4d1f83faf64ec55ade65/users/17e5391e8cf04e38b917d33171f19a72/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:20.834200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:20.851433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be080e-951e-461d-85ac-a434219898d5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:21.025946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8770961-a20c-4a05-934f-e5c463e247a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:21.027301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8770961-a20c-4a05-934f-e5c463e247a1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:21.028141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8770961-a20c-4a05-934f-e5c463e247a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:21.028826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8770961-a20c-4a05-934f-e5c463e247a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:21.030077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8770961-a20c-4a05-934f-e5c463e247a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:21.032150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8770961-a20c-4a05-934f-e5c463e247a1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:21.070591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2501/5002] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:21.071904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:21.092213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e98117-a8d2-410e-bb15-80b9479974a3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:21.101617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:21.102703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:21.103372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:21.104346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] PATH_INFO: `/v3/projects/5b7d12bfa3dd4d1f83faf64ec55ade65/users/17e5391e8cf04e38b917d33171f19a72/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:21.106888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7d12bfa3dd4d1f83faf64ec55ade65, user_id=17e5391e8cf04e38b917d33171f19a72, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:21.109869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:21.135881 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2502/5003] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:21 2022] PUT /identity/v3/projects/5b7d12bfa3dd4d1f83faf64ec55ade65/users/17e5391e8cf04e38b917d33171f19a72/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:21.149141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f50187-3254-4f3a-81b9-5c6017e90270 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:21.149929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f50187-3254-4f3a-81b9-5c6017e90270 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:21.150156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f50187-3254-4f3a-81b9-5c6017e90270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:21.223256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-15f50187-3254-4f3a-81b9-5c6017e90270 None None] MFA Rules not processed for user `17e5391e8cf04e38b917d33171f19a72`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:21.224477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-15f50187-3254-4f3a-81b9-5c6017e90270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:21.380807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:21.467897 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2502/5004] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:21.472477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:21.509594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746a1d-18ed-4169-af07-f20406d93bee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:21.563572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:21.568827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:21.571117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:21.571117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:21.571571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:21.572950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:21.639036 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2503/5005] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:21 2022] POST /identity/v3/services => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:21.645514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f50187-3254-4f3a-81b9-5c6017e90270 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:21.676224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f50187-3254-4f3a-81b9-5c6017e90270 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:21.709979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:21.710677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:21.711218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:21.711498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:21.712180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:21.714392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: On instance['enabled']: Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: On instance['enabled']: Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: 'False' Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise error Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application 'False' Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:21.724752 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:11:21.726451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application 'False' Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:21.727628 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2503/5006] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:21 2022] POST /identity/v3/endpoints => generated 283 bytes in 82 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 17 21:11:21.731676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:21.745926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cdc6c-c743-4059-b619-587fb3a4446e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:21.773921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:21.774885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:21.775766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:21.776223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:21.779782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:21.781970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: On instance['enabled']: Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: On instance['enabled']: Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: 'True' Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise error Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application 'True' Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:21.808452 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:11:21.810442 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application 'True' Sep 17 21:11:21.811703 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:21.821769 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2504/5007] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:21 2022] POST /identity/v3/endpoints => generated 281 bytes in 90 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 17 21:11:21.822696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:21.851299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562c1b-dc43-4524-a940-0e53ddcf4ff7 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:21.866517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:21.868090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:21.868485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:21.868593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:21.872570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:21.875694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:21.952350 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1212988772. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:21.952350 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:21.952350 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:22.029323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2504/5008] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:21 2022] POST /identity/v3/endpoints => generated 361 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:22.031591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.068570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfb822-818d-4d82-8802-3667eeb72fda tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.139591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.140738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.142510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.143134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/regions/tempest-region-1212988772` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.147675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1212988772)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.149027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.158901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.159874 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2505/5009] 10.222.0.26 () {58 vars in 1206 bytes} [Sat Sep 17 21:11:22 2022] GET /identity/v3/regions/tempest-region-1212988772 => generated 180 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:22.182363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2b9d4-083b-4a51-95b5-47b698c2bb91 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.192653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.193326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.193906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.194257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints/7fc69ef7c7054c8bb9c9c28b5dae2f5e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.218912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=7fc69ef7c7054c8bb9c9c28b5dae2f5e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.219983 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:22.219983 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:22.219983 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:22.227368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: On instance['enabled']: Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: On instance['enabled']: Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: 'False' Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise error Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application 'False' Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:22.244847 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:11:22.246617 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application 'False' Sep 17 21:11:22.247941 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:11:22.251807 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2505/5010] 10.222.0.26 () {60 vars in 1255 bytes} [Sat Sep 17 21:11:22 2022] PATCH /identity/v3/endpoints/7fc69ef7c7054c8bb9c9c28b5dae2f5e => generated 283 bytes in 94 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 17 21:11:22.256611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.279217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fbaac-df9e-42fa-9c40-6f021908d0b9 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.292963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.294535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.295009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.295517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints/7fc69ef7c7054c8bb9c9c28b5dae2f5e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.297802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=7fc69ef7c7054c8bb9c9c28b5dae2f5e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.302790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.340462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '7fc69ef7c7054c8bb9c9c28b5dae2f5e', 'request_id': 'req-7a173b83-d874-4f02-9d5c-001d6d638cf0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:22.377521 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2506/5011] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:11:22 2022] DELETE /identity/v3/endpoints/7fc69ef7c7054c8bb9c9c28b5dae2f5e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:22.380424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.400258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b8b22-ad5e-42c6-8912-978e24b9f415 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.442264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.442981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.443214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.443486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/regions/tempest-region-1212988772` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.444082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1212988772)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.451179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.514972 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2506/5012] 10.222.0.26 () {58 vars in 1209 bytes} [Sat Sep 17 21:11:22 2022] DELETE /identity/v3/regions/tempest-region-1212988772 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:22.520889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.563921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a173b83-d874-4f02-9d5c-001d6d638cf0 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.644359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.647922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.652081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.653224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.656235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.670755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.728978 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-617208256. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:11:22.728978 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:11:22.728978 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:22.793797 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2507/5013] 10.222.0.26 () {60 vars in 1156 bytes} [Sat Sep 17 21:11:22 2022] POST /identity/v3/endpoints => generated 358 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:22.796223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.823021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a763cb5-a1fa-41dd-a6ce-c26874df3b55 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.881843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.884834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.886561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.887077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/regions/tempest-region-617208256` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.887997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-617208256)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.889516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.892340 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2507/5014] 10.222.0.26 () {58 vars in 1203 bytes} [Sat Sep 17 21:11:22 2022] GET /identity/v3/regions/tempest-region-617208256 => generated 178 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:22.896581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.922351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5889de43-10c0-4432-beda-7baa902ba72e tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.942112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.946182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.946182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.946182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints/b59e30e0defc4ffe81cc859905a3b0d9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.963126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=b59e30e0defc4ffe81cc859905a3b0d9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.964778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: On instance['enabled']: Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: On instance['enabled']: Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: 'True' Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise error Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application 'True' Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:22.993185 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:11:22.995694 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['enabled']: Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application 'True' Sep 17 21:11:22.999214 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:11:23.000889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.001057 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2508/5015] 10.222.0.26 () {60 vars in 1255 bytes} [Sat Sep 17 21:11:22 2022] PATCH /identity/v3/endpoints/b59e30e0defc4ffe81cc859905a3b0d9 => generated 281 bytes in 102 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 17 21:11:23.014279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d172dc4-a024-4fbe-8049-9092eb09d557 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:23.023270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:23.024215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.024497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.025310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/endpoints/b59e30e0defc4ffe81cc859905a3b0d9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.028896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=b59e30e0defc4ffe81cc859905a3b0d9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:23.030851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:23.051896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'b59e30e0defc4ffe81cc859905a3b0d9', 'request_id': 'req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.086852 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2508/5016] 10.222.0.26 () {58 vars in 1236 bytes} [Sat Sep 17 21:11:22 2022] DELETE /identity/v3/endpoints/b59e30e0defc4ffe81cc859905a3b0d9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:23.099040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.122013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95401a-2079-44ba-9ee2-c570d8ec467d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:23.132654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=8,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:23.160381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:23.161415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.161978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.163013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/regions/tempest-region-617208256` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.164232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-617208256)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:23.167882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:23.208242 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2509/5017] 10.222.0.26 () {58 vars in 1206 bytes} [Sat Sep 17 21:11:23 2022] DELETE /identity/v3/regions/tempest-region-617208256 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:23.213846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.227953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b2b9-36dc-4d09-9d8e-a4fa8f5a55e4 tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:23.270591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17e5391e8cf04e38b917d33171f19a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:23.272236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.272518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.272824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] PATH_INFO: `/v3/services/17ec31f6e3bc4d458335e08569bce376` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.273577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorizing `identity:delete_service(service_id=17ec31f6e3bc4d458335e08569bce376)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:23.275624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:23.347589 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2509/5018] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:23 2022] DELETE /identity/v3/services/17ec31f6e3bc4d458335e08569bce376 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:23.352876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.377689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.379829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:23.508216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:23.537417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21a9f0-a13f-46d1-a1b3-1b36017a1ddb tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:23.574241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:23.575047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.575377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.575711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] PATH_INFO: `/v3/users/1dd1355261ef40b28e465d1193c82a78` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.578059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd1355261ef40b28e465d1193c82a78)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:23.581717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06929fcd-f866-4f9f-bf4b-371ba5f1798d tempest-EndpointsNegativeTestJSON-838615759 tempest-EndpointsNegativeTestJSON-838615759-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:23.583258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:23.595017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:23.599085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.599867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.600326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.604109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:23.706990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd1355261ef40b28e465d1193c82a78'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.708200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd1355261ef40b28e465d1193c82a78'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.728839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd1355261ef40b28e465d1193c82a78'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.730034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd1355261ef40b28e465d1193c82a78'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.770366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e599e9af-7b9c-4163-9a52-f46de5ecafb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd1355261ef40b28e465d1193c82a78'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.820788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:23.826023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:23.827511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2510/5019] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:23 2022] DELETE /identity/v3/users/1dd1355261ef40b28e465d1193c82a78 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:23.835375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.835738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.836017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.898727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:23.900446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:23.906631 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2510/5020] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:11:23.909876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.913704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:23.962797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:23.967736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13c908-ea73-4e2e-a53e-f0c13492be9d service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.001884 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2511/5021] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:24.014998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.018305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:24.088726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.089695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.089839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.090010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] PATH_INFO: `/v3/users/e8b19d9480444a44a48ac3c5efe7fe17` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.091234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b19d9480444a44a48ac3c5efe7fe17)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.093237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.130752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afd1a7-d63b-4a82-9b8e-6d9580de279e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.137689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f305b-8927-4c78-a05c-1c170449b5d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.138664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f305b-8927-4c78-a05c-1c170449b5d3 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.138936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f305b-8927-4c78-a05c-1c170449b5d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.139209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f305b-8927-4c78-a05c-1c170449b5d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.161437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f305b-8927-4c78-a05c-1c170449b5d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.163430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f305b-8927-4c78-a05c-1c170449b5d3 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.178638 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2512/5022] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:24.207009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b19d9480444a44a48ac3c5efe7fe17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.237435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b19d9480444a44a48ac3c5efe7fe17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.247519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:24.261186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b19d9480444a44a48ac3c5efe7fe17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.262504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b19d9480444a44a48ac3c5efe7fe17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.264737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b19d9480444a44a48ac3c5efe7fe17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.300168 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2511/5023] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:23 2022] DELETE /identity/v3/users/e8b19d9480444a44a48ac3c5efe7fe17 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:24.304947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.305846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.306369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.346219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:24.349973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:24.468507 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2513/5024] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:24.511754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.514012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:24.664046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a2148-69ac-4cc0-9f2f-053a2ac374a9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.675361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.676536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.676744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.677040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.699663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.704037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.733347 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2512/5025] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:24.892981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.895100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:24.914176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a37edf-865e-474e-8ff0-0f8373aae31e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.930815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.932366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.932740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.933111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] PATH_INFO: `/v3/projects/7f1dc1c379bd4cf19c10dc3fe7feed8a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.936349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1dc1c379bd4cf19c10dc3fe7feed8a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.940109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.073587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1dc1c379bd4cf19c10dc3fe7feed8a', 'request_id': 'req-a44f539d-c08a-4b10-b2d9-611f78a48b67'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.083029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1dc1c379bd4cf19c10dc3fe7feed8a', 'request_id': 'req-a44f539d-c08a-4b10-b2d9-611f78a48b67'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.123619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1dc1c379bd4cf19c10dc3fe7feed8a', 'request_id': 'req-a44f539d-c08a-4b10-b2d9-611f78a48b67'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.139726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:25.160435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] The token cache is being invalidate because project 7f1dc1c379bd4cf19c10dc3fe7feed8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:25.161377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.161760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.166297 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2514/5026] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:24 2022] DELETE /identity/v3/projects/7f1dc1c379bd4cf19c10dc3fe7feed8a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:25.170812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.187229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d27bb5-cbc1-4346-8c08-4f6ba3256da4 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.349304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.352748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.352992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.353277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] PATH_INFO: `/v3/users/17e5391e8cf04e38b917d33171f19a72` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.354808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e5391e8cf04e38b917d33171f19a72)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.356563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.477512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e5391e8cf04e38b917d33171f19a72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.502437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e5391e8cf04e38b917d33171f19a72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.527142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e5391e8cf04e38b917d33171f19a72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.529344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e5391e8cf04e38b917d33171f19a72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.530935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5badc485-8e5f-4549-a85c-aae0d39d4051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e5391e8cf04e38b917d33171f19a72'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.560123 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2513/5027] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:25 2022] DELETE /identity/v3/users/17e5391e8cf04e38b917d33171f19a72 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:25.914386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.915918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:25.942955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f539d-c08a-4b10-b2d9-611f78a48b67 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:26.091350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:26.093268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:26.093268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:26.093268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] PATH_INFO: `/v3/projects/f7a3c50d58d347d0a4467cd4e61b06e0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:26.094883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a3c50d58d347d0a4467cd4e61b06e0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:26.095901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:26.222437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a3c50d58d347d0a4467cd4e61b06e0', 'request_id': 'req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:26.238489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a3c50d58d347d0a4467cd4e61b06e0', 'request_id': 'req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:26.261360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a3c50d58d347d0a4467cd4e61b06e0', 'request_id': 'req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:26.306134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] The token cache is being invalidate because project f7a3c50d58d347d0a4467cd4e61b06e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:26.306709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:26.321189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7712e45b-f8df-4a96-b2e8-0012b2f7cc3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:26.340657 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2515/5028] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:25 2022] DELETE /identity/v3/projects/f7a3c50d58d347d0a4467cd4e61b06e0 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:26.345369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a90d3-cd9f-430a-957e-9a5969b39d7c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:26.345684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a90d3-cd9f-430a-957e-9a5969b39d7c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:26.345961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a90d3-cd9f-430a-957e-9a5969b39d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:26.397228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4b9a90d3-cd9f-430a-957e-9a5969b39d7c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:26.406218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4b9a90d3-cd9f-430a-957e-9a5969b39d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:26.438570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:26.593082 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2514/5029] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:26.598315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507077af-5477-4f3a-9f41-987087df4766 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:26.598315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507077af-5477-4f3a-9f41-987087df4766 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:26.598315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507077af-5477-4f3a-9f41-987087df4766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:26.658590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-507077af-5477-4f3a-9f41-987087df4766 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:26.661293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-507077af-5477-4f3a-9f41-987087df4766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:26.705582 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2516/5030] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:26.709636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a90d3-cd9f-430a-957e-9a5969b39d7c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:26.743785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a90d3-cd9f-430a-957e-9a5969b39d7c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:26.758659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:26.760963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:26.763112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:26.763434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:26.766550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:26.767115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:26.785673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:26.820807 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2515/5031] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:26.825135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507077af-5477-4f3a-9f41-987087df4766 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:26.838603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507077af-5477-4f3a-9f41-987087df4766 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:26.850000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:26.850644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:26.850924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:26.851196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:26.851926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:26.854552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:26.941995 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2517/5032] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:26 2022] POST /identity/v3/projects => generated 405 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:26.947012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:26.964045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c02f-13ac-4f33-aa93-8406c56a602f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:27.128534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:27.129758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.130276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.130767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.131657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:27.136249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:27.238159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:27.258527 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2516/5033] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:26 2022] POST /identity/v3/users => generated 303 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:27.319391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbe1e8-f7d1-42ba-a5d4-2598c6e80265 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:27.339708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:27.342167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:27.347221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:27.352034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.352385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.352657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.353347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:27.362786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:27.382405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db78f-710b-481c-a36c-804b8fb3b083 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:27.404445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:27.407443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.407958 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2518/5034] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:27.409541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.411323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:27.411703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] PATH_INFO: `/v3/projects/5b7d12bfa3dd4d1f83faf64ec55ade65` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.418668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7d12bfa3dd4d1f83faf64ec55ade65)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:27.429694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:27.459052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da54e55-0b62-4c3c-848c-3bf4ced7d02e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:27.483194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:27.503080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.504012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.504012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.510001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:27.517429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:27.546580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'request_id': 'req-c5469be2-731d-4768-80d9-15931d8b2137'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.557792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'request_id': 'req-c5469be2-731d-4768-80d9-15931d8b2137'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.566691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7d12bfa3dd4d1f83faf64ec55ade65', 'request_id': 'req-c5469be2-731d-4768-80d9-15931d8b2137'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.572125 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2519/5035] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:27.587672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] The token cache is being invalidate because project 5b7d12bfa3dd4d1f83faf64ec55ade65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:27.587672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.587672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c5469be2-731d-4768-80d9-15931d8b2137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.590959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2517/5036] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:27 2022] DELETE /identity/v3/projects/5b7d12bfa3dd4d1f83faf64ec55ade65 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:27.593167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:27.595577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506fb4-26d8-4660-b6c0-63eb8e3c7649 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.596043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506fb4-26d8-4660-b6c0-63eb8e3c7649 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.597156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506fb4-26d8-4660-b6c0-63eb8e3c7649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.604432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:27.635182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7b882-0d6f-4804-adc4-c332e3123236 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:27.641273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-14506fb4-26d8-4660-b6c0-63eb8e3c7649 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:27.643547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-14506fb4-26d8-4660-b6c0-63eb8e3c7649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:27.658792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:27.747948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:27.806637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacb9c0-702d-4b0a-bb58-275c71a1d6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:27.807305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacb9c0-702d-4b0a-bb58-275c71a1d6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.807586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacb9c0-702d-4b0a-bb58-275c71a1d6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.807912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacb9c0-702d-4b0a-bb58-275c71a1d6b8 admin admin] PATH_INFO: `/v3/projects/bbb68270a6474c41b16e24cae657407c/users/adfe77cd9de149508efbea44af748876/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.831515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacb9c0-702d-4b0a-bb58-275c71a1d6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb68270a6474c41b16e24cae657407c, user_id=adfe77cd9de149508efbea44af748876, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:27.834908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacb9c0-702d-4b0a-bb58-275c71a1d6b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:27.884527 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2518/5037] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:27.890921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e9458-606a-4419-ab54-31db938ea133 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.891443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e9458-606a-4419-ab54-31db938ea133 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.891866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e9458-606a-4419-ab54-31db938ea133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.892392 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2520/5038] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:27 2022] PUT /identity/v3/projects/bbb68270a6474c41b16e24cae657407c/users/adfe77cd9de149508efbea44af748876/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:27.898680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637175f-3cd4-4db8-afc9-605e6f880316 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.898969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637175f-3cd4-4db8-afc9-605e6f880316 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.902208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637175f-3cd4-4db8-afc9-605e6f880316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.942320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-894e9458-606a-4419-ab54-31db938ea133 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:27.943996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-894e9458-606a-4419-ab54-31db938ea133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:28.035492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9637175f-3cd4-4db8-afc9-605e6f880316 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:28.037985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9637175f-3cd4-4db8-afc9-605e6f880316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:28.081741 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2521/5039] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:28.094132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637175f-3cd4-4db8-afc9-605e6f880316 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.099220 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2519/5040] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:28.100799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e9458-606a-4419-ab54-31db938ea133 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.102421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-894e9458-606a-4419-ab54-31db938ea133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:28.108550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637175f-3cd4-4db8-afc9-605e6f880316 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.123745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.124593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.125228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.125634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.127684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:28.128060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.131123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.160176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2522/5041] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:28.164752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.183499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e67df1-228d-489f-960b-996dfb539571 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.197822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.199529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.202168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.202750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.203579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.208813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.216941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e9458-606a-4419-ab54-31db938ea133 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.241314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.241984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.244781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.245077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.277986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.282481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.310514 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2523/5042] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:28.315969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.326101 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2520/5043] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:28.531176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8a4aa-2105-4107-a075-2e32b490c482 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.546137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.549265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.550712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.551702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.552942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.557641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.619374 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2524/5044] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/users => generated 284 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:28.621092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.650445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967715f-c02d-4bcc-9a9e-581d12cb6a2d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.663632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.666330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.667584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.668107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.674067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.676456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.701095 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2521/5045] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:28.707787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.735755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2b705-cc28-4c19-bc77-24efdf03ddf4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.749391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22a5ac-3972-4d30-aeac-75561d87c8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.751460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a5ac-3972-4d30-aeac-75561d87c8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.751752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a5ac-3972-4d30-aeac-75561d87c8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.752704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22a5ac-3972-4d30-aeac-75561d87c8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.754727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22a5ac-3972-4d30-aeac-75561d87c8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.758800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22a5ac-3972-4d30-aeac-75561d87c8e8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.808432 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2525/5046] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:28.812441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.825371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3937c-f45b-4ae9-b553-372614cb0225 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.849320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.850128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.851938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.854279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] PATH_INFO: `/v3/projects/6c59f4f071e946d6a0eecb3f65a7de0d/users/0caa2bbd31554504943ca0c771e9e1f8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.919649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c59f4f071e946d6a0eecb3f65a7de0d, user_id=0caa2bbd31554504943ca0c771e9e1f8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.924338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.976110 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2522/5047] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:28 2022] PUT /identity/v3/projects/6c59f4f071e946d6a0eecb3f65a7de0d/users/0caa2bbd31554504943ca0c771e9e1f8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:28.986439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bd307-ee50-4ed3-ba84-92598f2f5d66 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.987132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bd307-ee50-4ed3-ba84-92598f2f5d66 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.987681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bd307-ee50-4ed3-ba84-92598f2f5d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.060218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b75bd307-ee50-4ed3-ba84-92598f2f5d66 None None] MFA Rules not processed for user `0caa2bbd31554504943ca0c771e9e1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:29.062589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b75bd307-ee50-4ed3-ba84-92598f2f5d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:29.201438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:29.277730 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2526/5048] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:29.278456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.396562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:29.502765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080edf02-c014-441a-b034-f18f9c04d020 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.516432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.517266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.517766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.518218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.519057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.525118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.574107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2523/5049] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:29 2022] POST /identity/v3/projects => generated 369 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:29.577366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bd307-ee50-4ed3-ba84-92598f2f5d66 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.592933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bd307-ee50-4ed3-ba84-92598f2f5d66 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.750328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.751485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.751767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.752039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.752701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.754727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.829209 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2527/5050] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:29 2022] POST /identity/v3/users => generated 291 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:29.830861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.847722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9fd15-5450-42d5-a759-1c04f4352032 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.858125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.858989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.859305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.859648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.860384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.863374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.892209 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2524/5051] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:29.893874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.908300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5504e-ea42-4382-a31c-5cc3689ae0c9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.926619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.927774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.928303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.929125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] PATH_INFO: `/v3/projects/e08a29b247f5439486596d3d96581443/users/f684e4d8386a418a9f1e03adc72d9d80/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:30.001193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08a29b247f5439486596d3d96581443, user_id=f684e4d8386a418a9f1e03adc72d9d80, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:30.004442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:30.045670 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2528/5052] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:29 2022] PUT /identity/v3/projects/e08a29b247f5439486596d3d96581443/users/f684e4d8386a418a9f1e03adc72d9d80/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:30.049671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:30.068779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dcef-d37e-4e9a-8dab-56854be21d6d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:30.255560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24aacf2-7cea-40cf-ac39-69e6f06d8486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:30.256721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24aacf2-7cea-40cf-ac39-69e6f06d8486 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:30.257384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24aacf2-7cea-40cf-ac39-69e6f06d8486 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:30.257865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24aacf2-7cea-40cf-ac39-69e6f06d8486 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:30.258688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24aacf2-7cea-40cf-ac39-69e6f06d8486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:30.260676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24aacf2-7cea-40cf-ac39-69e6f06d8486 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:30.288864 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2525/5053] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:30.290569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:30.316032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2257-1278-4743-bc7c-faf18ee8773a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:30.321347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:30.322374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:30.323025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:30.323704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] PATH_INFO: `/v3/projects/e08a29b247f5439486596d3d96581443/users/f684e4d8386a418a9f1e03adc72d9d80/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:30.326254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08a29b247f5439486596d3d96581443, user_id=f684e4d8386a418a9f1e03adc72d9d80, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:30.329384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:30.364391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2529/5054] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:30 2022] PUT /identity/v3/projects/e08a29b247f5439486596d3d96581443/users/f684e4d8386a418a9f1e03adc72d9d80/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:30.366728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cfd6-edfa-452e-8ba2-e644a2d3ae8d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:30.367020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cfd6-edfa-452e-8ba2-e644a2d3ae8d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:30.367295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cfd6-edfa-452e-8ba2-e644a2d3ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:30.435890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c739cfd6-edfa-452e-8ba2-e644a2d3ae8d None None] MFA Rules not processed for user `f684e4d8386a418a9f1e03adc72d9d80`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:30.437632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c739cfd6-edfa-452e-8ba2-e644a2d3ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:30.618552 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2526/5055] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:30.619818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:30.636548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea6071-f471-48f0-b387-e1090a5bec51 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:30.658040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:30.658719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:30.659001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:30.659317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:30.660038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:30.667477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:30.714212 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2530/5056] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:11:30 2022] POST /identity/v3/domains => generated 274 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:30.716248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739cfd6-edfa-452e-8ba2-e644a2d3ae8d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:30.727040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739cfd6-edfa-452e-8ba2-e644a2d3ae8d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:30.739953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:30.878930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:30.880782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:30.881068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:30.881343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:30.885004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:30.887163 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:30.887163 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:30.887163 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:30.891214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:30.983463 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2527/5057] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:30 2022] POST /identity/v3/groups => generated 281 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:30.983812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:30.997893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21676f8a-7f90-43fb-acf4-dc9f52da749f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.019650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:31.020181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.020489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.020768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/a09d6ad84d154ced83da7e5a0e032f52` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:31.046033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:update_group(group_id=a09d6ad84d154ced83da7e5a0e032f52)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:31.049949 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:31.049949 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:31.049949 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:31.051260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:31.125475 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2531/5058] 10.222.0.26 () {60 vars in 1247 bytes} [Sat Sep 17 21:11:30 2022] PATCH /identity/v3/groups/a09d6ad84d154ced83da7e5a0e032f52 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:31.131213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:31.146606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c1f3-a582-4131-87ca-0ceacc491d04 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.158929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdb93d-114e-4449-a69f-e387ea1814cb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:31.159597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdb93d-114e-4449-a69f-e387ea1814cb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.160022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdb93d-114e-4449-a69f-e387ea1814cb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.160300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdb93d-114e-4449-a69f-e387ea1814cb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/a09d6ad84d154ced83da7e5a0e032f52` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:31.174287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdb93d-114e-4449-a69f-e387ea1814cb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:get_group(group_id=a09d6ad84d154ced83da7e5a0e032f52)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:31.175344 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-dbbdb93d-114e-4449-a69f-e387ea1814cb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:31.175344 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:31.175344 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:31.176560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdb93d-114e-4449-a69f-e387ea1814cb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:31.185033 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2528/5059] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:31 2022] GET /identity/v3/groups/a09d6ad84d154ced83da7e5a0e032f52 => generated 293 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:31.186262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:31.201613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0e323-35ba-4e36-9410-a1500fe7e072 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.236895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:31.237626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.238079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.238357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/a09d6ad84d154ced83da7e5a0e032f52` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:31.240014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:update_group(group_id=a09d6ad84d154ced83da7e5a0e032f52)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:31.242718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.243791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.244466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:31.253878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:31.295675 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2532/5060] 10.222.0.26 () {60 vars in 1246 bytes} [Sat Sep 17 21:11:31 2022] PATCH /identity/v3/groups/a09d6ad84d154ced83da7e5a0e032f52 => generated 309 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:31.296629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:31.320269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:31.324460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:31.326057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40440718-a67d-49dd-a5e8-e36c59fb643f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.333796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:31.334767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.335202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.335645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/a09d6ad84d154ced83da7e5a0e032f52` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:31.358683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_group(group_id=a09d6ad84d154ced83da7e5a0e032f52)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:31.365018 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:31.365018 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:31.365018 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:31.366744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:31.505178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2529/5061] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:31.513806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:31.516506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:31.519675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:UPDATE=2,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:31.546937 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2533/5062] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:31 2022] DELETE /identity/v3/groups/a09d6ad84d154ced83da7e5a0e032f52 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:31.567329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:31.733899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17d487-bc01-429b-97e8-58560139614f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.739508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:31.740507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.740942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.741390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:31.742227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:31.743765 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:31.743765 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:31.743765 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:31.745400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:31.769308 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2534/5063] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:31 2022] POST /identity/v3/groups => generated 325 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:31.774247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:31.787108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce28d-58b3-41a3-bf3c-cec1edda5f70 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.792854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:31.794012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.794486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.795556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:31.796800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:31.798958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:31.834197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c90eaf-25bb-4065-bb7a-48d79f71dd73 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.843500 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2535/5064] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:31 2022] POST /identity/v3/users => generated 321 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:31.850438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:31.853988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:31.925379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1795b-5254-4acd-aa4d-b3133d78c115 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:31.956236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:31.958970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:31.960113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:31.966718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/336e60bbd37544fbae654a2bb595a1ae` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:32.024077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:32.027119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:32.027293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=336e60bbd37544fbae654a2bb595a1ae)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:32.027417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:32.032121 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:32.032121 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:32.032121 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:32.033923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:32.034983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:32.165639 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2536/5065] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:11:31 2022] PUT /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/336e60bbd37544fbae654a2bb595a1ae => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:32.174407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:32.259548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:32.262828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:32.289452 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2530/5066] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:11:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 776 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:11:32.636651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d04e8-dce5-45c8-9c33-7d2684e4c3a0 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:32.653540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:32.654949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:32.655408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:32.655678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:32.656333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:32.680658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:32.745057 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2537/5067] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:32 2022] POST /identity/v3/users => generated 323 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:32.750012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:32.803831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be744cf4-ffb8-4c42-ad2a-33e53451c859 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:32.859212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:32.861437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:32.919765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:32.920757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/6fa00e2668474c42b7ffd512980bb343` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.001707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=6fa00e2668474c42b7ffd512980bb343)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:33.002784 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:33.002784 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:33.002784 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:33.004848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:33.128408 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2531/5068] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:11:32 2022] PUT /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/6fa00e2668474c42b7ffd512980bb343 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:33.130642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:33.152034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6934b-4b7e-4d7c-9f70-03c1c71d23b1 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:33.276338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:33.436977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:33.437661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:33.437943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:33.438221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.438872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:33.459230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:33.562928 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2538/5069] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:33 2022] POST /identity/v3/users => generated 323 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:33.567422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:33.613806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc9c12-13b2-41ed-818c-556965e4f511 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:33.622839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:33.624084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:33.624574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:33.625413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/ef9259491492441791865ac8218cbce0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.670756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=ef9259491492441791865ac8218cbce0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:33.675991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:33.822598 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2532/5070] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:11:33 2022] PUT /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/ef9259491492441791865ac8218cbce0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:33.826867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:33.862631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff9ff5-a2c6-4475-9d2e-a870f04e5826 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:33.979586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ae25e-8e15-42c6-8097-8a768330af1b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:33.980668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae25e-8e15-42c6-8097-8a768330af1b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:33.981300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae25e-8e15-42c6-8097-8a768330af1b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:33.981872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae25e-8e15-42c6-8097-8a768330af1b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.986399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ae25e-8e15-42c6-8097-8a768330af1b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=6838973335344cf99a67f083a1d79a13)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:33.987933 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-637ae25e-8e15-42c6-8097-8a768330af1b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:33.987933 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:33.987933 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:33.998142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ae25e-8e15-42c6-8097-8a768330af1b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.047511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.054053 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2539/5071] 10.222.0.26 () {58 vars in 1242 bytes} [Sat Sep 17 21:11:33 2022] GET /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users => generated 1083 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:34.064447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff6f4-f89e-4be4-b6a0-9aa7cd8bbf44 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.106456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.140643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.141595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.141595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/336e60bbd37544fbae654a2bb595a1ae` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.149251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=336e60bbd37544fbae654a2bb595a1ae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.159707 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:34.159707 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:34.159707 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:34.169598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.190935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea615-4ead-4ad0-9e09-05ec3690c377 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.192367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea615-4ead-4ad0-9e09-05ec3690c377 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.192874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea615-4ead-4ad0-9e09-05ec3690c377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.228889 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2533/5072] 10.222.0.26 () {58 vars in 1342 bytes} [Sat Sep 17 21:11:34 2022] HEAD /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/336e60bbd37544fbae654a2bb595a1ae => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:34.241256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.250297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-493ea615-4ead-4ad0-9e09-05ec3690c377 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:34.254176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-493ea615-4ead-4ad0-9e09-05ec3690c377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:34.277302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e7afc-ac4f-4d89-9847-81b9316b42bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.288441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.289719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.291303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.291908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/336e60bbd37544fbae654a2bb595a1ae` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.295839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=336e60bbd37544fbae654a2bb595a1ae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.298314 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:34.298314 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:34.298314 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:34.301713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.330716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-94f5e7e3-b6ff-43d9-97c0-78136c23c5cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '336e60bbd37544fbae654a2bb595a1ae'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:34.369842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2534/5073] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:11:34 2022] DELETE /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/336e60bbd37544fbae654a2bb595a1ae => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:34.373181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812092-7802-495c-b97e-e6c738d2124b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.374474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812092-7802-495c-b97e-e6c738d2124b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.375139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812092-7802-495c-b97e-e6c738d2124b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.410012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:34.419898 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2540/5074] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:34.420967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea615-4ead-4ad0-9e09-05ec3690c377 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.435044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-65812092-7802-495c-b97e-e6c738d2124b None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:34.437034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-65812092-7802-495c-b97e-e6c738d2124b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:34.468458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea615-4ead-4ad0-9e09-05ec3690c377 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.576979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:34.600122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.600917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.601305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.601722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/6fa00e2668474c42b7ffd512980bb343` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.605791 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2535/5075] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:34.607308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65812092-7802-495c-b97e-e6c738d2124b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.610391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=6fa00e2668474c42b7ffd512980bb343)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.611553 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:34.611553 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:34.611553 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:34.613044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.650308 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2541/5076] 10.222.0.26 () {58 vars in 1342 bytes} [Sat Sep 17 21:11:34 2022] HEAD /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/6fa00e2668474c42b7ffd512980bb343 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:34.654964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.675971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5538c-7e81-4b83-85fe-05004c0fb8d6 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.688237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.689033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.689409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.690962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/6fa00e2668474c42b7ffd512980bb343` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.698921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=6fa00e2668474c42b7ffd512980bb343)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.699450 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:34.699450 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:34.699450 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:34.701118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.718366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '6fa00e2668474c42b7ffd512980bb343'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:34.751779 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2542/5077] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:11:34 2022] DELETE /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/6fa00e2668474c42b7ffd512980bb343 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:34.753409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.768636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65812092-7802-495c-b97e-e6c738d2124b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.897943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27ff67-3d4e-4bfb-90e1-12bf21570f91 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.917014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.918168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.918725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.919219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/ef9259491492441791865ac8218cbce0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.921733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=ef9259491492441791865ac8218cbce0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.924040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.927688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84067277-ab02-404b-bef2-770642b45beb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.928288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067277-ab02-404b-bef2-770642b45beb service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.928575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067277-ab02-404b-bef2-770642b45beb service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.928851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067277-ab02-404b-bef2-770642b45beb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.966599 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2543/5078] 10.222.0.26 () {58 vars in 1342 bytes} [Sat Sep 17 21:11:34 2022] HEAD /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/ef9259491492441791865ac8218cbce0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:34.971382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.997731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640dbd22-5558-4b57-90e4-1da806514701 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.031626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.034785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.035428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.036289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users/ef9259491492441791865ac8218cbce0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.041431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:35.042242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=6838973335344cf99a67f083a1d79a13, user_id=ef9259491492441791865ac8218cbce0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.049234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.064494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ef9259491492441791865ac8218cbce0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:35.073079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84067277-ab02-404b-bef2-770642b45beb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.076700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84067277-ab02-404b-bef2-770642b45beb service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.092112 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2544/5079] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:11:34 2022] DELETE /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users/ef9259491492441791865ac8218cbce0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:35.096250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.101376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2536/5080] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:11:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:11:35.281538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e056f27-83fe-498b-b2a2-072e7efa0064 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.296181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.297382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.297681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.298914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.303127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=6838973335344cf99a67f083a1d79a13)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.307376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.346684 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2545/5081] 10.222.0.26 () {58 vars in 1242 bytes} [Sat Sep 17 21:11:35 2022] GET /identity/v3/groups/6838973335344cf99a67f083a1d79a13/users => generated 145 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:35.348419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84067277-ab02-404b-bef2-770642b45beb service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.365890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84067277-ab02-404b-bef2-770642b45beb service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.392340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.396082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.396566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.397017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users/ef9259491492441791865ac8218cbce0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.401297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ef9259491492441791865ac8218cbce0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.405892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.561208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9259491492441791865ac8218cbce0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:35.592858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9259491492441791865ac8218cbce0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:35.604319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9259491492441791865ac8218cbce0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:35.605184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9259491492441791865ac8218cbce0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:35.606203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-438de24d-0b9b-42d8-8bfb-1746022ae4b9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9259491492441791865ac8218cbce0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:35.649866 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2537/5082] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:35 2022] DELETE /identity/v3/users/ef9259491492441791865ac8218cbce0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:35.655703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.659218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:35.691775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799b3b-4873-4964-9702-a82a6ed5ad3a tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.755237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:35.796483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.797178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.797702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.844795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.846600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.846911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.850533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users/6fa00e2668474c42b7ffd512980bb343` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.857680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6fa00e2668474c42b7ffd512980bb343)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.861969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.864023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:35.865787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:36.074915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa00e2668474c42b7ffd512980bb343'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.080538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa00e2668474c42b7ffd512980bb343'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.108845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa00e2668474c42b7ffd512980bb343'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.110846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa00e2668474c42b7ffd512980bb343'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.121116 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2538/5083] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:36.138647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa00e2668474c42b7ffd512980bb343'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.139116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:36.144750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:36.176570 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2546/5084] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:35 2022] DELETE /identity/v3/users/6fa00e2668474c42b7ffd512980bb343 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:36.181282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:36.182302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:36.349343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484e66e-f6eb-42d2-9517-de28faaddacc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:36.355931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:36.377961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c906a58-30e0-4836-99e6-1d5e4a06d080 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:36.386415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:36.390730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:36.391262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:36.391687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users/336e60bbd37544fbae654a2bb595a1ae` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:36.394368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_user(user_id=336e60bbd37544fbae654a2bb595a1ae)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:36.403061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:36.535639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336e60bbd37544fbae654a2bb595a1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.537855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336e60bbd37544fbae654a2bb595a1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.545536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:36.551013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:36.551868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:36.552436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:36.552984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:36.560119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336e60bbd37544fbae654a2bb595a1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.561491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336e60bbd37544fbae654a2bb595a1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.591930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336e60bbd37544fbae654a2bb595a1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:36.631569 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2547/5085] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:36 2022] DELETE /identity/v3/users/336e60bbd37544fbae654a2bb595a1ae => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:36.633173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:36.635536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:36.744721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:36.747137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:36.761719 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2539/5086] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:11:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:11:36.864781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0df39c-8bb6-4b99-a15e-ce7132597306 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:36.876946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:36.877978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:36.878516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:36.878793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6838973335344cf99a67f083a1d79a13` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:36.882444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_group(group_id=6838973335344cf99a67f083a1d79a13)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:36.884629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:36.993722 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2548/5087] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:36 2022] DELETE /identity/v3/groups/6838973335344cf99a67f083a1d79a13 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:37.000134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:37.033558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725657ca-8253-4082-8f54-f3982a637d96 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:37.378673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:37.383103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:37.384671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:37.385055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:37.386905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:37.395041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:37.452004 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2540/5088] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:36 2022] POST /identity/v3/groups => generated 326 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:37.453455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:37.470979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3e72e-825a-48fb-8448-372c4f7c2159 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:37.490358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:37.493784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:37.494062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:37.494339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:37.494999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:37.509328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:37.658587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2549/5089] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:37 2022] POST /identity/v3/groups => generated 326 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:37.682846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:37.755712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33520b5-09a7-4da5-ad20-a9ebaefc304b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:37.815395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:37.816038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:37.816326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:37.816604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:37.822838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:37.834021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:37.896274 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2541/5090] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:37 2022] POST /identity/v3/groups => generated 327 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:37.897197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:37.915962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092f48-f5da-4e07-8aa4-1da9de775c2b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:37.944066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57baa2-a157-4e6b-a6d7-dc272f66b29b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:37.946635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57baa2-a157-4e6b-a6d7-dc272f66b29b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:37.946918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57baa2-a157-4e6b-a6d7-dc272f66b29b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:37.949702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57baa2-a157-4e6b-a6d7-dc272f66b29b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:37.952224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57baa2-a157-4e6b-a6d7-dc272f66b29b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:37.953323 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-ce57baa2-a157-4e6b-a6d7-dc272f66b29b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:37.953323 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:37.953323 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:37.961380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57baa2-a157-4e6b-a6d7-dc272f66b29b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:38.001943 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2550/5091] 10.222.0.26 () {58 vars in 1125 bytes} [Sat Sep 17 21:11:37 2022] GET /identity/v3/groups => generated 1489 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:38.002612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:38.027026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24d421-1339-49e0-bf6a-de5759f4ffe9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:38.040632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:38.041975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.042476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.042904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/6187ae9813134799856158ea7a9cfa12` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.050763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_group(group_id=6187ae9813134799856158ea7a9cfa12)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:38.052412 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:38.052412 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:38.052412 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:38.054649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.055038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.055344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:38.056315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.107644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:38.110376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:38.131613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:38.208412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2542/5092] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:37 2022] DELETE /identity/v3/groups/6187ae9813134799856158ea7a9cfa12 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:38.214197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:38.268144 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2551/5093] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:38.298915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:38.300829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:38.319114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:38.414744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ddfd-472d-4953-a076-e9f52f37fbcb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:38.430416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:38.431386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.433794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.434478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/68bd67ff89804d3389381202d9d25988` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.453026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_group(group_id=68bd67ff89804d3389381202d9d25988)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:38.455881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:38.461233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784de88-f180-4195-942d-d9e02b15e957 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:38.569008 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2543/5094] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:38 2022] DELETE /identity/v3/groups/68bd67ff89804d3389381202d9d25988 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:38.573171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:38.617833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:38.618636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.619018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.619394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.763090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bced54-4d9f-40b3-8091-936909ddd5bb tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:38.775288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:38.777403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.779258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.780319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/5bd2721e203748f98936790bdae41c8b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.796623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_group(group_id=5bd2721e203748f98936790bdae41c8b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:38.799055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:38.861955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:38.864671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:38.898077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=10,keystone:SELECT=90 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:38.900308 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2552/5095] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:11:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:11:38.954943 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2544/5096] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:38 2022] DELETE /identity/v3/groups/5bd2721e203748f98936790bdae41c8b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:38.957086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:38.987759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f825d3-7e66-4aa8-a1fa-1b3447f6df6d service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:39.102871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:39.158970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:39.159881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.161959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.162541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.163913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:39.170830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:39.253234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:39.256751 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2553/5097] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:38 2022] POST /identity/v3/users => generated 323 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:39.268395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84a6a-55c5-4427-a447-eba5eb12abd2 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:39.286467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:39.289202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.291067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.291503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.292494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:39.295735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:39.340368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:39.351453 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2545/5098] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:39 2022] POST /identity/v3/groups => generated 325 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:39.407879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b77c6-10b0-4678-9cc6-86e5f31a6b22 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:39.429703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:39.433113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.433113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.433799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/1269262318a24cdd9a26815079841274/users/469e683f24ad41cd8526f11b0a57d818` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.528653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=1269262318a24cdd9a26815079841274, user_id=469e683f24ad41cd8526f11b0a57d818)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:39.543861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:39.659810 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2554/5099] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:11:39 2022] PUT /identity/v3/groups/1269262318a24cdd9a26815079841274/users/469e683f24ad41cd8526f11b0a57d818 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:39.666336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:39.692639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2434aa-7721-489e-a2a3-80ba229ef560 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:39.939264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:39.939898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.940175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.940444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.941757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:39.943813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.046571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2546/5100] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:39 2022] POST /identity/v3/groups => generated 325 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:40.048202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.086964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0b5a-d684-48e7-9068-08600846413f tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.107173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.108186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.109683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.112945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/29cda46937e843db9c2e3a45da88d706/users/469e683f24ad41cd8526f11b0a57d818` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.139768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=29cda46937e843db9c2e3a45da88d706, user_id=469e683f24ad41cd8526f11b0a57d818)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.151471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.216239 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2555/5101] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:11:40 2022] PUT /identity/v3/groups/29cda46937e843db9c2e3a45da88d706/users/469e683f24ad41cd8526f11b0a57d818 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:40.219171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.239094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad3831-7527-4214-a83c-1630cdfb13cc tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.334974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.335916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.336267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.336720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users/469e683f24ad41cd8526f11b0a57d818/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.338468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=469e683f24ad41cd8526f11b0a57d818)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.341806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.379144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.388043 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2547/5102] 10.222.0.26 () {58 vars in 1242 bytes} [Sat Sep 17 21:11:40 2022] GET /identity/v3/users/469e683f24ad41cd8526f11b0a57d818/groups => generated 836 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:40.413343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a0483-9ac2-412b-88a1-2910b84047b3 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.459838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.460861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.461393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.462007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/29cda46937e843db9c2e3a45da88d706` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.463669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_group(group_id=29cda46937e843db9c2e3a45da88d706)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.466371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.584318 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2556/5103] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:40 2022] DELETE /identity/v3/groups/29cda46937e843db9c2e3a45da88d706 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:40.589714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.620773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa6993-633f-4dc0-996c-efdb80d1d32b tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.780583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:40.792598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.794573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.795042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.795464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/groups/1269262318a24cdd9a26815079841274` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.797393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_group(group_id=1269262318a24cdd9a26815079841274)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.800461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.946330 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2548/5104] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:40 2022] DELETE /identity/v3/groups/1269262318a24cdd9a26815079841274 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:40.951885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.968949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad5f41-ef40-4c09-9728-4e3f64a071ae tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:41.175247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:41.177919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:41.178215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:41.181246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/users/469e683f24ad41cd8526f11b0a57d818` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:41.188078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_user(user_id=469e683f24ad41cd8526f11b0a57d818)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:41.191713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:41.270182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:41.301641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469e683f24ad41cd8526f11b0a57d818'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.302522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469e683f24ad41cd8526f11b0a57d818'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.325327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469e683f24ad41cd8526f11b0a57d818'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.326453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469e683f24ad41cd8526f11b0a57d818'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.371323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ef31cdc5-9ed7-4577-a061-ce42fc0c2454 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469e683f24ad41cd8526f11b0a57d818'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.400502 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2557/5105] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:40 2022] DELETE /identity/v3/users/469e683f24ad41cd8526f11b0a57d818 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:41.409623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:41.413812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:41.440418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12d36-9663-4b15-8c7d-aa61ee4343f9 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:41.753783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:41.755602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:41.756011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:41.756428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/domains/84dd13da037a4c4c9763f24031417cd0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:41.762258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=84dd13da037a4c4c9763f24031417cd0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:41.764193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:41.816990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63618-3670-4615-bf10-11653b86809f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:41.817365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63618-3670-4615-bf10-11653b86809f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:41.817529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63618-3670-4615-bf10-11653b86809f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:41.834822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '84dd13da037a4c4c9763f24031417cd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.835958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '84dd13da037a4c4c9763f24031417cd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.914725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '84dd13da037a4c4c9763f24031417cd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.915677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '84dd13da037a4c4c9763f24031417cd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:41.921856 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2549/5106] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:11:41 2022] PATCH /identity/v3/domains/84dd13da037a4c4c9763f24031417cd0 => generated 275 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:41.926568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:41.928592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:41.984832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8ee63618-3670-4615-bf10-11653b86809f None None] MFA Rules not processed for user `adfe77cd9de149508efbea44af748876`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:41.992004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8ee63618-3670-4615-bf10-11653b86809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:42.086205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089353c-8d68-4616-a907-86a4dbce2ef7 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:42.103612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f684e4d8386a418a9f1e03adc72d9d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e08a29b247f5439486596d3d96581443', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:42.104631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:42.105082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:42.105560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] PATH_INFO: `/v3/domains/84dd13da037a4c4c9763f24031417cd0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:42.134603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=84dd13da037a4c4c9763f24031417cd0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:42.138014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:42.161734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:42.179295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '84dd13da037a4c4c9763f24031417cd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.228927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '84dd13da037a4c4c9763f24031417cd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.235299 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2558/5107] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:42.241013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63618-3670-4615-bf10-11653b86809f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:42.243330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8ee63618-3670-4615-bf10-11653b86809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:42.354831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84dd13da037a4c4c9763f24031417cd0', 'request_id': 'req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.368893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84dd13da037a4c4c9763f24031417cd0', 'request_id': 'req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.379789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84dd13da037a4c4c9763f24031417cd0', 'request_id': 'req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.384822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63618-3670-4615-bf10-11653b86809f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:42.421218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] The token cache is being invalidate because project 84dd13da037a4c4c9763f24031417cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:42.421974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.423334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.452457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '84dd13da037a4c4c9763f24031417cd0', 'request_id': 'req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.457556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '84dd13da037a4c4c9763f24031417cd0', 'request_id': 'req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.458680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '84dd13da037a4c4c9763f24031417cd0', 'request_id': 'req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:42.467091 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2550/5108] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:11:41 2022] DELETE /identity/v3/domains/84dd13da037a4c4c9763f24031417cd0 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:42.471340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:42.474633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:42.475843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:42.510729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:42.512305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:42.512611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:42.512891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:42.515020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:42.517951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:42.589977 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2559/5109] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:42 2022] POST /identity/v3/projects => generated 405 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:42.593879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:42.598393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:42.649051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6aa2a-00ff-49f1-8b83-362ced77b7a4 tempest-GroupsV3TestJSON-1034284967 tempest-GroupsV3TestJSON-1034284967-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:42.871112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:42.873410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:42.875318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:42.875601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] PATH_INFO: `/v3/users/0caa2bbd31554504943ca0c771e9e1f8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:42.878330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0caa2bbd31554504943ca0c771e9e1f8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:42.889356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:42.892073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac429aa-3c42-46d3-a2d0-13db37d217d4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:42.904278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:42.908693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:42.912536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:42.913056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:42.913731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:42.917390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:42.924474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:42.997164 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2560/5110] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:42 2022] POST /identity/v3/users => generated 303 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:43.006596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.022931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.074376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0caa2bbd31554504943ca0c771e9e1f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.088652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49096969-8c59-4115-92c7-f1a2da8e1f82 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.111592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0caa2bbd31554504943ca0c771e9e1f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.119192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.120210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.125889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.126172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] PATH_INFO: `/v3/users/7e12ad7a93ae4a9a95b2a54c1a2b9845` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.127565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e12ad7a93ae4a9a95b2a54c1a2b9845)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.132582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0caa2bbd31554504943ca0c771e9e1f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.134154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0caa2bbd31554504943ca0c771e9e1f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.134885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0caa2bbd31554504943ca0c771e9e1f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.138016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:43.159114 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2551/5111] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:42 2022] DELETE /identity/v3/users/0caa2bbd31554504943ca0c771e9e1f8 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:43.168258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.171401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.314945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e12ad7a93ae4a9a95b2a54c1a2b9845'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.318204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e12ad7a93ae4a9a95b2a54c1a2b9845'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.336714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e12ad7a93ae4a9a95b2a54c1a2b9845'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.338574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e12ad7a93ae4a9a95b2a54c1a2b9845'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.364974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82b57277-90e0-4fb0-942d-6caaf69fb6d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e12ad7a93ae4a9a95b2a54c1a2b9845'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.386803 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2561/5112] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:42 2022] DELETE /identity/v3/users/7e12ad7a93ae4a9a95b2a54c1a2b9845 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:43.397526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2dfca-993d-4dc1-8b15-5a48c47b2fff None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.397906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2dfca-993d-4dc1-8b15-5a48c47b2fff None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.399452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2dfca-993d-4dc1-8b15-5a48c47b2fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.408015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c67df-1735-42fd-a544-6955dc91df87 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.478734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-47d2dfca-993d-4dc1-8b15-5a48c47b2fff None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:43.486196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-47d2dfca-993d-4dc1-8b15-5a48c47b2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.599572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:43.622692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8417-65c3-4eaf-88f4-a382c6862ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.623990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8417-65c3-4eaf-88f4-a382c6862ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.624233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8417-65c3-4eaf-88f4-a382c6862ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.624933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8417-65c3-4eaf-88f4-a382c6862ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.626338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb8417-65c3-4eaf-88f4-a382c6862ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.628589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb8417-65c3-4eaf-88f4-a382c6862ee7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:43.654873 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2552/5113] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:43.660517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.660788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.661207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.700173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2562/5114] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:43.705153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.705297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.707158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.716517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:43.717843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.749448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:43.751657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.765422 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2563/5115] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:43.767011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.769119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.791254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:43.793295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ef4b3-e4ac-47c5-936e-aec816bc4a0a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.804049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.805409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.805781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.806089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.807440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.810148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:43.825964 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2564/5116] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:43.833115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.835628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.923023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2553/5117] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:43.923023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.928614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.991551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74e064-d6a3-4603-90aa-b2851e568121 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.003873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.004139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.004423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.004691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.038667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.044519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.062401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcd2f6-9b43-472b-89c4-ca6e43a9420f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.067708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.070265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.070563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.070836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.073930 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2565/5118] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:44.075925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.085003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.086699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.102340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f85d8-2559-4465-9f64-5c5f3c44bf98 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.116428 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2554/5119] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:44.117414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.119921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.120851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.121292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.121810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] PATH_INFO: `/v3/projects/81efda2512d54722b767bdf07a566236/users/b2704d86338949ba9bb87bf60ee24146/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.122764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:44.172859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81efda2512d54722b767bdf07a566236, user_id=b2704d86338949ba9bb87bf60ee24146, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.179256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.222895 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2566/5120] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:44 2022] PUT /identity/v3/projects/81efda2512d54722b767bdf07a566236/users/b2704d86338949ba9bb87bf60ee24146/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:44.314402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56bf8-2d1a-4619-b283-6251ecda34a5 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.506403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.508052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.508052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.508466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.540112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:44.619959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.622190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.638687 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2555/5121] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:11:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:11:44.966132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.979646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:44.986388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.990573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:45.161427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af406c-3c35-4114-a564-170cad696a07 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:45.165197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995be31-6e61-468e-8346-0ce83f3be100 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:45.173493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:45.174579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:45.175045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:45.175502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] PATH_INFO: `/v3/projects/6c59f4f071e946d6a0eecb3f65a7de0d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:45.176825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c59f4f071e946d6a0eecb3f65a7de0d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:45.181732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:45.186030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:45.192034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:45.192034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:45.192034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] PATH_INFO: `/v3/projects/9818fed65901412b8da672532cc5bc52` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:45.195522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9818fed65901412b8da672532cc5bc52)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:45.203449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:45.295782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:45.343647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c59f4f071e946d6a0eecb3f65a7de0d', 'request_id': 'req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.355467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c59f4f071e946d6a0eecb3f65a7de0d', 'request_id': 'req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.376159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c59f4f071e946d6a0eecb3f65a7de0d', 'request_id': 'req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.399651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] The token cache is being invalidate because project 6c59f4f071e946d6a0eecb3f65a7de0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:45.400874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.401729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.407627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9818fed65901412b8da672532cc5bc52', 'request_id': 'req-02a205a5-4e65-48ba-9e3d-9cfa5900e403'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.408584 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2567/5122] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:44 2022] DELETE /identity/v3/projects/6c59f4f071e946d6a0eecb3f65a7de0d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:45.412607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:45.420487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:45.432581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9818fed65901412b8da672532cc5bc52', 'request_id': 'req-02a205a5-4e65-48ba-9e3d-9cfa5900e403'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.471976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9818fed65901412b8da672532cc5bc52', 'request_id': 'req-02a205a5-4e65-48ba-9e3d-9cfa5900e403'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.495052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] The token cache is being invalidate because project 9818fed65901412b8da672532cc5bc52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:45.519124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.520162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-02a205a5-4e65-48ba-9e3d-9cfa5900e403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:45.523761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2556/5123] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:44 2022] DELETE /identity/v3/projects/9818fed65901412b8da672532cc5bc52 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:45.529994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddee0b4-4097-47f5-b8df-c4841afbc238 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:45.530621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddee0b4-4097-47f5-b8df-c4841afbc238 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:45.531257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddee0b4-4097-47f5-b8df-c4841afbc238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:45.597609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9ddee0b4-4097-47f5-b8df-c4841afbc238 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:45.601347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9ddee0b4-4097-47f5-b8df-c4841afbc238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:45.628468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c598d29-8397-4095-ab3f-f5dbcfbf1fe4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:45.802780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:45.824572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:45.830475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:45.830760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:45.831037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] PATH_INFO: `/v3/users/f684e4d8386a418a9f1e03adc72d9d80` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:45.833862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f684e4d8386a418a9f1e03adc72d9d80)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:45.835885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:45.862483 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2557/5124] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:45.873449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd37c-1558-464d-9b76-36c6eaee6251 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:45.874004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd37c-1558-464d-9b76-36c6eaee6251 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:45.874820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bd37c-1558-464d-9b76-36c6eaee6251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:45.971865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c67bd37c-1558-464d-9b76-36c6eaee6251 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:45.974691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c67bd37c-1558-464d-9b76-36c6eaee6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:46.006591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2558/5125] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:46.009829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bd37c-1558-464d-9b76-36c6eaee6251 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:46.015429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f684e4d8386a418a9f1e03adc72d9d80'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:46.016529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f684e4d8386a418a9f1e03adc72d9d80'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:46.029330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bd37c-1558-464d-9b76-36c6eaee6251 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:46.031398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f684e4d8386a418a9f1e03adc72d9d80'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:46.032251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f684e4d8386a418a9f1e03adc72d9d80'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:46.046015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:46.046647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:46.048160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:46.048313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:46.048853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:46.049172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:46.052385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:46.056717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f684e4d8386a418a9f1e03adc72d9d80'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:46.059609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:46.083190 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2559/5126] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:46.085991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:46.089782 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2568/5127] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:45 2022] DELETE /identity/v3/users/f684e4d8386a418a9f1e03adc72d9d80 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:46.093379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:46.119506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd91b33-f8d2-4b67-9aa7-39ede82d47c5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:46.248555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:46.249425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:46.251304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:46.251746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:46.252745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:46.255034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:46.307680 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2560/5128] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:46 2022] POST /identity/v3/projects => generated 389 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:46.312280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:46.323218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c25bcc-5894-4da6-bc68-6a7399040407 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:46.510365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:46.510732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:46.511163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:46.511896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:46.512611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:46.517272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:46.565850 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2569/5129] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:46 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:46.571007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:46.588723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921fca4-32cf-4e96-915c-190de1f7bd62 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:46.599486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:46.600127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:46.600408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:46.600872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:46.601337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:46.609412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:46.631128 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2561/5130] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:46.635578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:46.655809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5959ede-df05-4cad-9abc-8e88523eee36 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:46.672200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e223b9e-bbde-4bd0-8e3d-dc43ef1eff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:46.673112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223b9e-bbde-4bd0-8e3d-dc43ef1eff14 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:46.673891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223b9e-bbde-4bd0-8e3d-dc43ef1eff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:46.674293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e223b9e-bbde-4bd0-8e3d-dc43ef1eff14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:46.675082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e223b9e-bbde-4bd0-8e3d-dc43ef1eff14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:46.680528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e223b9e-bbde-4bd0-8e3d-dc43ef1eff14 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:46.709146 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2570/5131] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:46.710574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:46.728968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fca18-f298-419e-b401-fe4aaf5aa743 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:46.743191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:46.743883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:46.744342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:46.744796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] PATH_INFO: `/v3/projects/96f88828c3634d04977ccf21eb350e88/users/058e5024f3fd4f00af564894a781cb0f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:46.777926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f88828c3634d04977ccf21eb350e88, user_id=058e5024f3fd4f00af564894a781cb0f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:46.780929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:46.835467 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2562/5132] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:46 2022] PUT /identity/v3/projects/96f88828c3634d04977ccf21eb350e88/users/058e5024f3fd4f00af564894a781cb0f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:46.848422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:46.848941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:46.849616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:46.948739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] MFA Rules not processed for user `058e5024f3fd4f00af564894a781cb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:46.950494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:47.178016 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2571/5133] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:47.195601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:47.197390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:47.264317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:47.267122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:47.457863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:47.467520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f4ee9-e7d0-430a-aaad-5bcc259dabc7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:47.476846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1420b15-a99e-4968-8a72-efa9b79b510e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:47.484529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:47.485687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56476835-073b-4a5e-a719-21c1b54982cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:47.486454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476835-073b-4a5e-a719-21c1b54982cd service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:47.486702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:47.486828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476835-073b-4a5e-a719-21c1b54982cd service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:47.486999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476835-073b-4a5e-a719-21c1b54982cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:47.487254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:47.488907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] PATH_INFO: `/v3/projects/e08a29b247f5439486596d3d96581443` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:47.493769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08a29b247f5439486596d3d96581443)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:47.496885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:47.535182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56476835-073b-4a5e-a719-21c1b54982cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:47.540437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56476835-073b-4a5e-a719-21c1b54982cd service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:47.565049 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2563/5134] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:47.578080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de868a0-2b66-4b73-83df-eaa4a5697d60 None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:47.578653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de868a0-2b66-4b73-83df-eaa4a5697d60 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:47.579090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de868a0-2b66-4b73-83df-eaa4a5697d60 None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:47.580803 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2564/5135] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:11:47 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:11:47.590066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:47.590636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:47.591558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:47.642815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08a29b247f5439486596d3d96581443', 'request_id': 'req-7962f5ab-895c-4779-b340-9164ab1fb826'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:47.654845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:47.663396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08a29b247f5439486596d3d96581443', 'request_id': 'req-7962f5ab-895c-4779-b340-9164ab1fb826'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:47.708505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08a29b247f5439486596d3d96581443', 'request_id': 'req-7962f5ab-895c-4779-b340-9164ab1fb826'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:47.712234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:47.713514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:47.726795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] The token cache is being invalidate because project e08a29b247f5439486596d3d96581443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:47.727173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:47.728325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7962f5ab-895c-4779-b340-9164ab1fb826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:47.731525 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2572/5136] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:47 2022] DELETE /identity/v3/projects/e08a29b247f5439486596d3d96581443 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:47.740275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccec19-fccf-4493-b578-3acd10e95ad8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:47.741528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccec19-fccf-4493-b578-3acd10e95ad8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:47.743167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccec19-fccf-4493-b578-3acd10e95ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:47.816442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ccccec19-fccf-4493-b578-3acd10e95ad8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:47.817711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ccccec19-fccf-4493-b578-3acd10e95ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:47.833795 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2565/5137] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:47.893609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:47.894846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:47.911197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b0fb6-5070-459d-af27-9459fda876c2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:47.915501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:47.916416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:47.917229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:47.917721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:47.941889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:47.943909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:47.948771 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2573/5138] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:47.950717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf8376-9327-467b-9772-ea2b51265169 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:47.950888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf8376-9327-467b-9772-ea2b51265169 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:47.951157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf8376-9327-467b-9772-ea2b51265169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:47.963903 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2566/5139] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:11:47.979327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-28cf8376-9327-467b-9772-ea2b51265169 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:47.980874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-28cf8376-9327-467b-9772-ea2b51265169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:47.989193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2574/5140] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:47.993568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:48.012899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd23e31-4366-4f59-96a4-bfc5ebbf6637 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:48.019894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:48.021580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:48.022033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:48.022362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:48.025698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:48.025698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:48.025698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:48.039657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2567/5141] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:48.043671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf8376-9327-467b-9772-ea2b51265169 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:48.066004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf8376-9327-467b-9772-ea2b51265169 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:48.075385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:48.076732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:48.077054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:48.077524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:48.078288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:48.081408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:48.143607 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2575/5142] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:48 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:48.147698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:48.172940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627011df-a475-4454-b404-8b955bdc91ef admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:48.284089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:48.284892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:48.285285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:48.286307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:48.287397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:48.290374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:48.336793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2568/5143] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:48 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:48.338103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:48.349832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8595-d4bb-4294-bc85-cd4cad4f7501 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:48.379694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:48.383758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:48.385321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:48.388118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:48.390428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:48.392538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:48.437781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:48.447536 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2576/5144] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:48.453448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339c1d4-1d32-4601-9517-6e62a4e676f1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:48.458950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef5a72-4358-45e9-ad2d-91fec52c7ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:48.459587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5a72-4358-45e9-ad2d-91fec52c7ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:48.459872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5a72-4358-45e9-ad2d-91fec52c7ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:48.460148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5a72-4358-45e9-ad2d-91fec52c7ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:48.460811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef5a72-4358-45e9-ad2d-91fec52c7ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:48.462510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef5a72-4358-45e9-ad2d-91fec52c7ea9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:48.487103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:48.490435 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2569/5145] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:48.494796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6d5c-3f6d-4f29-b267-2c03656ffca4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:48.500517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:48.501205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:48.501669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:48.501950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] PATH_INFO: `/v3/projects/9bf6442c28b64b5dab719d6b16828702/users/59136b959c564536ab3aee7749c7dd2d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:48.521398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf6442c28b64b5dab719d6b16828702, user_id=59136b959c564536ab3aee7749c7dd2d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:48.524400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:48.563819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab26f9-a6b4-40ec-bab1-b0299f6208b1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:48.564109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab26f9-a6b4-40ec-bab1-b0299f6208b1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:48.564455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab26f9-a6b4-40ec-bab1-b0299f6208b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:48.576536 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2577/5146] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:48 2022] PUT /identity/v3/projects/9bf6442c28b64b5dab719d6b16828702/users/59136b959c564536ab3aee7749c7dd2d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:48.637554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-52ab26f9-a6b4-40ec-bab1-b0299f6208b1 None None] MFA Rules not processed for user `59136b959c564536ab3aee7749c7dd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:48.642787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52ab26f9-a6b4-40ec-bab1-b0299f6208b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:48.799827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:48.842720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2570/5147] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:48.847642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:48.908593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:49.057925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4e557-d2c6-46e0-bccc-0842be2f1848 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:49.064340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:49.064977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:49.065362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:49.065672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:49.066396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:49.071039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:49.154348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab26f9-a6b4-40ec-bab1-b0299f6208b1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:49.156443 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2578/5148] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:48 2022] POST /identity/v3/projects => generated 373 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:49.179314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab26f9-a6b4-40ec-bab1-b0299f6208b1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:49.315231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:49.315915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:49.316143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:49.316409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:49.317081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:49.319706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:49.377658 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2571/5149] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:49 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:49.379163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:49.405673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3afb-2471-49cb-8cdf-dfe89e41c573 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:49.431236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:49.433342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:49.435501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:49.435919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:49.437455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:49.445869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:49.480167 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2579/5150] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:49.484625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:49.502830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6d529-2566-465c-a1cd-dadaff6eefe0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:49.528029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:49.530977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:49.533256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:49.535580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] PATH_INFO: `/v3/projects/415450d18f554bb5a99e5aa8259378f0/users/c0b56d0e8a7c498eb9de8c25e7aee359/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:49.587400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415450d18f554bb5a99e5aa8259378f0, user_id=c0b56d0e8a7c498eb9de8c25e7aee359, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:49.591078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:49.653119 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2572/5151] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:49 2022] PUT /identity/v3/projects/415450d18f554bb5a99e5aa8259378f0/users/c0b56d0e8a7c498eb9de8c25e7aee359/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:49.658912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:49.673908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f6775-c3b2-4ecb-87d2-31bbaa13653a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:49.933400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c22a95-adc3-4339-8f0a-ef61b74e5336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:49.934298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22a95-adc3-4339-8f0a-ef61b74e5336 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:49.934772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22a95-adc3-4339-8f0a-ef61b74e5336 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:49.935316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c22a95-adc3-4339-8f0a-ef61b74e5336 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:49.936225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c22a95-adc3-4339-8f0a-ef61b74e5336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:49.940577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c22a95-adc3-4339-8f0a-ef61b74e5336 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:49.964420 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2580/5152] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:49.968963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:49.988902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93590f-85e7-4e0d-98cb-4912a8c7fb80 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:50.026476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:50.037877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:50.038458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:50.039997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] PATH_INFO: `/v3/projects/415450d18f554bb5a99e5aa8259378f0/users/c0b56d0e8a7c498eb9de8c25e7aee359/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:50.046708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415450d18f554bb5a99e5aa8259378f0, user_id=c0b56d0e8a7c498eb9de8c25e7aee359, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:50.067801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:50.096493 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2573/5153] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:49 2022] PUT /identity/v3/projects/415450d18f554bb5a99e5aa8259378f0/users/c0b56d0e8a7c498eb9de8c25e7aee359/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:50.107043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcccce2-d362-4a2b-8135-2caf314a3c79 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:50.107427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcccce2-d362-4a2b-8135-2caf314a3c79 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:50.107866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcccce2-d362-4a2b-8135-2caf314a3c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:50.186675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bfcccce2-d362-4a2b-8135-2caf314a3c79 None None] MFA Rules not processed for user `c0b56d0e8a7c498eb9de8c25e7aee359`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:50.191222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bfcccce2-d362-4a2b-8135-2caf314a3c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:50.415645 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2581/5154] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:50.420185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:50.440680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539791c-cf09-424b-8d0a-0b49ecd86752 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:50.452589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:50.453414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:50.453863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:50.454344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:50.455476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:50.460223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:50.502265 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2574/5155] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:11:50 2022] POST /identity/v3/domains => generated 277 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:50.507508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcccce2-d362-4a2b-8135-2caf314a3c79 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:50.522187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcccce2-d362-4a2b-8135-2caf314a3c79 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:50.589600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:50.714722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:50.716871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:50.717378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:50.717927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:50.719073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:50.721690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:50.816644 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2582/5156] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:50 2022] POST /identity/v3/projects => generated 402 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:50.820649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:50.842064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920cbe9-eadf-4897-90ab-8b6a43b0befd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:50.982782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:50.985272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:50.986021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:50.986631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:50.987562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:51.004475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:51.046639 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2575/5157] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:11:50 2022] POST /identity/v3/groups => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:51.048314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:51.074695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a13a76-5910-4664-bf8e-7fc20c2c9ba4 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:51.094203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:51.095835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:51.096307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:51.096784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:51.097670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:51.108628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:51.158765 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2583/5158] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:51 2022] POST /identity/v3/users => generated 442 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:51.163787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:51.191606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f1c00-a09a-4bfb-b4b5-b329f1ee02c5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:51.214361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:51.215930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:51.218512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:51.220188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:51.223922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:51.226979 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:51.226979 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:51.226979 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:51.229183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:51.259705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2576/5159] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:11:51 2022] POST /identity/v3/roles => generated 239 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:51.261641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:51.276159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf66a6d-9d99-4b94-94fc-a9b624d15ed0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:51.297938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:51.298877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:51.302630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:51.303646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/groups/1b72e523613c4da690068577ccc6a603/roles/e5a6d35535c44d7fb1443c4bf07761a6/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:51.316758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b620963c219546c2af51ae1813c0c649, group_id=1b72e523613c4da690068577ccc6a603, role_id=e5a6d35535c44d7fb1443c4bf07761a6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:51.320728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:51.355545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2584/5160] 10.222.0.26 () {60 vars in 1585 bytes} [Sat Sep 17 21:11:51 2022] PUT /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/groups/1b72e523613c4da690068577ccc6a603/roles/e5a6d35535c44d7fb1443c4bf07761a6/inherited_to_projects => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:51.357817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:51.405716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1c71-7e1b-4b4d-bc54-ff8d240084f5 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:51.427703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:51.536828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:51.537971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:51.538283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:51.538726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/groups/1b72e523613c4da690068577ccc6a603/roles/e5a6d35535c44d7fb1443c4bf07761a6/inherited_to_projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:51.543534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:check_grant(project_id=b620963c219546c2af51ae1813c0c649, group_id=1b72e523613c4da690068577ccc6a603, role_id=e5a6d35535c44d7fb1443c4bf07761a6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:51.544668 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:51.544668 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:51.544668 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:51.546750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:51.572660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2577/5161] 10.222.0.26 () {58 vars in 1567 bytes} [Sat Sep 17 21:11:51 2022] HEAD /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/groups/1b72e523613c4da690068577ccc6a603/roles/e5a6d35535c44d7fb1443c4bf07761a6/inherited_to_projects => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:51.576912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:51.590970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ad915-1834-4360-92bc-165089ebdc1e tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:51.606397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:51.607377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:51.607872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:51.608825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/groups/1b72e523613c4da690068577ccc6a603/roles/e5a6d35535c44d7fb1443c4bf07761a6/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:51.612270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=b620963c219546c2af51ae1813c0c649, group_id=1b72e523613c4da690068577ccc6a603, role_id=e5a6d35535c44d7fb1443c4bf07761a6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:51.613963 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:51.613963 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:51.613963 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:51.616645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:51.640278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role e5a6d35535c44d7fb1443c4bf07761a6 was removed from group 1b72e523613c4da690068577ccc6a603 on project b620963c219546c2af51ae1813c0c649. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:51.640702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:51.642854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:51.712098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2585/5162] 10.222.0.26 () {58 vars in 1569 bytes} [Sat Sep 17 21:11:51 2022] DELETE /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/groups/1b72e523613c4da690068577ccc6a603/roles/e5a6d35535c44d7fb1443c4bf07761a6/inherited_to_projects => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:51.716501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:51.719993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:51.757717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcd5c6-9c09-44e8-88cc-1e3a57972c3f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:51.912057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:51.915562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:51.916083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:51.916526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles/e5a6d35535c44d7fb1443c4bf07761a6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:51.921953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e5a6d35535c44d7fb1443c4bf07761a6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:51.926339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:51.931307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:51.933382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:52.037036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role e5a6d35535c44d7fb1443c4bf07761a6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:52.037494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:52.039206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:52.044309 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2578/5163] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:51 2022] DELETE /identity/v3/roles/e5a6d35535c44d7fb1443c4bf07761a6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:52.058165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:52.078959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:52.100304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4876499-077b-46d9-9256-01db8962f8bb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:52.218242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:52.218884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:52.219163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:52.219433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:52.221363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:52.246835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613f3c-be7f-4437-be1e-8ec58195973c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:52.252084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:52.255332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:52.255765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:52.256260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:52.256543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:52.257742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:52.260706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:52.293234 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2579/5164] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:11:52 2022] POST /identity/v3/roles => generated 240 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:52.296283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:52.316994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864c64b-d8a2-49b2-ae22-b14a27d78eb0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:52.325075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:52.328080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:52.328499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:52.328845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/06210c5ddb6a4d34b7221ee1a6ebffe7/inherited_to_projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:52.333649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:52.354868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b620963c219546c2af51ae1813c0c649, user_id=741524fe132845cf8800955dba37a85b, role_id=06210c5ddb6a4d34b7221ee1a6ebffe7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:52.365731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:52.401566 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2580/5165] 10.222.0.26 () {60 vars in 1582 bytes} [Sat Sep 17 21:11:52 2022] PUT /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/06210c5ddb6a4d34b7221ee1a6ebffe7/inherited_to_projects => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:52.407712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:52.423717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:52.426075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:52.442655 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2586/5166] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:11:52.519876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156c76b-1d55-4562-998d-a08dba67e30a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:52.532723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:52.538006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:52.538006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:52.538006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/06210c5ddb6a4d34b7221ee1a6ebffe7/inherited_to_projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:52.540910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:check_grant(project_id=b620963c219546c2af51ae1813c0c649, user_id=741524fe132845cf8800955dba37a85b, role_id=06210c5ddb6a4d34b7221ee1a6ebffe7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:52.544684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:52.563549 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2581/5167] 10.222.0.26 () {58 vars in 1564 bytes} [Sat Sep 17 21:11:52 2022] HEAD /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/06210c5ddb6a4d34b7221ee1a6ebffe7/inherited_to_projects => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:52.565029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:52.580938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e550abb-686a-4e60-9801-597301e97c00 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:52.587753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:52.588743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:52.589094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:52.589497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/06210c5ddb6a4d34b7221ee1a6ebffe7/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:52.592989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=b620963c219546c2af51ae1813c0c649, user_id=741524fe132845cf8800955dba37a85b, role_id=06210c5ddb6a4d34b7221ee1a6ebffe7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:52.596049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:52.603314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 06210c5ddb6a4d34b7221ee1a6ebffe7 was removed from user 741524fe132845cf8800955dba37a85b on project b620963c219546c2af51ae1813c0c649. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:52.603614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:52.604328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:52.627471 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2587/5168] 10.222.0.26 () {58 vars in 1566 bytes} [Sat Sep 17 21:11:52 2022] DELETE /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/06210c5ddb6a4d34b7221ee1a6ebffe7/inherited_to_projects => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:52.633935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:52.636682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:52.709385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634bce3-7818-4183-a210-27bc5e382afd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:52.888762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:52.889430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:52.904483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:52.904807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles/06210c5ddb6a4d34b7221ee1a6ebffe7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:52.909165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=06210c5ddb6a4d34b7221ee1a6ebffe7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:52.913843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.045596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 06210c5ddb6a4d34b7221ee1a6ebffe7 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:53.045912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.046564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.055539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2582/5169] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:52 2022] DELETE /identity/v3/roles/06210c5ddb6a4d34b7221ee1a6ebffe7 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:53.055763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.082651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163e9e-3982-4360-b2bc-17e5c7f99d47 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:53.188533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:53.191627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.191914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.192190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.192888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:53.194947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.235568 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2588/5170] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:11:53 2022] POST /identity/v3/roles => generated 239 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:53.239564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.241405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:53.257035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897ff2d-3389-4b8c-b047-b739c25b20f2 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:53.262180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:53.262974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.263323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.263789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/1f3e38a671694f03a0c902edc54d0537/inherited_to_projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.266788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=4809f69d83914c828b3971244546c011, group_id=1b72e523613c4da690068577ccc6a603, role_id=1f3e38a671694f03a0c902edc54d0537)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:53.269628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.289801 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2583/5171] 10.222.0.26 () {60 vars in 1582 bytes} [Sat Sep 17 21:11:53 2022] PUT /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/1f3e38a671694f03a0c902edc54d0537/inherited_to_projects => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:53.293943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.307361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142373fc-196c-4bf6-b5e8-14c408291c00 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:53.427102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.429865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:53.440401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:53.441422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.441808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.442184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.452097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=4809f69d83914c828b3971244546c011, group_id=1b72e523613c4da690068577ccc6a603)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:53.453275 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:53.453275 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:53.453275 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:53.455241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.480619 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2589/5172] 10.222.0.26 () {58 vars in 1464 bytes} [Sat Sep 17 21:11:53 2022] GET /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/inherited_to_projects => generated 447 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:53.485803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.508023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f43b21-c0fd-4070-b0ae-acbe18b3fed8 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:53.512918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:53.513630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.513913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.514257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/1f3e38a671694f03a0c902edc54d0537/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.516658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=4809f69d83914c828b3971244546c011, group_id=1b72e523613c4da690068577ccc6a603, role_id=1f3e38a671694f03a0c902edc54d0537)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:53.517716 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:53.517716 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:53.517716 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:53.519701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.533033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:53.535086 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2590/5173] 10.222.0.26 () {58 vars in 1564 bytes} [Sat Sep 17 21:11:53 2022] HEAD /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/1f3e38a671694f03a0c902edc54d0537/inherited_to_projects => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:53.540410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.565447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898784b0-8753-4386-90c3-608515a2556f tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:53.567462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cc48a-bb10-494e-8bdb-65c2203f3dd1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:53.573148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:53.574174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.574707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.575121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] PATH_INFO: `/v3/users/1fbd80f0e5bf46b78a79fd821e9ff89e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.576555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fbd80f0e5bf46b78a79fd821e9ff89e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:53.579070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.579236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:53.579867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.581359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.581676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/1f3e38a671694f03a0c902edc54d0537/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.587550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=4809f69d83914c828b3971244546c011, group_id=1b72e523613c4da690068577ccc6a603, role_id=1f3e38a671694f03a0c902edc54d0537)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:53.595863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.603618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:53.606560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 1f3e38a671694f03a0c902edc54d0537 was removed from group 1b72e523613c4da690068577ccc6a603 on domain 4809f69d83914c828b3971244546c011. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:53.606886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.613673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.646034 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2591/5174] 10.222.0.26 () {58 vars in 1566 bytes} [Sat Sep 17 21:11:53 2022] DELETE /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/groups/1b72e523613c4da690068577ccc6a603/roles/1f3e38a671694f03a0c902edc54d0537/inherited_to_projects => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:53.651093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.653343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:53.695704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fbd80f0e5bf46b78a79fd821e9ff89e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.720308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fbd80f0e5bf46b78a79fd821e9ff89e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.752946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fbd80f0e5bf46b78a79fd821e9ff89e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.753844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fbd80f0e5bf46b78a79fd821e9ff89e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.755104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9766e34b-d155-4b18-b458-77344cfc0182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fbd80f0e5bf46b78a79fd821e9ff89e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:53.775298 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2584/5175] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:53 2022] DELETE /identity/v3/users/1fbd80f0e5bf46b78a79fd821e9ff89e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:53.780831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.781266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.781725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.789455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bca621-389d-4ccd-af5e-bd60c4afe57d tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:53.815914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:53.817540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:53.909825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:53.910188 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2585/5176] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:53.914639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:53.915080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:53.915384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles/1f3e38a671694f03a0c902edc54d0537` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:53.917215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1f3e38a671694f03a0c902edc54d0537)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:53.918933 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:53.918933 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:53.918933 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:53.920050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:53.926214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:53.928425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:54.005067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 1f3e38a671694f03a0c902edc54d0537 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:54.005500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:54.006358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:54.008549 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2592/5177] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:53 2022] DELETE /identity/v3/roles/1f3e38a671694f03a0c902edc54d0537 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:54.014418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.017975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:54.018850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278a66-5b1c-4caf-a440-34a925484aac None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.037613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:54.101577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.102209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.102499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.103106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.106884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:54.143778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb96e-f66e-4134-b762-af9737057e06 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.152084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.153115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.153627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.154142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.154997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.158169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.182491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:54.193897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2593/5178] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:11:54 2022] POST /identity/v3/roles => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:54.198784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.223835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.227253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.230465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc585e-83a1-41af-a71d-926b7ba7be89 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.244600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.245437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.246353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.246353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/0e3b295a9d6d4ada8508b4c086c8c8b7/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.250636 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2586/5179] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:54.251832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=4809f69d83914c828b3971244546c011, user_id=741524fe132845cf8800955dba37a85b, role_id=0e3b295a9d6d4ada8508b4c086c8c8b7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.255090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.273932 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2594/5180] 10.222.0.26 () {60 vars in 1579 bytes} [Sat Sep 17 21:11:54 2022] PUT /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/0e3b295a9d6d4ada8508b4c086c8c8b7/inherited_to_projects => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:54.278052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.294416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da75cd-dbf2-445a-8d79-3a9415083757 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.439548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.440886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.441358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.442046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/inherited_to_projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.445049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=4809f69d83914c828b3971244546c011, user_id=741524fe132845cf8800955dba37a85b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.446540 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:54.446540 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:54.446540 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:54.449926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.493236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.493708 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2587/5181] 10.222.0.26 () {58 vars in 1461 bytes} [Sat Sep 17 21:11:54 2022] GET /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/inherited_to_projects => generated 447 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:54.507340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1bb80-41cf-4c57-8e14-dc4fa7a26ad1 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.515532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.516632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.517401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.517984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/0e3b295a9d6d4ada8508b4c086c8c8b7/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.521722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=4809f69d83914c828b3971244546c011, user_id=741524fe132845cf8800955dba37a85b, role_id=0e3b295a9d6d4ada8508b4c086c8c8b7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.526034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.558708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.560376 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2595/5182] 10.222.0.26 () {58 vars in 1561 bytes} [Sat Sep 17 21:11:54 2022] HEAD /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/0e3b295a9d6d4ada8508b4c086c8c8b7/inherited_to_projects => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:54.576545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5aa73-a64a-41a7-a120-8ee228b802db tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.590285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.592602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.592982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.593430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/0e3b295a9d6d4ada8508b4c086c8c8b7/inherited_to_projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.596538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=4809f69d83914c828b3971244546c011, user_id=741524fe132845cf8800955dba37a85b, role_id=0e3b295a9d6d4ada8508b4c086c8c8b7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.597785 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:54.597785 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:54.597785 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:54.616688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.629497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 0e3b295a9d6d4ada8508b4c086c8c8b7 was removed from user 741524fe132845cf8800955dba37a85b on domain 4809f69d83914c828b3971244546c011. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:54.629497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:54.630800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:54.660354 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2588/5183] 10.222.0.26 () {58 vars in 1563 bytes} [Sat Sep 17 21:11:54 2022] DELETE /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/0e3b295a9d6d4ada8508b4c086c8c8b7/inherited_to_projects => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:54.664856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.669754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:54.674053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.675919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:54.688480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af7350-1949-47b1-a72e-5b54e0090f72 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.800980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.802168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.802689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.803240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles/0e3b295a9d6d4ada8508b4c086c8c8b7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.805696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:54.806989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0e3b295a9d6d4ada8508b4c086c8c8b7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.809191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.868922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba3a69-41a5-4109-b709-468c379e4256 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:54.888435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:54.889260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:54.889378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:54.893771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] PATH_INFO: `/v3/projects/4ceec62fde59414ebb34572d8f8fbcfe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:54.896649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ceec62fde59414ebb34572d8f8fbcfe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:54.899637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:54.911925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 0e3b295a9d6d4ada8508b4c086c8c8b7 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:54.912640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:54.914208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:54.919264 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2596/5184] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:54 2022] DELETE /identity/v3/roles/0e3b295a9d6d4ada8508b4c086c8c8b7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:54.924798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:54.928874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:55.033431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ceec62fde59414ebb34572d8f8fbcfe', 'request_id': 'req-968c16a0-90e3-44af-be22-8c720d2bf5d3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:55.047933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ceec62fde59414ebb34572d8f8fbcfe', 'request_id': 'req-968c16a0-90e3-44af-be22-8c720d2bf5d3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:55.057684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ceec62fde59414ebb34572d8f8fbcfe', 'request_id': 'req-968c16a0-90e3-44af-be22-8c720d2bf5d3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:55.095199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] The token cache is being invalidate because project 4ceec62fde59414ebb34572d8f8fbcfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:55.096604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:55.098790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-968c16a0-90e3-44af-be22-8c720d2bf5d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:55.103152 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2589/5185] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:54 2022] DELETE /identity/v3/projects/4ceec62fde59414ebb34572d8f8fbcfe => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:55.108872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96973-28b1-4558-9a76-1a3bcaba8a44 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.109833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96973-28b1-4558-9a76-1a3bcaba8a44 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.110994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96973-28b1-4558-9a76-1a3bcaba8a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.139435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d4b96973-28b1-4558-9a76-1a3bcaba8a44 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:55.141846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d4b96973-28b1-4558-9a76-1a3bcaba8a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:55.146409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab019a-a9d6-40d4-b5e2-bb5efabfe761 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:55.228459 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2590/5186] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:55.239419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ccbf8-5af0-4067-9369-ec23b7f4ae65 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.242929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ccbf8-5af0-4067-9369-ec23b7f4ae65 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.242929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ccbf8-5af0-4067-9369-ec23b7f4ae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.274628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:55.295545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fb9ccbf8-5af0-4067-9369-ec23b7f4ae65 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:55.298574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fb9ccbf8-5af0-4067-9369-ec23b7f4ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:55.316813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:55.318591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.318876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.320463 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2591/5187] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:55.320558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.320961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:55.321259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ccbf8-5af0-4067-9369-ec23b7f4ae65 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:55.329252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:55.336087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ccbf8-5af0-4067-9369-ec23b7f4ae65 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:55.343112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:55.345596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.346026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.346397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.347304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:55.347777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:55.349670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:55.364210 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2597/5188] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:11:54 2022] POST /identity/v3/roles => generated 240 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:55.366467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:55.370493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:55.374862 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2592/5189] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:55.380441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:55.403913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5afcb1-5402-4637-9ebf-0c1488777c5b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:55.418737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe69f0-cfa0-42ba-9537-0c70e8fada25 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:55.420797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:55.421436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.421740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.422367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.423017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:55.425113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:55.432103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:55.432742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.433022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.433289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.435135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:55.439744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:55.508243 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2593/5190] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:55 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:55.509228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:55.580440 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2598/5191] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:55 2022] POST /identity/v3/projects => generated 410 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:55.580774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:55.583338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:55.664412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4c6d2-e64a-4cad-b81a-0553003d70d0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:55.802426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ede8d-adde-4538-86a8-e122332ea403 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:55.817258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:55.820520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.822335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.822601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/05b0775029984f0ea494dcccb40ef78b/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.828944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=4809f69d83914c828b3971244546c011, user_id=741524fe132845cf8800955dba37a85b, role_id=05b0775029984f0ea494dcccb40ef78b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:55.837451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:55.869601 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2599/5192] 10.222.0.26 () {60 vars in 1579 bytes} [Sat Sep 17 21:11:55 2022] PUT /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/05b0775029984f0ea494dcccb40ef78b/inherited_to_projects => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:55.873484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:55.886249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:55.886690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:55.887008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:55.887284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:55.888957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:55.890950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:55.942332 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2594/5193] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:55 2022] POST /identity/v3/users => generated 300 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:55.946484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:55.968769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981288df-b9ba-4080-81d3-360cfba2217c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:55.998472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653b842-ab0b-4e6a-9909-2f4dccc61190 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.007262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.007909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.008295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.008581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.009322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Adding query filter params (scope.project.id=b620963c219546c2af51ae1813c0c649, user.id=741524fe132845cf8800955dba37a85b) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:56.009659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.010872 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:56.010872 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:11:56.010872 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:11:56.012384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:56.092843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.093740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.094211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.094828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.095720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.097724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:56.114456 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2595/5194] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:55 2022] GET /identity/v3/roles => generated 1651 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:11:56.116377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:56.120792 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2600/5195] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:11:55 2022] GET /identity/v3/role_assignments?scope.project.id=b620963c219546c2af51ae1813c0c649&user.id=741524fe132845cf8800955dba37a85b&effective => generated 648 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:56.125838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:56.139460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783d2d3-b570-4b88-84c5-505188b26622 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.147543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df467c-ed01-490a-96af-782dfa2f8d90 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.158565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.159213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.159497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.159772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.160572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Adding query filter params (scope.project.id=42ef8f2783f748adae48ce0887759372, user.id=741524fe132845cf8800955dba37a85b) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:56.160883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.167108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.167873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.168277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.168645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.169452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:56.170003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.172183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:56.172962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:56.195698 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2596/5196] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:56 2022] GET /identity/v3/roles => generated 1651 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:56.201120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:56.222615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458c00e-86ba-4e2c-a852-9577ca3b4d9d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.236922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd645df-67c3-4425-82dc-321354ceb48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.239308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd645df-67c3-4425-82dc-321354ceb48e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.239590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd645df-67c3-4425-82dc-321354ceb48e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.239859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd645df-67c3-4425-82dc-321354ceb48e admin admin] PATH_INFO: `/v3/projects/cbb1a70cd16e44b599ca8fbfc8ab2b68/users/a8779d6467bd4be0878e1bfa5a32fb2e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.251085 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2601/5197] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:11:56 2022] GET /identity/v3/role_assignments?scope.project.id=42ef8f2783f748adae48ce0887759372&user.id=741524fe132845cf8800955dba37a85b&effective => generated 648 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:56.255941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:56.272146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd645df-67c3-4425-82dc-321354ceb48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb1a70cd16e44b599ca8fbfc8ab2b68, user_id=a8779d6467bd4be0878e1bfa5a32fb2e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.276677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd645df-67c3-4425-82dc-321354ceb48e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:56.304806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3aed7-fa9f-4246-969e-391810c36bc0 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.306097 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2597/5198] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:56 2022] PUT /identity/v3/projects/cbb1a70cd16e44b599ca8fbfc8ab2b68/users/a8779d6467bd4be0878e1bfa5a32fb2e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:56.316476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.317169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.317696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.401314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] MFA Rules not processed for user `a8779d6467bd4be0878e1bfa5a32fb2e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:56.406248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:56.461527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:56.487435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.488966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.489324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.489596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/05b0775029984f0ea494dcccb40ef78b/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.492405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=4809f69d83914c828b3971244546c011, user_id=741524fe132845cf8800955dba37a85b, role_id=05b0775029984f0ea494dcccb40ef78b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.495641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:56.507918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 05b0775029984f0ea494dcccb40ef78b was removed from user 741524fe132845cf8800955dba37a85b on domain 4809f69d83914c828b3971244546c011. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:56.508266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:56.508970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:56.539745 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2602/5199] 10.222.0.26 () {58 vars in 1563 bytes} [Sat Sep 17 21:11:56 2022] DELETE /identity/v3/OS-INHERIT/domains/4809f69d83914c828b3971244546c011/users/741524fe132845cf8800955dba37a85b/roles/05b0775029984f0ea494dcccb40ef78b/inherited_to_projects => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:56.544420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:56.548637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:56.577174 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2598/5200] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:56.590740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:56.594024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:56.701501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b5b88-6935-4652-b6b0-ad3091a17c93 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.717746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.718308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.718649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.718934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.720120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Adding query filter params (scope.project.id=b620963c219546c2af51ae1813c0c649, user.id=741524fe132845cf8800955dba37a85b) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:56.720433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.723007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:56.847044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1353bba-e463-46d0-86b7-61938a20909d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.848060 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2603/5201] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:11:56 2022] GET /identity/v3/role_assignments?scope.project.id=b620963c219546c2af51ae1813c0c649&user.id=741524fe132845cf8800955dba37a85b&effective => generated 228 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:56.852434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:56.868380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:56.875366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68479c-0e68-4b19-8eb0-cd7321e7843b tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:56.889657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:56.890302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:56.891493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:56.891632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:56.895293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Adding query filter params (scope.project.id=42ef8f2783f748adae48ce0887759372, user.id=741524fe132845cf8800955dba37a85b) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:56.895293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:56.899379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:57.010709 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2604/5202] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:11:56 2022] GET /identity/v3/role_assignments?scope.project.id=42ef8f2783f748adae48ce0887759372&user.id=741524fe132845cf8800955dba37a85b&effective => generated 228 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:57.015695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:57.026238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:57.027211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:57.027719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:57.028213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:57.042039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069488-1d19-49c3-bdc3-94a6f113d5f6 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:57.061949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:57.064942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:57.065231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:57.065696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/projects/42ef8f2783f748adae48ce0887759372` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:57.068202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_project(project_id=42ef8f2783f748adae48ce0887759372)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:57.071553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:57.153100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:57.173079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ef8f2783f748adae48ce0887759372', 'request_id': 'req-1a87862f-a315-4690-bf1b-19f88294795a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.187221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ef8f2783f748adae48ce0887759372', 'request_id': 'req-1a87862f-a315-4690-bf1b-19f88294795a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.190158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:57.192057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:57.207904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ef8f2783f748adae48ce0887759372', 'request_id': 'req-1a87862f-a315-4690-bf1b-19f88294795a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.208151 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2599/5203] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:57.237677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] The token cache is being invalidate because project 42ef8f2783f748adae48ce0887759372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:57.237968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.240006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.247183 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2605/5204] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:57 2022] DELETE /identity/v3/projects/42ef8f2783f748adae48ce0887759372 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:57.249137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:57.268223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:57.270207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:57.276681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d3bf5-71ea-4af5-8e03-0475e055daae service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:57.420927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:57.421763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:57.422050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:57.422327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles/05b0775029984f0ea494dcccb40ef78b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:57.423956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=05b0775029984f0ea494dcccb40ef78b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:57.425782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:57.477921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87862f-a315-4690-bf1b-19f88294795a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:57.492353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:57.493568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:57.494476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:57.495096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] PATH_INFO: `/v3/users/a8779d6467bd4be0878e1bfa5a32fb2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:57.496829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8779d6467bd4be0878e1bfa5a32fb2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:57.498249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 05b0775029984f0ea494dcccb40ef78b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:57.498558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.500306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.500812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:57.511558 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2600/5205] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:57 2022] DELETE /identity/v3/roles/05b0775029984f0ea494dcccb40ef78b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:57.512891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:57.517738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:57.609796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:57.637381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8779d6467bd4be0878e1bfa5a32fb2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.638425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8779d6467bd4be0878e1bfa5a32fb2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.659711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8779d6467bd4be0878e1bfa5a32fb2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.660767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8779d6467bd4be0878e1bfa5a32fb2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.704622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b2985-ac36-4a1c-b6f6-b4ff2648e4a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:57.707272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-44c4dfab-428e-435b-84c7-352c031bfe6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8779d6467bd4be0878e1bfa5a32fb2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:57.720049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:57.720608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:57.720893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:57.721169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:57.721878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:57.723537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:57.735598 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2606/5206] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:57 2022] DELETE /identity/v3/users/a8779d6467bd4be0878e1bfa5a32fb2e => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:57.740095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:57.740798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:57.742086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:57.776545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2601/5207] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:11:57 2022] POST /identity/v3/roles => generated 240 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:57.778183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:57.783047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:57.794271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:57.797342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:57.814255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba403ec-34a9-48df-bb27-3a4201593180 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:57.943394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:57.961692 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2607/5208] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:57.978249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:57.979061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:57.979436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:57.979804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:57.980769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:57.982977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:58.011036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:58.012855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:58.073222 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2602/5209] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:57 2022] POST /identity/v3/projects => generated 409 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:58.077517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:58.142023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49634f-e1f0-45f5-9993-c1d43251dd19 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:58.258477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715d799-b9ca-4568-9e0f-eb10051f73a3 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:58.271356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:58.272377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:58.273017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:58.273475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/64f25fdf53c34a598f6ce9ce14a61f58/inherited_to_projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:58.287426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b620963c219546c2af51ae1813c0c649, user_id=741524fe132845cf8800955dba37a85b, role_id=64f25fdf53c34a598f6ce9ce14a61f58)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:58.296919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:58.298465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:58.299117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:58.299385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:58.299653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:58.329375 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2603/5210] 10.222.0.26 () {60 vars in 1582 bytes} [Sat Sep 17 21:11:58 2022] PUT /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/64f25fdf53c34a598f6ce9ce14a61f58/inherited_to_projects => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:58.334174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:58.390411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:58.476241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354ca7d-e1d7-4085-8c25-44e2c28648fe tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:58.477621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:58.479349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:58.481710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:58.482717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:58.483000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:58.483274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:58.484016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Adding query filter params (scope.project.id=ee8901a40ef74be892e231165c602ea4, user.id=741524fe132845cf8800955dba37a85b) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:58.484322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:58.485811 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:11:58.485811 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:11:58.485811 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:11:58.486964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:58.505774 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2608/5211] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:11:58.572390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2604/5212] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:11:58 2022] GET /identity/v3/role_assignments?scope.project.id=ee8901a40ef74be892e231165c602ea4&user.id=741524fe132845cf8800955dba37a85b&effective => generated 649 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:58.576762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:58.597686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ebf3b-8c33-4db5-8138-49fceec3055a service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:58.605206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:58.606162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:58.606550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:58.606934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/64f25fdf53c34a598f6ce9ce14a61f58/inherited_to_projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:58.609971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=b620963c219546c2af51ae1813c0c649, user_id=741524fe132845cf8800955dba37a85b, role_id=64f25fdf53c34a598f6ce9ce14a61f58)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:58.614226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:58.623061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 64f25fdf53c34a598f6ce9ce14a61f58 was removed from user 741524fe132845cf8800955dba37a85b on project b620963c219546c2af51ae1813c0c649. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:58.623513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:58.624924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:58.660774 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2609/5213] 10.222.0.26 () {58 vars in 1566 bytes} [Sat Sep 17 21:11:58 2022] DELETE /identity/v3/OS-INHERIT/projects/b620963c219546c2af51ae1813c0c649/users/741524fe132845cf8800955dba37a85b/roles/64f25fdf53c34a598f6ce9ce14a61f58/inherited_to_projects => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:58.661970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:58.663804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:58.701497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702755cd-9479-4993-8f2f-4668aec5335c tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:58.720720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:58.798905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:58.799440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:58.799774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:58.800097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:58.801576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Adding query filter params (scope.project.id=ee8901a40ef74be892e231165c602ea4, user.id=741524fe132845cf8800955dba37a85b) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:58.801576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:58.803176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:58.877064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2605/5214] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:11:58 2022] GET /identity/v3/role_assignments?scope.project.id=ee8901a40ef74be892e231165c602ea4&user.id=741524fe132845cf8800955dba37a85b&effective => generated 228 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:58.880943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:58.900281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81649a-9ba9-4516-be47-a087b3852ffb tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:58.932591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:58.934300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:58.934559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:58.934833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/projects/ee8901a40ef74be892e231165c602ea4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:58.938480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ee8901a40ef74be892e231165c602ea4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:58.949773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:59.057411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8901a40ef74be892e231165c602ea4', 'request_id': 'req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.072321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8901a40ef74be892e231165c602ea4', 'request_id': 'req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.105986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8901a40ef74be892e231165c602ea4', 'request_id': 'req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.133856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] The token cache is being invalidate because project ee8901a40ef74be892e231165c602ea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:59.134155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.134831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.140791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2610/5215] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:58 2022] DELETE /identity/v3/projects/ee8901a40ef74be892e231165c602ea4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:59.141982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:59.144462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:59.168710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af070f0-3387-4a71-ba39-d6bec50f0686 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:59.323543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:59.324661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:59.325144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:59.325890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/roles/64f25fdf53c34a598f6ce9ce14a61f58` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:59.329211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_role(role_id=64f25fdf53c34a598f6ce9ce14a61f58)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:59.331166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:59.378765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invalidating the token cache because role 64f25fdf53c34a598f6ce9ce14a61f58 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:59.379359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.380570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.386602 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2606/5216] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:59 2022] DELETE /identity/v3/roles/64f25fdf53c34a598f6ce9ce14a61f58 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:59.388322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:59.399930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55ee31-74b6-48d3-b24c-6dcadac03e9a tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:59.503959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:59.505062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:59.505600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:59.506079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/users/741524fe132845cf8800955dba37a85b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:59.508236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_user(user_id=741524fe132845cf8800955dba37a85b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:59.512077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:59.606594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:59.643580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '741524fe132845cf8800955dba37a85b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.644838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '741524fe132845cf8800955dba37a85b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.666632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '741524fe132845cf8800955dba37a85b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.667581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '741524fe132845cf8800955dba37a85b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.717957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '741524fe132845cf8800955dba37a85b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:59.755303 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2611/5217] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:59 2022] DELETE /identity/v3/users/741524fe132845cf8800955dba37a85b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:59.759453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:59.761169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:59.784459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927a22f-43f5-437f-8b2e-2cbb263b0eec tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:59.879001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:59.916160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:59.916909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:59.917255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:59.917628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/groups/1b72e523613c4da690068577ccc6a603` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:59.923680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_group(group_id=1b72e523613c4da690068577ccc6a603)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:59.923680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:00.017664 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2607/5218] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:11:59 2022] DELETE /identity/v3/groups/1b72e523613c4da690068577ccc6a603 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:00.022096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:00.023820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:00.051540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c127cb3-0834-48d5-bae4-2285f2a827cd tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:00.146686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:00.150811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:00.220276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:00.221254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:00.221710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:00.222130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/projects/b620963c219546c2af51ae1813c0c649` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:00.224157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b620963c219546c2af51ae1813c0c649)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:00.227329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:00.306734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a372b1c-7972-4f59-b6d8-a0aaa2e3a9e9 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:00.327079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b620963c219546c2af51ae1813c0c649', 'request_id': 'req-45ccfc73-b004-40be-8f6f-66f7d24eb246'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.336186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b620963c219546c2af51ae1813c0c649', 'request_id': 'req-45ccfc73-b004-40be-8f6f-66f7d24eb246'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.353925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b620963c219546c2af51ae1813c0c649', 'request_id': 'req-45ccfc73-b004-40be-8f6f-66f7d24eb246'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.379422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] The token cache is being invalidate because project b620963c219546c2af51ae1813c0c649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:00.379645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.381036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.385661 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2612/5219] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:00 2022] DELETE /identity/v3/projects/b620963c219546c2af51ae1813c0c649 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:00.387145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:00.388924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:00.438352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:00.439249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:00.439675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:00.440091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] PATH_INFO: `/v3/projects/cbb1a70cd16e44b599ca8fbfc8ab2b68` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:00.441833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb1a70cd16e44b599ca8fbfc8ab2b68)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:00.443968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:00.480754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:00.525329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccfc73-b004-40be-8f6f-66f7d24eb246 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:00.539521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:00.555427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb1a70cd16e44b599ca8fbfc8ab2b68', 'request_id': 'req-c62d5fab-e291-4610-abe8-a7073101dc6f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.576532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb1a70cd16e44b599ca8fbfc8ab2b68', 'request_id': 'req-c62d5fab-e291-4610-abe8-a7073101dc6f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.584054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb1a70cd16e44b599ca8fbfc8ab2b68', 'request_id': 'req-c62d5fab-e291-4610-abe8-a7073101dc6f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.610537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] The token cache is being invalidate because project cbb1a70cd16e44b599ca8fbfc8ab2b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:00.611450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.613995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c62d5fab-e291-4610-abe8-a7073101dc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.620236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2608/5220] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:00 2022] DELETE /identity/v3/projects/cbb1a70cd16e44b599ca8fbfc8ab2b68 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:00.627381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4b50-9790-4b94-be5a-35c5f326c53f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:00.627381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4b50-9790-4b94-be5a-35c5f326c53f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:00.627708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4b50-9790-4b94-be5a-35c5f326c53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:00.675137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:00.675865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:00.676142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:00.676410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/domains/4809f69d83914c828b3971244546c011` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:00.678767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-635a4b50-9790-4b94-be5a-35c5f326c53f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:00.680138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-635a4b50-9790-4b94-be5a-35c5f326c53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:00.680265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4809f69d83914c828b3971244546c011)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:00.681760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:00.726537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4809f69d83914c828b3971244546c011'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.728665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4809f69d83914c828b3971244546c011'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.785133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4809f69d83914c828b3971244546c011'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.786147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4809f69d83914c828b3971244546c011'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:00.790479 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2613/5221] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:12:00 2022] PATCH /identity/v3/domains/4809f69d83914c828b3971244546c011 => generated 278 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:12:00.795527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:00.798732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:00.834046 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2609/5222] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:00.850848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea66c2-2a8f-4094-a407-a6c5c1e72799 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:00.854121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea66c2-2a8f-4094-a407-a6c5c1e72799 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:00.854121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea66c2-2a8f-4094-a407-a6c5c1e72799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:00.904629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-68ea66c2-2a8f-4094-a407-a6c5c1e72799 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:00.907246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-68ea66c2-2a8f-4094-a407-a6c5c1e72799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:00.973036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bcaf8-ccde-4933-b869-02b703075a59 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:00.987585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0b56d0e8a7c498eb9de8c25e7aee359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '415450d18f554bb5a99e5aa8259378f0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:00.991137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:00.992560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:00.992855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] PATH_INFO: `/v3/domains/4809f69d83914c828b3971244546c011` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.054009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4809f69d83914c828b3971244546c011)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:01.056626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:01.082239 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2610/5223] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:01.084485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea66c2-2a8f-4094-a407-a6c5c1e72799 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:01.096657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea66c2-2a8f-4094-a407-a6c5c1e72799 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:01.119287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:01.120079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:01.120456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:01.121092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.122036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:01.122919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:01.125157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:01.145308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4809f69d83914c828b3971244546c011'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.150379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4809f69d83914c828b3971244546c011'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.176429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:01.177750 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2611/5224] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:01.183215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:01.232893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4809f69d83914c828b3971244546c011', 'request_id': 'req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.236346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27066c8a-4414-4b04-9b77-5f8d109bf040 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:01.254084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4809f69d83914c828b3971244546c011', 'request_id': 'req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.279917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4809f69d83914c828b3971244546c011', 'request_id': 'req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.327854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] The token cache is being invalidate because project 4809f69d83914c828b3971244546c011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:01.328245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.329541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.355744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4809f69d83914c828b3971244546c011', 'request_id': 'req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.359752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4809f69d83914c828b3971244546c011', 'request_id': 'req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.362786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4809f69d83914c828b3971244546c011', 'request_id': 'req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.378970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:01.388188 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2614/5225] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:12:00 2022] DELETE /identity/v3/domains/4809f69d83914c828b3971244546c011 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:01.389692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:01.396975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:01.413338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:01.415552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:01.417362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:01.417798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.418921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:01.423012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:01.434807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d2bf7-4f08-4597-b4c2-40f99d0b1bc7 tempest-InheritsV3TestJSON-2113794421 tempest-InheritsV3TestJSON-2113794421-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:01.451066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:01.451200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:01.451779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:01.451779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] PATH_INFO: `/v3/users/59136b959c564536ab3aee7749c7dd2d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.453021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59136b959c564536ab3aee7749c7dd2d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:01.461392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:01.478298 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2612/5226] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:01 2022] POST /identity/v3/projects => generated 383 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:01.484556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:01.485781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:01.573811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59136b959c564536ab3aee7749c7dd2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.575649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59136b959c564536ab3aee7749c7dd2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.599895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59136b959c564536ab3aee7749c7dd2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.600908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59136b959c564536ab3aee7749c7dd2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.630669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb9f824b-0861-4817-b593-89e6c30cd529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59136b959c564536ab3aee7749c7dd2d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:01.644386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb72bf-d02a-4e5a-a593-63c7f073da94 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:01.662368 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2615/5227] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:01 2022] DELETE /identity/v3/users/59136b959c564536ab3aee7749c7dd2d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:01.662621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:01.662621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:01.663220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:01.663684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.664116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:01.664390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:01.664714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.665961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:01.668837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:01.705840 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2613/5228] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:01 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:01.707227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:01.709695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:01.731763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc0452-addc-4ef1-be36-825c7d67b731 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:01.732752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:01.736988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:01.830758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:01.858024 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2616/5229] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:01.879187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:01.881251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:01.881841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:01.882408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.883232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:01.885207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:01.916763 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2614/5230] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:01.917055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:01.918776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:01.925217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:01.948938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e8ec-4383-4cd6-a695-92af93f3fc24 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:01.961608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:01.962275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:01.964911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:01.965517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:01.967410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:01.970912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:02.010579 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2615/5231] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:02.012333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:02.024260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2501813-7f9a-40ed-89b3-a520aefc737e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:02.039695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725ec7b-546c-4fe3-a3b9-cab63bd26bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:02.040587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ec7b-546c-4fe3-a3b9-cab63bd26bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:02.040997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ec7b-546c-4fe3-a3b9-cab63bd26bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:02.042131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ec7b-546c-4fe3-a3b9-cab63bd26bd0 admin admin] PATH_INFO: `/v3/projects/3723f4e7f0bb4a71ac7aaf46def6b4f6/users/2d249eb9b0124c339d8e730c96649171/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:02.065680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cef3e-e88e-4a89-9787-ca759b21eef1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:02.091428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725ec7b-546c-4fe3-a3b9-cab63bd26bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3723f4e7f0bb4a71ac7aaf46def6b4f6, user_id=2d249eb9b0124c339d8e730c96649171, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:02.094327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725ec7b-546c-4fe3-a3b9-cab63bd26bd0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:02.094548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:02.095430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:02.100987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:02.101452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:02.131655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:02.133324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:02.143918 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2616/5232] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:02 2022] PUT /identity/v3/projects/3723f4e7f0bb4a71ac7aaf46def6b4f6/users/2d249eb9b0124c339d8e730c96649171/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:02.145574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d6f68-465f-4af4-8e3d-5b2e36ed195b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:02.145869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d6f68-465f-4af4-8e3d-5b2e36ed195b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:02.146145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d6f68-465f-4af4-8e3d-5b2e36ed195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:02.172679 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2617/5233] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:02.201716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a52d6f68-465f-4af4-8e3d-5b2e36ed195b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:02.219532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a52d6f68-465f-4af4-8e3d-5b2e36ed195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:02.294530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:02.384025 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2617/5234] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:02.417616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:02.419499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:02.438468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd71c7f-f599-454b-8473-266ea93233d6 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:02.585662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:02.586450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:02.586873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:02.587258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:02.612050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:02.614051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:02.632946 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2618/5235] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:03.333405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06847245-4f1a-4ddf-aabd-feadec057988 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:03.336661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06847245-4f1a-4ddf-aabd-feadec057988 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:03.336661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06847245-4f1a-4ddf-aabd-feadec057988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:03.387070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-06847245-4f1a-4ddf-aabd-feadec057988 None None] MFA Rules not processed for user `b2704d86338949ba9bb87bf60ee24146`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:03.397201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-06847245-4f1a-4ddf-aabd-feadec057988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:03.402728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:03.405650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:03.451414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed548f8f-afcb-4581-8996-e2ea32b75b57 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:03.500300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:03.507379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:03.509106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:03.510120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] PATH_INFO: `/v3/users/058e5024f3fd4f00af564894a781cb0f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:03.518793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] RBAC: Authorizing `identity:delete_user(user_id=058e5024f3fd4f00af564894a781cb0f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:03.524207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:03.611282 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2618/5236] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:03.622639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:03.622967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:03.623205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:03.663906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058e5024f3fd4f00af564894a781cb0f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:03.665017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058e5024f3fd4f00af564894a781cb0f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:03.679466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058e5024f3fd4f00af564894a781cb0f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:03.680633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058e5024f3fd4f00af564894a781cb0f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:03.699796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] MFA Rules not processed for user `adfe77cd9de149508efbea44af748876`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:03.701065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:03.703258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49c21857-678b-4463-9954-143cfdfcf24e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058e5024f3fd4f00af564894a781cb0f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:03.723227 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2619/5237] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:03 2022] DELETE /identity/v3/users/058e5024f3fd4f00af564894a781cb0f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:03.725158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:03.725433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:03.725673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:03.754286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:03.755696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:03.776803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:03.844324 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2619/5238] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:03.845608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:03.848409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:03.878870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d6cbc-9a93-4065-87b5-fc093d489b0f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:03.904325 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2620/5239] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:03.908572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:03.910202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:04.017664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:04.018344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:04.018644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:04.018885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] PATH_INFO: `/v3/projects/9bf6442c28b64b5dab719d6b16828702` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:04.025899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf6442c28b64b5dab719d6b16828702)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:04.027804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:04.047628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2d05a-35a7-4d24-8fde-c3e8dc162367 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:04.055442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:04.066634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:04.067552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:04.068387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:04.068743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:04.071984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:04.135816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf6442c28b64b5dab719d6b16828702', 'request_id': 'req-93ef1400-4683-43ac-be76-cf2f051e72b4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.157631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf6442c28b64b5dab719d6b16828702', 'request_id': 'req-93ef1400-4683-43ac-be76-cf2f051e72b4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.167909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf6442c28b64b5dab719d6b16828702', 'request_id': 'req-93ef1400-4683-43ac-be76-cf2f051e72b4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.203111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] The token cache is being invalidate because project 9bf6442c28b64b5dab719d6b16828702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:04.203417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.209974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.211920 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2620/5240] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:03 2022] DELETE /identity/v3/projects/9bf6442c28b64b5dab719d6b16828702 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:04.212837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:04.215641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:04.266195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:04.268309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:04.303254 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2621/5241] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:03 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:04.305570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:04.333534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eeee5-592f-423d-b5f3-f2df470c3f39 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:04.386687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef1400-4683-43ac-be76-cf2f051e72b4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:04.396886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:04.397204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:04.398396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:04.398682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:04.403147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:04.535767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:04.547403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:04.548950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:04.549164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:04.550146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:04.550576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:04.550994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] PATH_INFO: `/v3/users/c0b56d0e8a7c498eb9de8c25e7aee359` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:04.553583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b56d0e8a7c498eb9de8c25e7aee359)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:04.556138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:04.579535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2621/5242] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:04.663028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b56d0e8a7c498eb9de8c25e7aee359'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.666003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b56d0e8a7c498eb9de8c25e7aee359'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.691380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b56d0e8a7c498eb9de8c25e7aee359'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.692376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b56d0e8a7c498eb9de8c25e7aee359'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.727801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b56d0e8a7c498eb9de8c25e7aee359'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:04.764671 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2622/5243] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:04 2022] DELETE /identity/v3/users/c0b56d0e8a7c498eb9de8c25e7aee359 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:04.945158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:04.947106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:04.968836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803846a-a2d2-4b97-8996-65890a2e7ee5 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:05.068938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:05.087191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:05.094449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:05.094739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:05.095045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] PATH_INFO: `/v3/users/c544e8e6f7a741c985c92ed7437397cd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:05.096484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c544e8e6f7a741c985c92ed7437397cd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:05.098576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:05.213562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c544e8e6f7a741c985c92ed7437397cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:05.240463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c544e8e6f7a741c985c92ed7437397cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:05.268003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c544e8e6f7a741c985c92ed7437397cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:05.268718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c544e8e6f7a741c985c92ed7437397cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:05.269608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c544e8e6f7a741c985c92ed7437397cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:05.289248 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2622/5244] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:04 2022] DELETE /identity/v3/users/c544e8e6f7a741c985c92ed7437397cd => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:06.467472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:06.469778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:06.496399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a4f73-1bcd-4b8c-9612-5269b585fcff admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:06.546740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:06.548863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:06.694041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23098370-4d73-40f1-bd42-6f8338e52659 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:06.700940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:06.703186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:06.703476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:06.703763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] PATH_INFO: `/v3/projects/415450d18f554bb5a99e5aa8259378f0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:06.708392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:06.709372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] RBAC: Authorizing `identity:delete_project(project_id=415450d18f554bb5a99e5aa8259378f0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:06.710778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:06.711421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:06.712931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] PATH_INFO: `/v3/projects/6fa7e14863254ab3964ac907c168d9e6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:06.714585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:06.715623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa7e14863254ab3964ac907c168d9e6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:06.721530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:06.826848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa7e14863254ab3964ac907c168d9e6', 'request_id': 'req-de343f59-f326-4798-8ad7-871fa754b4fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.843440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa7e14863254ab3964ac907c168d9e6', 'request_id': 'req-de343f59-f326-4798-8ad7-871fa754b4fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.853909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa7e14863254ab3964ac907c168d9e6', 'request_id': 'req-de343f59-f326-4798-8ad7-871fa754b4fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.860745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415450d18f554bb5a99e5aa8259378f0', 'request_id': 'req-b3e00402-ac68-41f4-bd21-b565a0f3c28b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.869920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:06.872697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415450d18f554bb5a99e5aa8259378f0', 'request_id': 'req-b3e00402-ac68-41f4-bd21-b565a0f3c28b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.891079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] The token cache is being invalidate because project 6fa7e14863254ab3964ac907c168d9e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:06.891388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.892141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.894005 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2623/5245] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:06 2022] DELETE /identity/v3/projects/6fa7e14863254ab3964ac907c168d9e6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:06.897336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415450d18f554bb5a99e5aa8259378f0', 'request_id': 'req-b3e00402-ac68-41f4-bd21-b565a0f3c28b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.898305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:06.902207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:06.942399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] The token cache is being invalidate because project 415450d18f554bb5a99e5aa8259378f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:06.944371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.946007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b3e00402-ac68-41f4-bd21-b565a0f3c28b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:06.951909 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2623/5246] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:06 2022] DELETE /identity/v3/projects/415450d18f554bb5a99e5aa8259378f0 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:06.953805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782fba3-8b90-48c0-b186-95539d8f1f65 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:06.954195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782fba3-8b90-48c0-b186-95539d8f1f65 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:06.954628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782fba3-8b90-48c0-b186-95539d8f1f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:06.975303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:07.012800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3782fba3-8b90-48c0-b186-95539d8f1f65 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:07.015680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3782fba3-8b90-48c0-b186-95539d8f1f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:07.152490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de343f59-f326-4798-8ad7-871fa754b4fb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:07.235511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2624/5247] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:07.236995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60a29b-5612-42a1-9721-6210285341b5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.237287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60a29b-5612-42a1-9721-6210285341b5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.238906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60a29b-5612-42a1-9721-6210285341b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.295012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:07.295804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.296036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.296322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] PATH_INFO: `/v3/projects/96f88828c3634d04977ccf21eb350e88` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.297854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f88828c3634d04977ccf21eb350e88)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:07.300042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:07.321756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4a60a29b-5612-42a1-9721-6210285341b5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:07.330060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a60a29b-5612-42a1-9721-6210285341b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:07.342109 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2625/5248] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:07.343542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe07f5-1438-4c9b-b23a-4b74da5d9381 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.343837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe07f5-1438-4c9b-b23a-4b74da5d9381 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.344156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe07f5-1438-4c9b-b23a-4b74da5d9381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.394201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-03fe07f5-1438-4c9b-b23a-4b74da5d9381 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:07.397747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-03fe07f5-1438-4c9b-b23a-4b74da5d9381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:07.444080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f88828c3634d04977ccf21eb350e88', 'request_id': 'req-ccb68662-143f-454b-9c2a-562669bc6df0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:07.467092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f88828c3634d04977ccf21eb350e88', 'request_id': 'req-ccb68662-143f-454b-9c2a-562669bc6df0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:07.488580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f88828c3634d04977ccf21eb350e88', 'request_id': 'req-ccb68662-143f-454b-9c2a-562669bc6df0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:07.530918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] The token cache is being invalidate because project 96f88828c3634d04977ccf21eb350e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:07.532532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:07.536309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccb68662-143f-454b-9c2a-562669bc6df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:07.538716 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2624/5249] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:06 2022] DELETE /identity/v3/projects/96f88828c3634d04977ccf21eb350e88 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:07.544463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7380b-6564-4834-9adc-b287b8bff63d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.545650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7380b-6564-4834-9adc-b287b8bff63d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.546678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7380b-6564-4834-9adc-b287b8bff63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.563525 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2626/5250] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:07.563525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f959d0-cf62-49dc-acef-9e2d342f3e56 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.563525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f959d0-cf62-49dc-acef-9e2d342f3e56 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.563525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f959d0-cf62-49dc-acef-9e2d342f3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.617271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-59f7380b-6564-4834-9adc-b287b8bff63d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:07.618602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-59f7380b-6564-4834-9adc-b287b8bff63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:07.628988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-04f959d0-cf62-49dc-acef-9e2d342f3e56 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:07.634073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-04f959d0-cf62-49dc-acef-9e2d342f3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:07.648016 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2625/5251] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:07.651553 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2627/5252] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:07.652069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f959d0-cf62-49dc-acef-9e2d342f3e56 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:07.667375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7380b-6564-4834-9adc-b287b8bff63d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:07.696247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f959d0-cf62-49dc-acef-9e2d342f3e56 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:07.698821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7380b-6564-4834-9adc-b287b8bff63d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:07.706050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:07.706688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.706969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.707232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.707887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:07.708188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:07.709861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:07.710044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831a62f-df8c-460c-9511-ef58ad687e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:07.710828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831a62f-df8c-460c-9511-ef58ad687e3d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.711319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831a62f-df8c-460c-9511-ef58ad687e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.712081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831a62f-df8c-460c-9511-ef58ad687e3d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.713022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831a62f-df8c-460c-9511-ef58ad687e3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:07.714419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831a62f-df8c-460c-9511-ef58ad687e3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:07.717167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831a62f-df8c-460c-9511-ef58ad687e3d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:07.736317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2626/5253] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:07.738284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127774-bc48-4458-b473-bff6340acf1c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.738648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127774-bc48-4458-b473-bff6340acf1c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.738907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127774-bc48-4458-b473-bff6340acf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.748514 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2628/5254] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:07.748514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:07.772406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d641105-2fd2-4a4e-809c-8b8507bec4d5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:07.781433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5b127774-bc48-4458-b473-bff6340acf1c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:07.783207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5b127774-bc48-4458-b473-bff6340acf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:07.784725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:07.785601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.786065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.786504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.787612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:07.790957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:07.806904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2627/5255] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:07.808196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b127774-bc48-4458-b473-bff6340acf1c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:07.843969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b127774-bc48-4458-b473-bff6340acf1c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:07.875043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:07.875685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:07.877912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:07.878206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:07.885274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:07.887088 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2629/5256] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:07.887655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:07.893505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:07.904768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:07.953352 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2628/5257] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:07.954257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.036239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:08.048628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40414e41-5aaf-4bad-93b1-0d686e9a2055 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.133858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225bddb-1ef8-4c4f-bd0d-9e8a54e4dc96 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.144410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.145053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.148587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.148890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.149662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.151700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.206308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2629/5258] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:07 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:08.210263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.230212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.230956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.233931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.233931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.233931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:08.233931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.239597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.245894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ef996-8065-4bf6-90c4-fece91060123 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.262928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.263848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.264299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.264710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.265744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.268039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.279621 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2630/5259] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:08.280782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.309579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768820-baea-4e04-8ab4-3458c544fcfc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.343422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.344313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.344623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.344892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.346971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.351578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.355720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2630/5260] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:08 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:08.356695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.367627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfcce6-f878-4657-a1b4-670ec171e79b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.373122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.373813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.374091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.374727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.375382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.377576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.381641 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2631/5261] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:08.391415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.436919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce774-3754-4f1f-b447-f329f1e40377 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.439832 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2631/5262] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:08 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:08.449585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.600174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.601046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.601447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.601832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.602595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.607347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.613622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee60c4-7b03-44d1-a13a-66efe4a240b3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.624815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.626217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.627216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.627705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.628896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.630356 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2632/5263] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:08.631735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.631989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.647833 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2632/5264] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:08.649245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.650866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289435a2-93e2-4cae-a679-4b401084c509 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.672673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.673394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.673721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.673994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.674689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.682641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.685167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1e4f8-ef06-403c-ab1e-0e15f5990588 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.702379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.703970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.704492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.705687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.706752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.709290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.723434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2633/5265] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:08 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:08.729441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.730230 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2633/5266] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:08.733772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.745420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab002-a38d-4b72-98b6-2f8dc8d4e228 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.759503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53edfc-0a55-459b-adba-9f4254bb3e5d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.764616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.765408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.765832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.766244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] PATH_INFO: `/v3/projects/50b258589aee45009ce8526b94fc58fe/users/f346a34ac86145469065f96f6313ee19/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.777210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19905180-3959-46f7-8e42-72bc11c5979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.780864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19905180-3959-46f7-8e42-72bc11c5979b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.781167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19905180-3959-46f7-8e42-72bc11c5979b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.783529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19905180-3959-46f7-8e42-72bc11c5979b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.785547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b258589aee45009ce8526b94fc58fe, user_id=f346a34ac86145469065f96f6313ee19, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.786275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19905180-3959-46f7-8e42-72bc11c5979b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.787836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19905180-3959-46f7-8e42-72bc11c5979b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.789576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:08.829964 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2634/5267] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:08 2022] PUT /identity/v3/projects/50b258589aee45009ce8526b94fc58fe/users/f346a34ac86145469065f96f6313ee19/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:08.833589 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2634/5268] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:08.835045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:08.837441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b40bb0-3623-4263-827e-d5348abaa95e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.837690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b40bb0-3623-4263-827e-d5348abaa95e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.837965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b40bb0-3623-4263-827e-d5348abaa95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.924308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:08.940782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:08.953653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d9b40bb0-3623-4263-827e-d5348abaa95e None None] MFA Rules not processed for user `f346a34ac86145469065f96f6313ee19`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:08.954380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0210a-5589-4d99-8ebf-b35f3c89c090 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:08.955555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d9b40bb0-3623-4263-827e-d5348abaa95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:08.961453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:08.962784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:08.963022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:08.963391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] PATH_INFO: `/v3/projects/fb5d96e0cc3e44cf87621d02139e5d08/users/b0a6fe32bc05417785f59880fe99b1aa/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:08.986091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5d96e0cc3e44cf87621d02139e5d08, user_id=b0a6fe32bc05417785f59880fe99b1aa, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:08.989921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:09.018134 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2635/5269] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:08 2022] PUT /identity/v3/projects/fb5d96e0cc3e44cf87621d02139e5d08/users/b0a6fe32bc05417785f59880fe99b1aa/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:09.019296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:09.082492 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2635/5270] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:09.084748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f98dde-1a26-4716-841b-03f6b6b4aa99 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.085198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f98dde-1a26-4716-841b-03f6b6b4aa99 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.085709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f98dde-1a26-4716-841b-03f6b6b4aa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.132385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f6f98dde-1a26-4716-841b-03f6b6b4aa99 None None] MFA Rules not processed for user `b0a6fe32bc05417785f59880fe99b1aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:09.133805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f6f98dde-1a26-4716-841b-03f6b6b4aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:09.200733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10210b5-cf81-4802-bee8-fb422517f1d6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:09.216084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:09.216767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.217048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.217321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.218188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:09.220379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:09.237300 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2636/5271] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:09.238247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:09.252275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee0ca-c1a5-4198-ae86-008ffb9fb173 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:09.257232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:09.258172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.258457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.258733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.259416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:09.261648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:09.321881 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2636/5272] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:09.326241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f98dde-1a26-4716-841b-03f6b6b4aa99 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:09.332997 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2637/5273] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:09 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:09.334731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:09.342337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:09.385640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f98dde-1a26-4716-841b-03f6b6b4aa99 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:09.507279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:09.508161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.508574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.509112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] PATH_INFO: `/v3/projects/cfc26054d4a34fb2b2aef34fb4891b4c/users/caa2c2f4848e497f80d4a439c785c2b0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.540180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc26054d4a34fb2b2aef34fb4891b4c, user_id=caa2c2f4848e497f80d4a439c785c2b0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:09.543691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:09.578270 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2637/5274] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:09 2022] PUT /identity/v3/projects/cfc26054d4a34fb2b2aef34fb4891b4c/users/caa2c2f4848e497f80d4a439c785c2b0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:09.579808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:09.593746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bbe8e-c1ab-4e15-adcc-5dab72b929b3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:09.607578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:09.694520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:09.717755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3865f4-cba2-45ff-a119-9111ece70cd3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:09.724472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cb668-ddb1-4362-b0f2-cb2f385ef4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:09.725851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cb668-ddb1-4362-b0f2-cb2f385ef4fb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.726441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cb668-ddb1-4362-b0f2-cb2f385ef4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.726924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cb668-ddb1-4362-b0f2-cb2f385ef4fb admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.728007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cb668-ddb1-4362-b0f2-cb2f385ef4fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:09.730388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cb668-ddb1-4362-b0f2-cb2f385ef4fb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:09.740900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:09.741931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.742280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.742624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.786441 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2638/5275] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:09 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:09.790400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d4f6d-ed69-4990-871c-70ac040269da None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.790949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d4f6d-ed69-4990-871c-70ac040269da None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.791436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d4f6d-ed69-4990-871c-70ac040269da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.829040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:09.831126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:09.833769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fa2d4f6d-ed69-4990-871c-70ac040269da None None] MFA Rules not processed for user `caa2c2f4848e497f80d4a439c785c2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:09.835171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fa2d4f6d-ed69-4990-871c-70ac040269da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:09.842613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2638/5276] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:09.843799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:09.857881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb371bc5-8dfd-46c2-a7c6-eb0cf2159023 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:09.871417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:09.873088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.873088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.876414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.876895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:09.882255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:09.908286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2639/5277] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:09.908869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:09.936064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f58a6-04a0-435a-86fb-d7442cbd1616 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:09.941815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:09.942663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:09.943083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:09.943576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] PATH_INFO: `/v3/projects/7de24c8f75c94ca0b7b615a72be8da71/users/6e6d577e7c284fb887592f3b1a1d85eb/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:09.962009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de24c8f75c94ca0b7b615a72be8da71, user_id=6e6d577e7c284fb887592f3b1a1d85eb, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:09.981540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:10.018236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2639/5278] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:10.018236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d4f6d-ed69-4990-871c-70ac040269da None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:10.031075 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2640/5279] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:09 2022] PUT /identity/v3/projects/7de24c8f75c94ca0b7b615a72be8da71/users/6e6d577e7c284fb887592f3b1a1d85eb/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:10.036185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:10.055105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d4f6d-ed69-4990-871c-70ac040269da None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:10.221040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:10.222046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:10.222982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:10.224091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:10.224091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:10.226530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:10.251085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ca092-ba79-49b2-a38e-c2d743867694 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:10.275326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:10.276357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:10.277902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:10.278100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:10.280056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:10.281855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:10.320455 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2641/5280] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:10.327376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:10.335645 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2640/5281] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:10 2022] POST /identity/v3/projects => generated 387 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:10.337864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:10.373151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e89641-97a9-49d8-98b2-92aff10a4db5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:10.429418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:10.503251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:10.538552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8237a-2efa-4210-b198-c3f61ef3b083 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:10.547293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:10.548080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:10.548475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:10.548835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] PATH_INFO: `/v3/projects/7de24c8f75c94ca0b7b615a72be8da71/users/6e6d577e7c284fb887592f3b1a1d85eb/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:10.552261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7de24c8f75c94ca0b7b615a72be8da71, user_id=6e6d577e7c284fb887592f3b1a1d85eb, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:10.565968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:10.579191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:10.580043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:10.581573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:10.581967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] PATH_INFO: `/v3/users/b0a6fe32bc05417785f59880fe99b1aa` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:10.584490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a6fe32bc05417785f59880fe99b1aa)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:10.589448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:10.603785 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2641/5282] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:10 2022] PUT /identity/v3/projects/7de24c8f75c94ca0b7b615a72be8da71/users/6e6d577e7c284fb887592f3b1a1d85eb/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:10.604776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:10.714457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a6fe32bc05417785f59880fe99b1aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:10.716123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a6fe32bc05417785f59880fe99b1aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:10.740926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a6fe32bc05417785f59880fe99b1aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:10.745380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a6fe32bc05417785f59880fe99b1aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:10.764370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a9bab3b6-a273-449f-aad8-55c0e7bc0d66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a6fe32bc05417785f59880fe99b1aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:10.796450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2642/5283] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:10 2022] DELETE /identity/v3/users/b0a6fe32bc05417785f59880fe99b1aa => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:10.796450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:10.796450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:10.796450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:10.806158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1b2f6-5ca8-4207-8723-dcb6a6692da0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:10.882674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] MFA Rules not processed for user `6e6d577e7c284fb887592f3b1a1d85eb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:10.884769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:10.966621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da2514-8e99-4118-bc88-8c2f1536b3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:10.967498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da2514-8e99-4118-bc88-8c2f1536b3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:10.967816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da2514-8e99-4118-bc88-8c2f1536b3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:10.968329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da2514-8e99-4118-bc88-8c2f1536b3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:10.969030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da2514-8e99-4118-bc88-8c2f1536b3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:10.972706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da2514-8e99-4118-bc88-8c2f1536b3c0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.023658 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2642/5284] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:10 2022] POST /identity/v3/users => generated 294 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:11.023658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa420a-5777-4f55-9e31-81e5728f9b9d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.023658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa420a-5777-4f55-9e31-81e5728f9b9d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.023658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa420a-5777-4f55-9e31-81e5728f9b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.031416 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2643/5285] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:11.032694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.035581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.047160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-52aa420a-5777-4f55-9e31-81e5728f9b9d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:11.048538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52aa420a-5777-4f55-9e31-81e5728f9b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.051964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb09ff8-11df-4c75-aa2a-f1f2336a2846 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.057169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.057372 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2643/5286] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:11.058134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa420a-5777-4f55-9e31-81e5728f9b9d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.058338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.058811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.059085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.059742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.061370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.073096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa420a-5777-4f55-9e31-81e5728f9b9d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.076938 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2644/5287] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:11.077911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.079266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.079675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.080264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.080717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.081235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.082828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.084362 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:11.084362 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:11.084362 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:11.085913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.115720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2644/5288] 10.222.0.26 () {60 vars in 1185 bytes} [Sat Sep 17 21:12:11 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:11.116885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.127964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:11.130456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463497-ea7c-41ee-a3a7-f0306312f3af tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.138910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.139739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.140385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.144963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.145730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.147779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.165253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71faf7-8eae-4d5e-b64d-1910f3ad643b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.170957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.171396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.171913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.172245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.177252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2645/5289] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:11.178229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.183413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.185537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.198732 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2645/5290] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:11.200005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf9f48-c074-40f9-9a1d-cb0d25209922 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.200356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.205195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.206019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.206297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.206564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/4df5aa0924af41769263f058e2cd1ffc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.207213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=4df5aa0924af41769263f058e2cd1ffc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.208531 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:11.208531 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:11.208531 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:11.209223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.214976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c724e-ac0e-40a4-81a7-021a411e4a89 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.216726 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2646/5291] 10.222.0.26 () {58 vars in 1263 bytes} [Sat Sep 17 21:12:11 2022] GET /identity/v3/OS-OAUTH1/consumers/4df5aa0924af41769263f058e2cd1ffc => generated 221 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:11.220583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.221621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e099-1cbe-4640-9704-814481ff6e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.223384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e099-1cbe-4640-9704-814481ff6e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.224111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e099-1cbe-4640-9704-814481ff6e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.224578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e099-1cbe-4640-9704-814481ff6e8f admin admin] PATH_INFO: `/v3/projects/b2cf5878e6d4483b9d978ff5cb045a24/users/2e1a3388bac14ababdbc8e5e048e3263/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.233109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2d29c-cab2-4723-8195-29e75bfc27d0 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.238579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.239297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.239743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.240424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/4df5aa0924af41769263f058e2cd1ffc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.242779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=4df5aa0924af41769263f058e2cd1ffc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.244117 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:11.244117 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:11.244117 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:11.245025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.245293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invalidating token cache because consumer 4df5aa0924af41769263f058e2cd1ffc has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:11.245596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.246636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0e099-1cbe-4640-9704-814481ff6e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2cf5878e6d4483b9d978ff5cb045a24, user_id=2e1a3388bac14ababdbc8e5e048e3263, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.247799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.249838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0e099-1cbe-4640-9704-814481ff6e8f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.254511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:11.289335 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2646/5292] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:11 2022] PUT /identity/v3/projects/b2cf5878e6d4483b9d978ff5cb045a24/users/2e1a3388bac14ababdbc8e5e048e3263/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:11.291080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '4df5aa0924af41769263f058e2cd1ffc', 'request_id': 'req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.316890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.317181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.317519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.323387 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2647/5293] 10.222.0.26 () {58 vars in 1266 bytes} [Sat Sep 17 21:12:11 2022] DELETE /identity/v3/OS-OAUTH1/consumers/4df5aa0924af41769263f058e2cd1ffc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:11.324762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.327723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.352176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b44f9e-8bef-4b8d-ba89-c4c9c843f5db tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.409600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] MFA Rules not processed for user `2e1a3388bac14ababdbc8e5e048e3263`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:11.410858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.469309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.488168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.489098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.489800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.491063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.497193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.526842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2648/5294] 10.222.0.26 () {60 vars in 1185 bytes} [Sat Sep 17 21:12:11 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:11.530802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.547085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb00d44-1230-4a1b-96fa-54eb413dbae9 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.553245 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2647/5295] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:11.554516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.556318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.559169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.559835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.560273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.560564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.561173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=1ad478625a0c48539c5643eefe58785b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.563016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.569446 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2649/5296] 10.222.0.26 () {58 vars in 1263 bytes} [Sat Sep 17 21:12:11 2022] GET /identity/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b => generated 220 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:11.573847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.585430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a05fe-f755-43e9-984d-253e3f9d03ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.590452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.591161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.591452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.591733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.592331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=1ad478625a0c48539c5643eefe58785b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.594215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.594491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invalidating token cache because consumer 1ad478625a0c48539c5643eefe58785b has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:11.595279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.595943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.619429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '1ad478625a0c48539c5643eefe58785b', 'request_id': 'req-23916511-f1ae-4e51-8c49-42a1087ae431'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.655468 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2650/5297] 10.222.0.26 () {58 vars in 1266 bytes} [Sat Sep 17 21:12:11 2022] DELETE /identity/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:11.657374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.660185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.681434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916511-f1ae-4e51-8c49-42a1087ae431 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.695510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.696449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.696449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.696978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.697666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=1ad478625a0c48539c5643eefe58785b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.700925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Sep 17 21:12:11.730748 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 17 21:12:11.732061 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:11.736863 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2651/5298] 10.222.0.26 () {58 vars in 1263 bytes} [Sat Sep 17 21:12:11 2022] GET /identity/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b => generated 74 bytes in 81 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 21:12:11.738231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.767162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2b18e-1a02-4a9a-a0fd-89c87adec1a3 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.795910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.807190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.809335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.809335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.811764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=1ad478625a0c48539c5643eefe58785b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.816116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.816532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invalidating token cache because consumer 1ad478625a0c48539c5643eefe58785b has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:11.820685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.821988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Sep 17 21:12:11.856282 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 17 21:12:11.857590 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:11.860395 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2652/5299] 10.222.0.26 () {58 vars in 1266 bytes} [Sat Sep 17 21:12:11 2022] DELETE /identity/v3/OS-OAUTH1/consumers/1ad478625a0c48539c5643eefe58785b => generated 74 bytes in 123 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 21:12:11.861639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:11.863268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:11.928083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b907d55-adf6-401f-8817-671d242267f6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.935826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.936924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.937303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.938476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.939458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:11.941858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:11.988702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538b788-3585-4352-a1d7-85a39a3e4879 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:11.994689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:11.995433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:11.995805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:11.996091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] PATH_INFO: `/v3/projects/fb5d96e0cc3e44cf87621d02139e5d08` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:11.998731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5d96e0cc3e44cf87621d02139e5d08)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.003010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.016548 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2648/5300] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:11 2022] POST /identity/v3/projects => generated 389 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:12.017395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.019423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:12.039285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:12.060865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:12.089325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5d96e0cc3e44cf87621d02139e5d08', 'request_id': 'req-353a7089-c8cf-4b10-8481-cd2625d4632c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.104064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5d96e0cc3e44cf87621d02139e5d08', 'request_id': 'req-353a7089-c8cf-4b10-8481-cd2625d4632c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.113303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5d96e0cc3e44cf87621d02139e5d08', 'request_id': 'req-353a7089-c8cf-4b10-8481-cd2625d4632c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.125691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a47e70-db59-48b5-9b84-53076630e1c2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.128701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] The token cache is being invalidate because project fb5d96e0cc3e44cf87621d02139e5d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:12.128701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.129239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.131948 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2653/5301] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:11 2022] DELETE /identity/v3/projects/fb5d96e0cc3e44cf87621d02139e5d08 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:12.138119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.139669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:12.205009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ca54c-1408-4c6e-a3b9-e5c9fcdb2735 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.205800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca54c-1408-4c6e-a3b9-e5c9fcdb2735 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.206293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca54c-1408-4c6e-a3b9-e5c9fcdb2735 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.206654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca54c-1408-4c6e-a3b9-e5c9fcdb2735 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.207417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ca54c-1408-4c6e-a3b9-e5c9fcdb2735 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.208617 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-c01ca54c-1408-4c6e-a3b9-e5c9fcdb2735 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:12.208617 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:12.208617 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:12.209843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ca54c-1408-4c6e-a3b9-e5c9fcdb2735 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.225088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a7089-c8cf-4b10-8481-cd2625d4632c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.231128 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2649/5302] 10.222.0.26 () {60 vars in 1185 bytes} [Sat Sep 17 21:12:12 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:12.232361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e98d0c-7e89-4c77-9998-1c40b3eaa8bc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.232640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e98d0c-7e89-4c77-9998-1c40b3eaa8bc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.233198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e98d0c-7e89-4c77-9998-1c40b3eaa8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.234680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.236120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.236523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.236855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.237626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.239854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.256481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-97e98d0c-7e89-4c77-9998-1c40b3eaa8bc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:12.257934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97e98d0c-7e89-4c77-9998-1c40b3eaa8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:12.267737 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2654/5303] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:12 2022] POST /identity/v3/users => generated 301 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:12.269592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.278637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2650/5304] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:12.278637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e98d0c-7e89-4c77-9998-1c40b3eaa8bc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.280064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:12.285542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e98d0c-7e89-4c77-9998-1c40b3eaa8bc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.292532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581c67-1240-4c8f-ba04-7bafd7cef8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.293595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581c67-1240-4c8f-ba04-7bafd7cef8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.293826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581c67-1240-4c8f-ba04-7bafd7cef8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.294187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581c67-1240-4c8f-ba04-7bafd7cef8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.295190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581c67-1240-4c8f-ba04-7bafd7cef8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.297103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581c67-1240-4c8f-ba04-7bafd7cef8d4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.302575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6a7d-1394-4da5-b800-dffbb547900e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.310175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.311652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.313358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.314057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.315034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.317359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.328490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2651/5305] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:12.330161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558a5b-7cef-4cc5-80dc-a0e59f29f6b7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.330456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558a5b-7cef-4cc5-80dc-a0e59f29f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.330728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558a5b-7cef-4cc5-80dc-a0e59f29f6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.347542 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2655/5306] 10.222.0.26 () {60 vars in 1185 bytes} [Sat Sep 17 21:12:12 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:12.348508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.388161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26871ef5-e21b-452a-8e47-352b58efc63e tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.401225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.402116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.402814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.403185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] PATH_INFO: `/v3/projects/fb24a60f519e4ec38579b42bd14d81fd/users/e22b2eb3624d4950b0defc614e5a4476/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.422504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c2558a5b-7cef-4cc5-80dc-a0e59f29f6b7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:12.425508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c2558a5b-7cef-4cc5-80dc-a0e59f29f6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:12.449805 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2652/5307] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:12.450810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558a5b-7cef-4cc5-80dc-a0e59f29f6b7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.468085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb24a60f519e4ec38579b42bd14d81fd, user_id=e22b2eb3624d4950b0defc614e5a4476, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.470867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.477613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558a5b-7cef-4cc5-80dc-a0e59f29f6b7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.492568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.493226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.493545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.493833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.494374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.495395 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:12.495395 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:12.495395 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:12.496340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.500210 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2656/5308] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:12 2022] PUT /identity/v3/projects/fb24a60f519e4ec38579b42bd14d81fd/users/e22b2eb3624d4950b0defc614e5a4476/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:12.500210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.512658 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2653/5309] 10.222.0.26 () {58 vars in 1164 bytes} [Sat Sep 17 21:12:12 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 537 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:12.513661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.542562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eefda-cbce-4249-9172-affeb2eb0fd8 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.625014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1d238-8fd3-4689-bd74-0e6f5c1cfb5c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.625780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.626449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.626731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.627116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.628072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.629796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.633700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.635076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.635530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.635991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.637120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:12.637674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.639427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.643413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:12.645424 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2654/5310] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:12.646474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.660662 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2657/5311] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:12.661698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.697945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f52b3c-f579-4811-89ed-1977b21dfd1c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.704795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.705984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.706359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.708247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] PATH_INFO: `/v3/projects/fb24a60f519e4ec38579b42bd14d81fd/users/e22b2eb3624d4950b0defc614e5a4476/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.711704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb24a60f519e4ec38579b42bd14d81fd, user_id=e22b2eb3624d4950b0defc614e5a4476, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.714689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.732716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:12.739536 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2658/5312] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:12 2022] PUT /identity/v3/projects/fb24a60f519e4ec38579b42bd14d81fd/users/e22b2eb3624d4950b0defc614e5a4476/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:12.740711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:12.804092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b34436-9f72-46b0-8a1d-563d3451149a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.872641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc72b0-55ba-45ca-aa51-352c0c97cadf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:12.882395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0680-e6e3-403c-bf01-b3e148ef45d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.883026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0680-e6e3-403c-bf01-b3e148ef45d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.883312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0680-e6e3-403c-bf01-b3e148ef45d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.883836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0680-e6e3-403c-bf01-b3e148ef45d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.884611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0680-e6e3-403c-bf01-b3e148ef45d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.888763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0680-e6e3-403c-bf01-b3e148ef45d0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.904085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:12.905500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.906081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.906689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/948a12a18edd45deaf093e520f12422c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.907557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=948a12a18edd45deaf093e520f12422c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:12.909392 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:12.909392 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:12.909392 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:12.910977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:12.911466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invalidating token cache because consumer 948a12a18edd45deaf093e520f12422c has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:12.912166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.913827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.944310 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2659/5313] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:12 2022] POST /identity/v3/projects => generated 395 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:12.944310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:12.944310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:12.945541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:12.978261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '948a12a18edd45deaf093e520f12422c', 'request_id': 'req-366054d1-b7cb-457e-b387-846f5b8e8ffa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:12.992911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] MFA Rules not processed for user `e22b2eb3624d4950b0defc614e5a4476`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:12.994546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.010150 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2655/5314] 10.222.0.26 () {58 vars in 1266 bytes} [Sat Sep 17 21:12:12 2022] DELETE /identity/v3/OS-OAUTH1/consumers/948a12a18edd45deaf093e520f12422c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:13.011456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.015251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.051854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366054d1-b7cb-457e-b387-846f5b8e8ffa tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.116765 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2660/5315] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:13.117709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.119081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.239799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.240887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.241959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.242688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.243603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.257863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:13.258034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.281750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de7988-bb99-46f7-9014-f5c2aa41cd66 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.284492 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2656/5316] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:13 2022] POST /identity/v3/users => generated 298 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:13.285588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.295083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.296039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.296468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.296904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/1d61ecd85ce04a869a834dd5cdaf34c2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.297680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=1d61ecd85ce04a869a834dd5cdaf34c2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.299662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.300103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invalidating token cache because consumer 1d61ecd85ce04a869a834dd5cdaf34c2 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:13.300392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.300722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:13.301586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:13.321120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '1d61ecd85ce04a869a834dd5cdaf34c2', 'request_id': 'req-1840adc7-951e-4822-acd6-f0aa897f3b94'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:13.347866 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2661/5317] 10.222.0.26 () {58 vars in 1266 bytes} [Sat Sep 17 21:12:13 2022] DELETE /identity/v3/OS-OAUTH1/consumers/1d61ecd85ce04a869a834dd5cdaf34c2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:13.348748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.350139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.375095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:13.380140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840adc7-951e-4822-acd6-f0aa897f3b94 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.386851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.387812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.390245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.390520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.391184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.394691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.413630 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2662/5318] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:13.414438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.417619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.434508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0447e9b-4f87-421d-8560-1987cc74ec97 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.452328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55418e5-a63a-4dda-aa06-f4045a54329e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.460019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.462608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.462919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.463200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.465401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.466073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.469530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.469878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.470735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.471141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.476565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.501692 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2663/5319] 10.222.0.26 () {60 vars in 1185 bytes} [Sat Sep 17 21:12:13 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:13.502687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.520108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d51d0e-9775-48a5-a968-4c59c2360970 tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.529722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.530353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.531474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.531763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.532477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.534171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.550862 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2664/5320] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:13.550862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.564115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.566741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.567786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b83bb-74f7-4f56-9db6-33e162374e68 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.579314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.579578 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2657/5321] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:13.580406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.591991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.592425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.592717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/693167fc62bd412a986274253d5a50b3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.593565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=693167fc62bd412a986274253d5a50b3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.594167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412527a2-9291-4b55-951e-f0e0e880e4fc service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.594632 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:13.594632 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:13.594632 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:13.599995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e764e1-639a-4683-b42c-15896df5dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.600993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e764e1-639a-4683-b42c-15896df5dd6c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.601614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e764e1-639a-4683-b42c-15896df5dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.601987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e764e1-639a-4683-b42c-15896df5dd6c admin admin] PATH_INFO: `/v3/projects/b5ac35464b43400ea1f7305e7c48cb84/users/b2630775f52d44e1a031348d81e05d81/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.603027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.630834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e764e1-639a-4683-b42c-15896df5dd6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ac35464b43400ea1f7305e7c48cb84, user_id=b2630775f52d44e1a031348d81e05d81, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.634655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e764e1-639a-4683-b42c-15896df5dd6c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.640322 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2665/5322] 10.222.0.26 () {60 vars in 1285 bytes} [Sat Sep 17 21:12:13 2022] PATCH /identity/v3/OS-OAUTH1/consumers/693167fc62bd412a986274253d5a50b3 => generated 221 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:13.643213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.661291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2658/5323] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:13 2022] PUT /identity/v3/projects/b5ac35464b43400ea1f7305e7c48cb84/users/b2630775f52d44e1a031348d81e05d81/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:13.672634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2bcde-154e-4943-bb6c-3d7933e5612f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.673675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260b58d-47db-472b-9bfb-f72cecff815b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.674667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260b58d-47db-472b-9bfb-f72cecff815b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.677620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260b58d-47db-472b-9bfb-f72cecff815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.727919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2260b58d-47db-472b-9bfb-f72cecff815b None None] MFA Rules not processed for user `b2630775f52d44e1a031348d81e05d81`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:13.729188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2260b58d-47db-472b-9bfb-f72cecff815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.815624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.816484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.816628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.817109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/693167fc62bd412a986274253d5a50b3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.817227 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2659/5324] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:13.817580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=693167fc62bd412a986274253d5a50b3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.820931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.822835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260b58d-47db-472b-9bfb-f72cecff815b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.834836 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2666/5325] 10.222.0.26 () {58 vars in 1263 bytes} [Sat Sep 17 21:12:13 2022] GET /identity/v3/OS-OAUTH1/consumers/693167fc62bd412a986274253d5a50b3 => generated 221 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:13.839340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.861253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4dd1f-4abb-4478-901b-321c0f5ec9ea tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.876824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e6d577e7c284fb887592f3b1a1d85eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7de24c8f75c94ca0b7b615a72be8da71', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.877761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.878118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.878391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/693167fc62bd412a986274253d5a50b3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.878983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=693167fc62bd412a986274253d5a50b3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.880628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.880898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invalidating token cache because consumer 693167fc62bd412a986274253d5a50b3 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:13.881171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:13.881908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:13.909823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '693167fc62bd412a986274253d5a50b3', 'request_id': 'req-67d9460b-e021-4b04-aa1f-65e3d3f4263f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:13.915058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:13.940325 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2667/5326] 10.222.0.26 () {58 vars in 1266 bytes} [Sat Sep 17 21:12:13 2022] DELETE /identity/v3/OS-OAUTH1/consumers/693167fc62bd412a986274253d5a50b3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:13.945153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:13.947078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:13.959255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260b58d-47db-472b-9bfb-f72cecff815b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.970627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9460b-e021-4b04-aa1f-65e3d3f4263f tempest-OAUTHConsumersV3Test-49997969 tempest-OAUTHConsumersV3Test-49997969-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:13.975258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.976434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:13.977409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.977828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.978189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:13.978315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] PATH_INFO: `/v3/users/f346a34ac86145469065f96f6313ee19` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.978708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:13.979180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:13.980001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f346a34ac86145469065f96f6313ee19)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.980529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:13.982816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:13.984846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:14.047167 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2660/5327] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:13 2022] POST /identity/v3/projects => generated 395 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:14.049179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:14.051702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:14.080652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f346a34ac86145469065f96f6313ee19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:14.083082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:14.096051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f346a34ac86145469065f96f6313ee19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:14.113962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f346a34ac86145469065f96f6313ee19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:14.117312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f346a34ac86145469065f96f6313ee19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:14.117998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d552493d-f493-474c-aa69-3aac3004da63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f346a34ac86145469065f96f6313ee19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:14.145199 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2668/5328] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:13 2022] DELETE /identity/v3/users/f346a34ac86145469065f96f6313ee19 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:14.149627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.149963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.150159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.164757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f81996-667b-4b8a-b74a-37c25b58e3ba admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:14.194711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:14.196740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:14.303583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:14.304231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.304508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.304777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.308054 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2669/5329] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:12:14.308149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:14.309716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:14.354296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:14.356809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:14.371058 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2661/5330] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:14 2022] POST /identity/v3/users => generated 304 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:14.375944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:14.378030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:14.394936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c002e4-b608-419f-b767-89f1587b13f3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:14.401721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:14.402585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.402764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.403026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.403726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:14.405633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:14.422617 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2662/5331] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:14.424160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:14.456764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485dd9-a04d-4368-8bee-ce12fe92e27e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:14.465927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:14.468800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.469406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.469855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] PATH_INFO: `/v3/projects/bf63aa4c20a645e28d7eff4c2b1224f3/users/5a7e3b0e5300449da3104954fb9f56fb/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.497498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf63aa4c20a645e28d7eff4c2b1224f3, user_id=5a7e3b0e5300449da3104954fb9f56fb, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:14.504670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:14.508039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadef602-610e-4054-85cb-b898d16a4123 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:14.516591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:14.519971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.520272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.520714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.543182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:14.543584 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2663/5332] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:14 2022] PUT /identity/v3/projects/bf63aa4c20a645e28d7eff4c2b1224f3/users/5a7e3b0e5300449da3104954fb9f56fb/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:14.547345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:14.550727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:14.561430 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2670/5333] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:14.600496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:14.647995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa049a-1fa5-47d2-bd50-fd108555c613 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:14.659505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce205fb-1c26-4f86-b9fe-461129c13ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:14.661493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce205fb-1c26-4f86-b9fe-461129c13ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.661726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce205fb-1c26-4f86-b9fe-461129c13ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.662050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce205fb-1c26-4f86-b9fe-461129c13ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.664446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce205fb-1c26-4f86-b9fe-461129c13ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:14.666518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce205fb-1c26-4f86-b9fe-461129c13ce2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:14.690249 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2664/5334] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:14.690608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:14.712665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39807a0-9947-4d7a-a735-67eccaf21757 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:14.724342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:14.725494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.726511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.726671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] PATH_INFO: `/v3/projects/bf63aa4c20a645e28d7eff4c2b1224f3/users/5a7e3b0e5300449da3104954fb9f56fb/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.730613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf63aa4c20a645e28d7eff4c2b1224f3, user_id=5a7e3b0e5300449da3104954fb9f56fb, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:14.736982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:14.778581 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2671/5335] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:14 2022] PUT /identity/v3/projects/bf63aa4c20a645e28d7eff4c2b1224f3/users/5a7e3b0e5300449da3104954fb9f56fb/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:14.783200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999a0b-f86c-4e34-81d4-b305c1c5dbc7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:14.784264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999a0b-f86c-4e34-81d4-b305c1c5dbc7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:14.784725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999a0b-f86c-4e34-81d4-b305c1c5dbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:14.823331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-94999a0b-f86c-4e34-81d4-b305c1c5dbc7 None None] MFA Rules not processed for user `5a7e3b0e5300449da3104954fb9f56fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:14.825137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-94999a0b-f86c-4e34-81d4-b305c1c5dbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:14.981059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:14.981676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2665/5336] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:14.983905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:15.167288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e72adb-cdf8-4a6e-a588-5bd84543b273 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.182397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2630775f52d44e1a031348d81e05d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ac35464b43400ea1f7305e7c48cb84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.183028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.183315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.183664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.184312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Sep 17 21:12:15.198500 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.200731 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2672/5337] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:12:14 2022] POST /identity/v3/projects => generated 136 bytes in 220 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:15.206286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999a0b-f86c-4e34-81d4-b305c1c5dbc7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:15.220092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999a0b-f86c-4e34-81d4-b305c1c5dbc7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.233326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a7e3b0e5300449da3104954fb9f56fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.235048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.237254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.238053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.238434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.246442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Invalid input for field 'name': '' is too short Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: On instance['name']: Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: On instance['name']: Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: '' Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise error Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application '' Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.260371 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:12:15.263420 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application '' Sep 17 21:12:15.265375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.270224 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2666/5338] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:12:15 2022] POST /identity/v3/projects => generated 283 bytes in 65 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 17 21:12:15.270843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:15.279340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:15.290168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0fb8d-fdfd-46b0-8526-8f24bf35eb8c tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.311062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a7e3b0e5300449da3104954fb9f56fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.311932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.312381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.312896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.313824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.315934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: On instance['name']: Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: On instance['name']: Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise error Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:12:15.332806 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:15.334449 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application On instance['name']: Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 17 21:12:15.335553 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.338582 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2673/5339] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:12:15 2022] POST /identity/v3/projects => generated 412 bytes in 68 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 17 21:12:15.343908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:15.359706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cf16d-d557-4975-885f-7a212a701ec6 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.375956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a7e3b0e5300449da3104954fb9f56fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.376640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.376929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.377453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] PATH_INFO: `/v3/projects/6d30b464bbac463f99f5dab341a0acad` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.390406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6d30b464bbac463f99f5dab341a0acad)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.392345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Could not find project: 6d30b464bbac463f99f5dab341a0acad.: keystone.exception.ProjectNotFound: Could not find project: 6d30b464bbac463f99f5dab341a0acad. Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 17 21:12:15.421324 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6d30b464bbac463f99f5dab341a0acad. Sep 17 21:12:15.423685 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:15.428920 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2667/5340] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:15 2022] DELETE /identity/v3/projects/6d30b464bbac463f99f5dab341a0acad => generated 113 bytes in 85 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:15.432831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:15.455514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcff14c-a122-48db-afb1-9c5459c5a9d1 tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.461108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2630775f52d44e1a031348d81e05d81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ac35464b43400ea1f7305e7c48cb84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.461914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.462567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.462567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.463384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Sep 17 21:12:15.475912 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._list_projects() Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 17 21:12:15.477127 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.481157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2674/5341] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:12:15 2022] GET /identity/v3/projects => generated 135 bytes in 49 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:15.486099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:15.496983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ec739-4b3c-4755-8d33-efe03cdd712f tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.503938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a7e3b0e5300449da3104954fb9f56fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.504560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.504946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.505222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.505951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.507837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:15.570119 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2668/5342] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:15 2022] POST /identity/v3/projects => generated 358 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:15.573967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:15.600470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e57d9e-bb4d-4614-959a-1054d8ee2c93 tempest-ProjectsNegativeStaticTestJSON-947993448 tempest-ProjectsNegativeStaticTestJSON-947993448-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.713776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a7e3b0e5300449da3104954fb9f56fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.715154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.715495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.715725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.716859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.719919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:15.760082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.sql.core [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-2126762677' for key 'project.ixu_project_name_domain_id'") Sep 17 21:12:15.760082 ncdv-master-1 devstack@keystone.service[99166]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 21:12:15.760082 ncdv-master-1 devstack@keystone.service[99166]: [parameters: {'id': '4fde3de0778d4f46b19cfb35cffcf509', 'name': 'tempest-project-dup-2126762677', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 21:12:15.760082 ncdv-master-1 devstack@keystone.service[99166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-2126762677, project id 4fde3de0778d4f46b19cfb35cffcf509.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-2126762677, project id 4fde3de0778d4f46b19cfb35cffcf509. Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 21:12:15.785302 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-2126762677' for key 'project.ixu_project_name_domain_id'") Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 21:12:15.786441 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application yield resource Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise self.value Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 21:12:15.787720 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application session.commit() Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.flush() Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application compat.raise_( Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 21:12:15.789056 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 21:12:15.790397 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application conn.query(q) Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application result.read() Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 21:12:15.791606 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-2126762677' for key 'project.ixu_project_name_domain_id'") Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [parameters: {'id': '4fde3de0778d4f46b19cfb35cffcf509', 'name': 'tempest-project-dup-2126762677', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.792829 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-2126762677 at domain ID default. Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:15.795521 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-2126762677, project id 4fde3de0778d4f46b19cfb35cffcf509. Sep 17 21:12:15.800030 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:15.802764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:15.812059 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2675/5343] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:12:15 2022] POST /identity/v3/projects => generated 285 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 21:12:15.815147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922306cf-05a3-43fc-9294-843ed898b4cf tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:15.825892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a7e3b0e5300449da3104954fb9f56fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:15.828845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:15.833708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:15.834378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] PATH_INFO: `/v3/projects/b86d8d4da4c3480cbfaccb4255a8df5d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:15.836168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b86d8d4da4c3480cbfaccb4255a8df5d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:15.850723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:15.971045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86d8d4da4c3480cbfaccb4255a8df5d', 'request_id': 'req-70b346b9-bfc8-4447-b20b-552223d6ccdc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:15.981101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86d8d4da4c3480cbfaccb4255a8df5d', 'request_id': 'req-70b346b9-bfc8-4447-b20b-552223d6ccdc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:15.992327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86d8d4da4c3480cbfaccb4255a8df5d', 'request_id': 'req-70b346b9-bfc8-4447-b20b-552223d6ccdc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.002735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:16.010588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] The token cache is being invalidate because project b86d8d4da4c3480cbfaccb4255a8df5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:16.016524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.017737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.025426 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2669/5344] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:15 2022] DELETE /identity/v3/projects/b86d8d4da4c3480cbfaccb4255a8df5d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:16.031278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:16.033407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:16.164344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c178e4c-d386-4151-ba4d-c756b2ea52df tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:16.238609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:16.239030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:16.239305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:16.239577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] PATH_INFO: `/v3/users/b2630775f52d44e1a031348d81e05d81` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:16.241394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2630775f52d44e1a031348d81e05d81)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:16.243963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:16.312405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2630775f52d44e1a031348d81e05d81'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.314054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:16.331752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2630775f52d44e1a031348d81e05d81'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.343961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2630775f52d44e1a031348d81e05d81'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.346700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2630775f52d44e1a031348d81e05d81'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.351405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-66621027-51ec-46f3-a706-3d9194d619f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2630775f52d44e1a031348d81e05d81'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.378490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:16.380747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:16.410636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b346b9-bfc8-4447-b20b-552223d6ccdc tempest-ProjectsNegativeStaticTestJSON-973168690 tempest-ProjectsNegativeStaticTestJSON-973168690-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:16.414324 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2676/5345] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:16 2022] DELETE /identity/v3/users/b2630775f52d44e1a031348d81e05d81 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:16.418875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:16.419168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:16.419430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:16.426940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:16.428404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:16.428659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:16.428933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] PATH_INFO: `/v3/projects/50b258589aee45009ce8526b94fc58fe` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:16.431029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b258589aee45009ce8526b94fc58fe)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:16.435901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:16.458464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:16.459704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:16.551708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b258589aee45009ce8526b94fc58fe', 'request_id': 'req-cd16aa17-09e3-4919-9341-5b6a9b529653'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.561101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b258589aee45009ce8526b94fc58fe', 'request_id': 'req-cd16aa17-09e3-4919-9341-5b6a9b529653'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.574745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b258589aee45009ce8526b94fc58fe', 'request_id': 'req-cd16aa17-09e3-4919-9341-5b6a9b529653'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.611092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] The token cache is being invalidate because project 50b258589aee45009ce8526b94fc58fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:16.611403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.612306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.614812 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2670/5346] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:16 2022] DELETE /identity/v3/projects/50b258589aee45009ce8526b94fc58fe => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:16.619664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:16.622385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:16.630650 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2677/5347] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:16.653082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:16.654578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:16.764289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16aa17-09e3-4919-9341-5b6a9b529653 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:16.778040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:16.778809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:16.779092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:16.779364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] PATH_INFO: `/v3/users/6e6d577e7c284fb887592f3b1a1d85eb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:16.781645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6d577e7c284fb887592f3b1a1d85eb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:16.787210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:16.807788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147dd86-bf52-4554-a82f-697d7eb83f91 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:16.877761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183c577-90df-4210-83da-174661c9c1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:16.879134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183c577-90df-4210-83da-174661c9c1ab service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:16.879707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183c577-90df-4210-83da-174661c9c1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:16.880187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183c577-90df-4210-83da-174661c9c1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:16.896534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183c577-90df-4210-83da-174661c9c1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:16.898457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183c577-90df-4210-83da-174661c9c1ab service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:16.911898 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2678/5348] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:16.923519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6d577e7c284fb887592f3b1a1d85eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.924270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6d577e7c284fb887592f3b1a1d85eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.944202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6d577e7c284fb887592f3b1a1d85eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.944450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6d577e7c284fb887592f3b1a1d85eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.969190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6d577e7c284fb887592f3b1a1d85eb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:16.986638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:16.991456 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2671/5349] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:16 2022] DELETE /identity/v3/users/6e6d577e7c284fb887592f3b1a1d85eb => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:18.390806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:18.391377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:18.391377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:18.422045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] MFA Rules not processed for user `784233ee78764e7dbb774d3cb50165e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:18.423310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:18.436958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:18.554772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2679/5350] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:18.902375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:18.904436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.065734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:19.067163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.087409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e58e5-5210-4207-b79a-becb5d089424 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:19.111138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c637aaf-fa5c-49e9-a0d2-8f3539e444c6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:19.115573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:19.116247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.116992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.117265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] PATH_INFO: `/v3/projects/7de24c8f75c94ca0b7b615a72be8da71` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.119339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7de24c8f75c94ca0b7b615a72be8da71)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:19.124817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:19.125489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.126349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:19.128514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.129120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] PATH_INFO: `/v3/projects/b5ac35464b43400ea1f7305e7c48cb84` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.132864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ac35464b43400ea1f7305e7c48cb84)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:19.139439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:19.239452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7de24c8f75c94ca0b7b615a72be8da71', 'request_id': 'req-e6ae6430-bd4b-4cd7-a464-f38e648095b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.248315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7de24c8f75c94ca0b7b615a72be8da71', 'request_id': 'req-e6ae6430-bd4b-4cd7-a464-f38e648095b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.261291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7de24c8f75c94ca0b7b615a72be8da71', 'request_id': 'req-e6ae6430-bd4b-4cd7-a464-f38e648095b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.271044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ac35464b43400ea1f7305e7c48cb84', 'request_id': 'req-d7fa36f0-7670-4310-a215-8eb7deb1589d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.278514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] The token cache is being invalidate because project 7de24c8f75c94ca0b7b615a72be8da71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:19.279216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.281043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6ae6430-bd4b-4cd7-a464-f38e648095b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.287501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2672/5351] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:18 2022] DELETE /identity/v3/projects/7de24c8f75c94ca0b7b615a72be8da71 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:19.287928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ac35464b43400ea1f7305e7c48cb84', 'request_id': 'req-d7fa36f0-7670-4310-a215-8eb7deb1589d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.293305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00468b4-ab4d-486d-8f4e-2f309e8d645b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.293665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00468b4-ab4d-486d-8f4e-2f309e8d645b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.293889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00468b4-ab4d-486d-8f4e-2f309e8d645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.302175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ac35464b43400ea1f7305e7c48cb84', 'request_id': 'req-d7fa36f0-7670-4310-a215-8eb7deb1589d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.331072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] The token cache is being invalidate because project b5ac35464b43400ea1f7305e7c48cb84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:19.331349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.332053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.339373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2680/5352] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:19 2022] DELETE /identity/v3/projects/b5ac35464b43400ea1f7305e7c48cb84 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:19.340445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:19.342016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.361492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d00468b4-ab4d-486d-8f4e-2f309e8d645b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:19.362629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d00468b4-ab4d-486d-8f4e-2f309e8d645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.486018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2673/5353] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:19.494437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.494797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.494980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.500610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa36f0-7670-4310-a215-8eb7deb1589d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:19.510893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:19.511642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.511968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.512242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] PATH_INFO: `/v3/users/5a7e3b0e5300449da3104954fb9f56fb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.516471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7e3b0e5300449da3104954fb9f56fb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:19.524542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:19.538650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:19.540028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.552393 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2674/5354] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:19.553378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:19.555094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.598870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:19.603124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:19.616881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7e3b0e5300449da3104954fb9f56fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.648435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623650e-d6a7-43bf-bd74-0a5111a0ed64 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:19.661035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec126778-aea1-46ed-bfba-36e355d5a5f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:19.661718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec126778-aea1-46ed-bfba-36e355d5a5f2 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.661998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec126778-aea1-46ed-bfba-36e355d5a5f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.662264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec126778-aea1-46ed-bfba-36e355d5a5f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.676602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7e3b0e5300449da3104954fb9f56fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.698085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ec126778-aea1-46ed-bfba-36e355d5a5f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.705090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7e3b0e5300449da3104954fb9f56fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.706442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7e3b0e5300449da3104954fb9f56fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.708618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7e3b0e5300449da3104954fb9f56fb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:19.720965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec126778-aea1-46ed-bfba-36e355d5a5f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:19.723632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec126778-aea1-46ed-bfba-36e355d5a5f2 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:19.742150 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2681/5355] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:19 2022] DELETE /identity/v3/users/5a7e3b0e5300449da3104954fb9f56fb => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:19.746827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:19.755936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:19.761078 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2675/5356] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:19.900578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd51b2-446b-4737-93da-a3f8508407d4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:19.907578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:19.910790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.912705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.913080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.914232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:19.915218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:19.918786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:19.956803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61262c0-2d52-4544-afcf-2c239eb80410 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:19.957639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61262c0-2d52-4544-afcf-2c239eb80410 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:19.957977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61262c0-2d52-4544-afcf-2c239eb80410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:19.985236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2682/5357] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:19.986293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:20.009868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a4b56-0c2a-4604-8312-2a32820bc2a5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:20.025972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:20.026608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.026884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.027152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.027834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:20.029953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:20.036605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c61262c0-2d52-4544-afcf-2c239eb80410 None None] MFA Rules not processed for user `2d249eb9b0124c339d8e730c96649171`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:20.039243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c61262c0-2d52-4544-afcf-2c239eb80410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:20.099288 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2683/5358] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:19 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:20.100581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:20.225687 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2676/5359] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:20.238013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.238302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.240606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.261926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70ff1c-617e-46c7-8164-cde82b6ade19 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:20.285189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:20.287220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.287507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.287780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.290718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:20.293240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] MFA Rules not processed for user `2d249eb9b0124c339d8e730c96649171`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:20.294822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:20.295419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:20.350312 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2684/5360] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:20 2022] POST /identity/v3/users => generated 294 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:20.351314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:20.369027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474641da-87ed-49ae-bec9-330a31d5b71c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:20.384532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:20.386037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.386391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.386832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.387567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:20.389519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:20.404166 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2685/5361] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:20.408939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:20.421084 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2677/5362] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:20.425544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3538d3-1a15-468b-97dd-8a27c6d0cd15 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:20.437639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:20.439103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:20.448768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:20.449764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.450102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.450355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.451069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:20.455259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:20.458133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:20.489914 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2686/5363] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:12:20.493507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:20.536215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06238473-7d4b-4ae5-aa85-eca87d464b9f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:20.550791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9e12e-37a1-44bf-af3a-1079eab128ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:20.551972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e12e-37a1-44bf-af3a-1079eab128ba admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.552437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e12e-37a1-44bf-af3a-1079eab128ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.552924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e12e-37a1-44bf-af3a-1079eab128ba admin admin] PATH_INFO: `/v3/projects/2c1941b035064b0bb1c0be4f1870d01c/users/5e23682b95ea4ccf8ab384f182651d19/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.561911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:20.594829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9e12e-37a1-44bf-af3a-1079eab128ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1941b035064b0bb1c0be4f1870d01c, user_id=5e23682b95ea4ccf8ab384f182651d19, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:20.598207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9e12e-37a1-44bf-af3a-1079eab128ba admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:20.630674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2687/5364] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:20 2022] PUT /identity/v3/projects/2c1941b035064b0bb1c0be4f1870d01c/users/5e23682b95ea4ccf8ab384f182651d19/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:20.641190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.641856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.642355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.650906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf32cf8-18f6-44b6-8a14-365b9ad5fd9b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:20.757784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] MFA Rules not processed for user `5e23682b95ea4ccf8ab384f182651d19`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:20.760970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:20.877773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:20.878472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:20.880520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:20.880821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:20.941288 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2688/5365] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:12:20.945734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:20.950690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:21.063308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:21.070583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:21.091611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498857a-2f7e-4f62-a166-94228a2692ee None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:21.095206 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2678/5366] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:21.096338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:21.119554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:21.120272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.120613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.121047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.134512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:21.198214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba88f58-6431-4034-b487-da97ab6c907b service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:21.205937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:21.206766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.207048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.207322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.208237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:21.211458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:21.262052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2679/5367] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:21 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:21.263164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:21.266117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:21.308570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:21.310697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:21.314294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:21.325196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2689/5368] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:21.326262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:21.345308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c672e85-993a-4858-ab3e-cded047985e3 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:21.370169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:21.418845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdb57e-d85d-472a-b6d5-6a457c01622f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:21.431402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:21.432060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.432336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.432609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] PATH_INFO: `/v3/projects/bf63aa4c20a645e28d7eff4c2b1224f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.440106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf63aa4c20a645e28d7eff4c2b1224f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:21.443575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:21.485158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:21.485863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.486151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.486419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.487126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:21.489225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:21.533829 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2690/5369] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:21 2022] POST /identity/v3/users => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:21.537135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:21.545886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'request_id': 'req-ffdcb37d-267a-4e14-840d-d99b1312db44'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:21.568281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'request_id': 'req-ffdcb37d-267a-4e14-840d-d99b1312db44'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:21.582902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed6474-7139-49bb-b3ab-2878eebd6f4a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:21.586364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf63aa4c20a645e28d7eff4c2b1224f3', 'request_id': 'req-ffdcb37d-267a-4e14-840d-d99b1312db44'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:21.605220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] The token cache is being invalidate because project bf63aa4c20a645e28d7eff4c2b1224f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:21.605220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:21.607941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ffdcb37d-267a-4e14-840d-d99b1312db44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:21.616711 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2680/5370] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:21 2022] DELETE /identity/v3/projects/bf63aa4c20a645e28d7eff4c2b1224f3 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:21.618807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd82f-a905-4974-9842-d67e5fbf62a3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.619096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd82f-a905-4974-9842-d67e5fbf62a3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.619370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd82f-a905-4974-9842-d67e5fbf62a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.664133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a04bd82f-a905-4974-9842-d67e5fbf62a3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:21.665358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a04bd82f-a905-4974-9842-d67e5fbf62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:21.757085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:21.757801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.758083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.758364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.764934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:21.766716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:21.824449 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2691/5371] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:21.826654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:21.834279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:21.835121 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2681/5372] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:12:21.842920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f238149-4ba4-4fa4-82f4-f6cb1828ebd1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.843373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f238149-4ba4-4fa4-82f4-f6cb1828ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.843796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f238149-4ba4-4fa4-82f4-f6cb1828ebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.872540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02b3cc-d5f6-4bc8-a3e3-1b1ea87c04ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:21.878447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:21.879127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:21.879404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:21.879677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] PATH_INFO: `/v3/projects/03cbb3b2297d48b6aa22b240f3fd5057/users/e19057a0bb7b4fd7a89683bcaa243126/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:21.884949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7f238149-4ba4-4fa4-82f4-f6cb1828ebd1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:21.888298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f238149-4ba4-4fa4-82f4-f6cb1828ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:21.901682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cbb3b2297d48b6aa22b240f3fd5057, user_id=e19057a0bb7b4fd7a89683bcaa243126, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:21.905002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:21.909415 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2682/5373] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:21.913921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f238149-4ba4-4fa4-82f4-f6cb1828ebd1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:21.933595 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2692/5374] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:21 2022] PUT /identity/v3/projects/03cbb3b2297d48b6aa22b240f3fd5057/users/e19057a0bb7b4fd7a89683bcaa243126/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:21.938644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:21.941144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f238149-4ba4-4fa4-82f4-f6cb1828ebd1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.028478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:22.047184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.047871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.048301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.048599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.049334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:22.050007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.051810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.070010 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2683/5375] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:22.077023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.098463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b23c3c-abac-4de4-a773-ba6a4e9ad664 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.104867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ef780-73ad-4f44-a4ea-2e794b0ac221 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.112176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.112817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.113092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.113365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.114036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.122432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.123326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.123483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.123895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.124739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.126508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.133921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.177492 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2693/5376] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:22.178462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.189975 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2684/5377] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:22 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:22.194212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.198462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76007b8-82de-4c2c-918a-f903c086c196 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.209075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:22.322749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba711a9-aae4-45ac-bde3-c8f6363e9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.323724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba711a9-aae4-45ac-bde3-c8f6363e9f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.324256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba711a9-aae4-45ac-bde3-c8f6363e9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.324693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba711a9-aae4-45ac-bde3-c8f6363e9f95 admin admin] PATH_INFO: `/v3/projects/03cbb3b2297d48b6aa22b240f3fd5057/users/e19057a0bb7b4fd7a89683bcaa243126/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.327034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba711a9-aae4-45ac-bde3-c8f6363e9f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cbb3b2297d48b6aa22b240f3fd5057, user_id=e19057a0bb7b4fd7a89683bcaa243126, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.331057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba711a9-aae4-45ac-bde3-c8f6363e9f95 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.346006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d852-8270-4f17-b8cd-3b4804ad5104 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.379703 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2694/5378] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:22 2022] PUT /identity/v3/projects/03cbb3b2297d48b6aa22b240f3fd5057/users/e19057a0bb7b4fd7a89683bcaa243126/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:22.380905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ddfea-921a-4d09-93f8-df902ebab950 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.381190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ddfea-921a-4d09-93f8-df902ebab950 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.381486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ddfea-921a-4d09-93f8-df902ebab950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.448568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.449293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.449661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.450841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.451730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.455329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.458735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5e7ddfea-921a-4d09-93f8-df902ebab950 None None] MFA Rules not processed for user `e19057a0bb7b4fd7a89683bcaa243126`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:22.464756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e7ddfea-921a-4d09-93f8-df902ebab950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:22.531996 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2685/5379] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:22 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:22.540499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.582495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7a29-3b59-4e6d-bd67-f931c5c6437f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.596068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.596702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.596980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.597230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.597923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.599534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.629280 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2686/5380] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:22.631852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.683696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0516d-0cc7-45a4-9b53-654557d5f22b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.702124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.702872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.703255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.703628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.704361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.706149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.721890 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2687/5381] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:22.732669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2695/5382] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:22.734387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ddfea-921a-4d09-93f8-df902ebab950 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.738295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.752744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ddfea-921a-4d09-93f8-df902ebab950 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.759553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159f242-b0ca-41de-928d-824bbac5fd91 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:22.764681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.767034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.767537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.768589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.770054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.770838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e80cd3-013a-4d31-915f-bf1be49aab2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:22.772892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.773715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e80cd3-013a-4d31-915f-bf1be49aab2e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.774096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e80cd3-013a-4d31-915f-bf1be49aab2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.774468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e80cd3-013a-4d31-915f-bf1be49aab2e admin admin] PATH_INFO: `/v3/projects/f64db349fc5543df9aad6a136e7c0825/users/5496699511ed4c979f300e0b0ca51aef/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.813957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e80cd3-013a-4d31-915f-bf1be49aab2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64db349fc5543df9aad6a136e7c0825, user_id=5496699511ed4c979f300e0b0ca51aef, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:22.823424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e80cd3-013a-4d31-915f-bf1be49aab2e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:22.840729 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2696/5383] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:22 2022] POST /identity/v3/projects => generated 360 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:22.842339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:22.853124 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2688/5384] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:22 2022] PUT /identity/v3/projects/f64db349fc5543df9aad6a136e7c0825/users/5496699511ed4c979f300e0b0ca51aef/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:22.864421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1708-8863-458d-a245-3a19907b1b17 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:22.864688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1708-8863-458d-a245-3a19907b1b17 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:22.865062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1708-8863-458d-a245-3a19907b1b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:22.923382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:22.966918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b8ed1708-8863-458d-a245-3a19907b1b17 None None] MFA Rules not processed for user `5496699511ed4c979f300e0b0ca51aef`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:22.968991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b8ed1708-8863-458d-a245-3a19907b1b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:23.063932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a6ce5-b1cc-4026-9264-0be91495a83d tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.088991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:23.158541 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2689/5385] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:23.159642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1708-8863-458d-a245-3a19907b1b17 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.164032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.164928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.165348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.166168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-1065644968` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.167900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=3a5dc83e36714ac498b0501c79014eae, value=tempest-tag-1065644968)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.177162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.293372 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1065644968. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:12:23.293372 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:12:23.293372 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:23.303677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1708-8863-458d-a245-3a19907b1b17 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.317723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.318370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.318646 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2697/5386] 10.222.0.26 () {60 vars in 1333 bytes} [Sat Sep 17 21:12:22 2022] PUT /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-1065644968 => generated 0 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 291 bytes (1 switches on core 0) Sep 17 21:12:23.318789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.318952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.319790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.322571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.323735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.352724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53f482-d0d6-4452-b0d1-d3b01257d03e tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.362757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.367931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.370477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.373611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-1065644968` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.385256 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2690/5387] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:23 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:23.387138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.428633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=3a5dc83e36714ac498b0501c79014eae, value=tempest-tag-1065644968)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.439207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.493688 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2698/5388] 10.222.0.26 () {58 vars in 1314 bytes} [Sat Sep 17 21:12:23 2022] GET /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-1065644968 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:23.495245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.539386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705dcf-238b-4084-b251-d224f0016b5b tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.660903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b4338-1a76-4660-8647-81ae90b877a9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.681204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.681487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.683613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.684057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.687066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.692782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.775058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.776020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.776466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.776922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.781366 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2691/5389] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:23 2022] POST /identity/v3/users => generated 298 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:23.782672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=3a5dc83e36714ac498b0501c79014eae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.784250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.787244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.799953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a7cf3-b1d7-4838-9c91-71d1f16c88f1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.814808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.815438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.816161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.816490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.818391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.820375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.842062 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2692/5390] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:23.843105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.851005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:23.861053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804128f-4e7f-49ac-a508-1c71af3299f8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.867957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.868722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.870551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.870889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] PATH_INFO: `/v3/projects/681e940b4aa3456c8612db81e417a9c6/users/78b5704954b648f3946005f4f1c3fb5d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.879338 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2699/5391] 10.222.0.26 () {60 vars in 1265 bytes} [Sat Sep 17 21:12:23 2022] PUT /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags => generated 139 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:23.883222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.897010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] RBAC: Authorizing `identity:create_grant(project_id=681e940b4aa3456c8612db81e417a9c6, user_id=78b5704954b648f3946005f4f1c3fb5d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.900150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f96a7-b808-4d5c-b601-a695d2a35c3a tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:23.900325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.906367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:23.907086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:23.907474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:23.907916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:23.924935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2693/5392] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:23 2022] PUT /identity/v3/projects/681e940b4aa3456c8612db81e417a9c6/users/78b5704954b648f3946005f4f1c3fb5d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:23.926304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.929232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=3a5dc83e36714ac498b0501c79014eae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:23.932949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:23.939733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:23.966245 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2700/5393] 10.222.0.26 () {58 vars in 1245 bytes} [Sat Sep 17 21:12:23 2022] GET /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags => generated 139 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:23.968519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:23.992511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5670a22-9292-47c9-be93-b179ca868c44 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.090231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d39377-7ade-46b3-a72e-9bfcc9f8793d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.102338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.102978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.103256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.103526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.104189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.106038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:24.144571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.145319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.145680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.146031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-810893421` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.148034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=3a5dc83e36714ac498b0501c79014eae, value=tempest-tag-810893421)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.150378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:24.152634 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2694/5394] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:24.154557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:24.166724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa7588-70ef-47b8-a979-24199de8395c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.181501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcab478-25fd-4590-9b8a-fd1767075def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.186468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab478-25fd-4590-9b8a-fd1767075def admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.186754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab478-25fd-4590-9b8a-fd1767075def admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.187030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab478-25fd-4590-9b8a-fd1767075def admin admin] PATH_INFO: `/v3/projects/681e940b4aa3456c8612db81e417a9c6/users/78b5704954b648f3946005f4f1c3fb5d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.196736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcab478-25fd-4590-9b8a-fd1767075def admin admin] RBAC: Authorizing `identity:create_grant(project_id=681e940b4aa3456c8612db81e417a9c6, user_id=78b5704954b648f3946005f4f1c3fb5d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.204312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcab478-25fd-4590-9b8a-fd1767075def admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:24.250646 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2695/5395] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:24 2022] PUT /identity/v3/projects/681e940b4aa3456c8612db81e417a9c6/users/78b5704954b648f3946005f4f1c3fb5d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:24.252442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c775584-65d9-49df-be7b-1f5b500210dd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.252730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c775584-65d9-49df-be7b-1f5b500210dd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.253602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c775584-65d9-49df-be7b-1f5b500210dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.320375 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-810893421. To ensure interoperability, identifiers should be a valid uuid. Sep 17 21:12:24.320375 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 17 21:12:24.320375 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:24.332509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8c775584-65d9-49df-be7b-1f5b500210dd None None] MFA Rules not processed for user `78b5704954b648f3946005f4f1c3fb5d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:24.336057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8c775584-65d9-49df-be7b-1f5b500210dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:24.353786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2701/5396] 10.222.0.26 () {58 vars in 1314 bytes} [Sat Sep 17 21:12:23 2022] DELETE /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-810893421 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:24.358141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:24.433806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f80e1d-3518-41cc-b65a-1d099f14f1c3 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.559063 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2696/5397] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:24.560670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.561339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.561668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.561921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-810893421` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.565186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c775584-65d9-49df-be7b-1f5b500210dd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:24.579697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=3a5dc83e36714ac498b0501c79014eae, value=tempest-tag-810893421)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.579948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c775584-65d9-49df-be7b-1f5b500210dd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.582096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:24.586096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b5704954b648f3946005f4f1c3fb5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '681e940b4aa3456c8612db81e417a9c6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.587213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.587590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.588231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.588976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.591244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Could not find project tag: tempest-tag-810893421.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-810893421. Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Sep 17 21:12:24.601824 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Sep 17 21:12:24.602968 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:24.602968 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:24.602968 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Sep 17 21:12:24.602968 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Sep 17 21:12:24.602968 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-810893421. Sep 17 21:12:24.602968 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:24.608262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2702/5398] 10.222.0.26 () {58 vars in 1311 bytes} [Sat Sep 17 21:12:24 2022] GET /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags/tempest-tag-810893421 => generated 106 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:24.609222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:24.626978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba059f20-5e97-4213-9b62-80db567b5fd1 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.637656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.638456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.638738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.639262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.640644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=3a5dc83e36714ac498b0501c79014eae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.642987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:24.695777 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2697/5399] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:24 2022] POST /identity/v3/projects => generated 359 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:24.697754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:24.738692 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2703/5400] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:12:24 2022] DELETE /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:24.743057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:24.784962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bab52-aaba-4fc5-aa42-7c17b4fe552f tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.799002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:24.801126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:24.887394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46536812-64b3-49bf-9beb-cbb07b14a187 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:24.893215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5496699511ed4c979f300e0b0ca51aef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64db349fc5543df9aad6a136e7c0825', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.893900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.894180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.894494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] PATH_INFO: `/v3/projects/220cdce14aa24967ac50694f354fe52f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.895918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] RBAC: Authorizing `identity:delete_project(project_id=220cdce14aa24967ac50694f354fe52f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Sep 17 21:12:24.903059 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 17 21:12:24.904107 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:24.905117 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2698/5401] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:24 2022] DELETE /identity/v3/projects/220cdce14aa24967ac50694f354fe52f => generated 136 bytes in 208 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:24.908385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:24.960580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:24.961608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:24.962048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:24.962485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:24.982967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=3a5dc83e36714ac498b0501c79014eae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:24.985357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:24.996586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f5c55-9098-4085-9b60-7b3b7ab96ecc tempest-ProjectsNegativeTestJSON-2033046706 tempest-ProjectsNegativeTestJSON-2033046706-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:25.003785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b5704954b648f3946005f4f1c3fb5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '681e940b4aa3456c8612db81e417a9c6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:25.004779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:25.005123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:25.005449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] PATH_INFO: `/v3/projects/220cdce14aa24967ac50694f354fe52f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:25.012646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] RBAC: Authorizing `identity:delete_project(project_id=220cdce14aa24967ac50694f354fe52f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:25.023398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:25.032685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2704/5402] 10.222.0.26 () {58 vars in 1245 bytes} [Sat Sep 17 21:12:24 2022] GET /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae/tags => generated 66 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 17 21:12:25.038838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:25.056527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139da14-e76d-4c3f-9f42-40881250e2f6 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:25.062538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19057a0bb7b4fd7a89683bcaa243126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03cbb3b2297d48b6aa22b240f3fd5057', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:25.063192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:25.063472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:25.063794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] PATH_INFO: `/v3/projects/3a5dc83e36714ac498b0501c79014eae` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:25.065504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3a5dc83e36714ac498b0501c79014eae)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:25.067941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:25.105206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220cdce14aa24967ac50694f354fe52f', 'request_id': 'req-7627f247-8784-474a-b367-7b61a87d9fb1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.112822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220cdce14aa24967ac50694f354fe52f', 'request_id': 'req-7627f247-8784-474a-b367-7b61a87d9fb1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.130196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220cdce14aa24967ac50694f354fe52f', 'request_id': 'req-7627f247-8784-474a-b367-7b61a87d9fb1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.153169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] The token cache is being invalidate because project 220cdce14aa24967ac50694f354fe52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:25.153638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.154523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.161715 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2699/5403] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:24 2022] DELETE /identity/v3/projects/220cdce14aa24967ac50694f354fe52f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:25.182105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:25.192406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:25.237572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5dc83e36714ac498b0501c79014eae', 'request_id': 'req-e8a8086c-792c-4609-898e-835b77e22dd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.279398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5dc83e36714ac498b0501c79014eae', 'request_id': 'req-e8a8086c-792c-4609-898e-835b77e22dd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.303566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5dc83e36714ac498b0501c79014eae', 'request_id': 'req-e8a8086c-792c-4609-898e-835b77e22dd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.367324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] The token cache is being invalidate because project 3a5dc83e36714ac498b0501c79014eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:25.367651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.371788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.386631 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2705/5404] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:25 2022] DELETE /identity/v3/projects/3a5dc83e36714ac498b0501c79014eae => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:25.388131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:25.392733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:25.483738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627f247-8784-474a-b367-7b61a87d9fb1 tempest-ProjectsNegativeTestJSON-410257828 tempest-ProjectsNegativeTestJSON-410257828-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:25.498432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:25.596549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8086c-792c-4609-898e-835b77e22dd0 tempest-IdentityV3ProjectTagsTest-38119369 tempest-IdentityV3ProjectTagsTest-38119369-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:25.613844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:25.615129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:25.615412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:25.615685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] PATH_INFO: `/v3/users/5e23682b95ea4ccf8ab384f182651d19` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:25.619266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e23682b95ea4ccf8ab384f182651d19)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:25.621347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:25.647959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:25.649684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:25.649969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:25.651406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] PATH_INFO: `/v3/users/5496699511ed4c979f300e0b0ca51aef` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:25.654828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5496699511ed4c979f300e0b0ca51aef)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:25.663034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:25.685632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:25.746304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e23682b95ea4ccf8ab384f182651d19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.794825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e23682b95ea4ccf8ab384f182651d19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.801903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5496699511ed4c979f300e0b0ca51aef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.803212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5496699511ed4c979f300e0b0ca51aef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.815866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e23682b95ea4ccf8ab384f182651d19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.816824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e23682b95ea4ccf8ab384f182651d19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.818427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eeb40859-c0d4-4aa9-b45f-536ff1226b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e23682b95ea4ccf8ab384f182651d19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.830026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5496699511ed4c979f300e0b0ca51aef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.830675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5496699511ed4c979f300e0b0ca51aef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.863397 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2706/5405] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:25 2022] DELETE /identity/v3/users/5e23682b95ea4ccf8ab384f182651d19 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:25.865044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:25.865329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:25.865630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:25.869247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0799e30-1ce7-46d1-aa68-d33747e686d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5496699511ed4c979f300e0b0ca51aef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:25.899473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2700/5406] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:25 2022] DELETE /identity/v3/users/5496699511ed4c979f300e0b0ca51aef => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:25.918691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:25.919061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:25.921448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:25.933514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:25.938072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:25.979025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:25.982206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:26.104140 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2707/5407] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:26.120493 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2701/5408] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:26.179623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:26.182644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:26.183181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:26.187024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:26.250643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:26.308835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d172f1-c79a-4bf4-872f-0d8d04431eff None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:26.313975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:26.315066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:26.315561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:26.316058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:26.319518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c54ebb-86c3-4402-abe1-fab34b3630d9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:26.329065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:26.330124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:26.330637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:26.331148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:26.335576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:26.337408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:26.354414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:26.357734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:26.367037 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2708/5409] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:26.377290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2702/5410] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:12:27.935319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:27.937550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:27.969174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4c874-ba0c-4de2-a8c4-9fe48e99c4a1 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:27.986562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:27.987233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:27.987507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:27.987778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] PATH_INFO: `/v3/projects/f64db349fc5543df9aad6a136e7c0825` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:27.988988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64db349fc5543df9aad6a136e7c0825)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:27.991127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:28.093118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:28.128824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64db349fc5543df9aad6a136e7c0825', 'request_id': 'req-82e7509e-ee72-4633-a219-38f2f7248c76'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.150468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64db349fc5543df9aad6a136e7c0825', 'request_id': 'req-82e7509e-ee72-4633-a219-38f2f7248c76'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.164523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64db349fc5543df9aad6a136e7c0825', 'request_id': 'req-82e7509e-ee72-4633-a219-38f2f7248c76'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.195600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] The token cache is being invalidate because project f64db349fc5543df9aad6a136e7c0825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:28.195955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.198134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.209162 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2709/5411] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:27 2022] DELETE /identity/v3/projects/f64db349fc5543df9aad6a136e7c0825 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:28.213916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:28.230628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391357b-f2f7-4bbc-a4fc-79e51c32e438 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:28.365049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:28.366114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.366540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.367114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] PATH_INFO: `/v3/users/78b5704954b648f3946005f4f1c3fb5d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:28.368706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b5704954b648f3946005f4f1c3fb5d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:28.371185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:28.468425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b5704954b648f3946005f4f1c3fb5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.471579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b5704954b648f3946005f4f1c3fb5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.504250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b5704954b648f3946005f4f1c3fb5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.505705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b5704954b648f3946005f4f1c3fb5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.557007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b5704954b648f3946005f4f1c3fb5d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.599644 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2703/5412] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:28 2022] DELETE /identity/v3/users/78b5704954b648f3946005f4f1c3fb5d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:28.697838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:28.699756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:28.734253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7509e-ee72-4633-a219-38f2f7248c76 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:28.906638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:28.908725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.910827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.911119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] PATH_INFO: `/v3/projects/2c1941b035064b0bb1c0be4f1870d01c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:28.914908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1941b035064b0bb1c0be4f1870d01c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:28.916929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.028283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1941b035064b0bb1c0be4f1870d01c', 'request_id': 'req-87ab8efc-2b3e-4e24-a967-79214cac60a2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.043111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1941b035064b0bb1c0be4f1870d01c', 'request_id': 'req-87ab8efc-2b3e-4e24-a967-79214cac60a2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.055637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1941b035064b0bb1c0be4f1870d01c', 'request_id': 'req-87ab8efc-2b3e-4e24-a967-79214cac60a2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.087407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] The token cache is being invalidate because project 2c1941b035064b0bb1c0be4f1870d01c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:29.088019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.089987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.092845 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2710/5413] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:28 2022] DELETE /identity/v3/projects/2c1941b035064b0bb1c0be4f1870d01c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:29.097427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.099022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:29.129127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722bfeb-2e00-472e-a72a-1ffbc1558da0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.216519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:29.234686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.235788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.236289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.236823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] PATH_INFO: `/v3/users/e19057a0bb7b4fd7a89683bcaa243126` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.238642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e19057a0bb7b4fd7a89683bcaa243126)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.243194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.401843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e19057a0bb7b4fd7a89683bcaa243126'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.402683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e19057a0bb7b4fd7a89683bcaa243126'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.421326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e19057a0bb7b4fd7a89683bcaa243126'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.423674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e19057a0bb7b4fd7a89683bcaa243126'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.469750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cf774b94-200d-4772-98a4-8f7c68714b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e19057a0bb7b4fd7a89683bcaa243126'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:29.488458 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2704/5414] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:29 2022] DELETE /identity/v3/users/e19057a0bb7b4fd7a89683bcaa243126 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:30.348089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:30.348842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:30.376873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab8efc-2b3e-4e24-a967-79214cac60a2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:30.482608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:30.504073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:30.505530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:30.506591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:30.507102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] PATH_INFO: `/v3/projects/681e940b4aa3456c8612db81e417a9c6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:30.509828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681e940b4aa3456c8612db81e417a9c6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:30.512979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:30.623224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681e940b4aa3456c8612db81e417a9c6', 'request_id': 'req-a8dfac4f-eb48-4078-a0f7-8eec660a0836'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:30.642260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681e940b4aa3456c8612db81e417a9c6', 'request_id': 'req-a8dfac4f-eb48-4078-a0f7-8eec660a0836'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:30.652070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681e940b4aa3456c8612db81e417a9c6', 'request_id': 'req-a8dfac4f-eb48-4078-a0f7-8eec660a0836'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:30.677929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] The token cache is being invalidate because project 681e940b4aa3456c8612db81e417a9c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:30.678236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:30.679062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a8dfac4f-eb48-4078-a0f7-8eec660a0836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:30.681706 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2711/5415] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:30 2022] DELETE /identity/v3/projects/681e940b4aa3456c8612db81e417a9c6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:30.693215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969f254-8d35-4307-8339-debc2e58af2a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:30.693726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969f254-8d35-4307-8339-debc2e58af2a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:30.693897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969f254-8d35-4307-8339-debc2e58af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:30.719077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0969f254-8d35-4307-8339-debc2e58af2a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:30.720836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0969f254-8d35-4307-8339-debc2e58af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:30.819013 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2705/5416] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:30.827265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cced64-3bfc-4aca-b702-8ebb830be058 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:30.828492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cced64-3bfc-4aca-b702-8ebb830be058 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:30.828736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cced64-3bfc-4aca-b702-8ebb830be058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:30.859465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-48cced64-3bfc-4aca-b702-8ebb830be058 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:30.860616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-48cced64-3bfc-4aca-b702-8ebb830be058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:30.872277 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2712/5417] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:30.877272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969f254-8d35-4307-8339-debc2e58af2a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:30.888631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969f254-8d35-4307-8339-debc2e58af2a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:30.894012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:30.894907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:30.895299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:30.895658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:30.896580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:30.896974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:30.899222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:30.916082 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2706/5418] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:30.921474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cced64-3bfc-4aca-b702-8ebb830be058 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:30.931269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cced64-3bfc-4aca-b702-8ebb830be058 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:30.937223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:30.937983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:30.938392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:30.938831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:30.939908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:30.942900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:31.000321 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2713/5419] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:30 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:31.005342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:31.024002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d88945-3b91-4bdf-b559-8f6f26f5842c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:31.109380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:31.119899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:31.120735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.121206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.122546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.123045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:31.128408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:31.135285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:31.137107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:31.152200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785601a-ac06-4d74-a1fe-012520c3cdc0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:31.157702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:31.158384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.159178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.159178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] PATH_INFO: `/v3/users/f3dd6e7db0af433d9746af0ec6ae6e08` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.160834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dd6e7db0af433d9746af0ec6ae6e08)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:31.163867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:31.191483 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2707/5420] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:31 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:31.195375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:31.214828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b7356-6381-4f3b-8c99-b77b7e6a4bcb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:31.223611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:31.224523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.225540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.225979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.226802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:31.229782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:31.241434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dd6e7db0af433d9746af0ec6ae6e08'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.249775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2708/5421] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:12:31.254377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:31.267082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dd6e7db0af433d9746af0ec6ae6e08'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.270232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5693e19-29dc-4a49-8daa-b2d7f8aa10d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:31.278249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:31.280175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dd6e7db0af433d9746af0ec6ae6e08'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.280601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.281038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.281256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dd6e7db0af433d9746af0ec6ae6e08'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.282072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.282312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e185b78-fcd9-42e5-8f1d-dbbb2a9316d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dd6e7db0af433d9746af0ec6ae6e08'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.283148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:31.285199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:31.303983 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2714/5422] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:31 2022] DELETE /identity/v3/users/f3dd6e7db0af433d9746af0ec6ae6e08 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:31.308464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.310378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.310828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.341095 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2709/5423] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:31.345175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:31.347041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:31.348126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:31.350323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:31.362632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998ae31-6449-4fa7-bbd9-f51b24aaa109 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:31.376173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:31.467591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2715/5424] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:31.468463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:31.469902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:31.470813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:31.472031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.472526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.472979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] PATH_INFO: `/v3/projects/ad85b9c7350e45a3b4598302209b3aa5/users/bfafce367b684265bd8263f39da04048/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.493841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4431eb-d9f1-4bf2-b4e1-4015644974b6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:31.501733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad85b9c7350e45a3b4598302209b3aa5, user_id=bfafce367b684265bd8263f39da04048, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:31.506015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:31.506250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:31.506922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.507203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.507487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] PATH_INFO: `/v3/projects/03cbb3b2297d48b6aa22b240f3fd5057` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.508899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03cbb3b2297d48b6aa22b240f3fd5057)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:31.512319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:31.532300 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2710/5425] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:31 2022] PUT /identity/v3/projects/ad85b9c7350e45a3b4598302209b3aa5/users/bfafce367b684265bd8263f39da04048/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:31.533244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:31.542962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:31.578220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03cbb3b2297d48b6aa22b240f3fd5057', 'request_id': 'req-42f7a03d-497b-4ee1-9d86-0f4419dcf553'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.591075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03cbb3b2297d48b6aa22b240f3fd5057', 'request_id': 'req-42f7a03d-497b-4ee1-9d86-0f4419dcf553'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.600428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03cbb3b2297d48b6aa22b240f3fd5057', 'request_id': 'req-42f7a03d-497b-4ee1-9d86-0f4419dcf553'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.620768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] The token cache is being invalidate because project 03cbb3b2297d48b6aa22b240f3fd5057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:31.621393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.622798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-42f7a03d-497b-4ee1-9d86-0f4419dcf553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:31.627045 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2716/5426] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:31 2022] DELETE /identity/v3/projects/03cbb3b2297d48b6aa22b240f3fd5057 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:31.633630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa33a9-f74b-4547-a025-08a54a3692f0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.652382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa33a9-f74b-4547-a025-08a54a3692f0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.653111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa33a9-f74b-4547-a025-08a54a3692f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.661761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a14557-c5b7-4e5f-8e95-444dfb5e3b26 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:31.718319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-24aa33a9-f74b-4547-a025-08a54a3692f0 None None] MFA Rules not processed for user `bfafce367b684265bd8263f39da04048`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:31.719999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-24aa33a9-f74b-4547-a025-08a54a3692f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:31.762989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:31.763782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.764354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.764784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.767805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:31.813176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:31.970023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2717/5427] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:31.977227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b95ad-a9b9-4643-a6b7-30c3c05a9052 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:31.977579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b95ad-a9b9-4643-a6b7-30c3c05a9052 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:31.977871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b95ad-a9b9-4643-a6b7-30c3c05a9052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:31.979705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:31.983510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.010171 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2711/5428] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:32.010678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.020817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b33b95ad-a9b9-4643-a6b7-30c3c05a9052 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:32.021854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b33b95ad-a9b9-4643-a6b7-30c3c05a9052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:32.023109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:32.041514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2718/5429] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:32.050944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90086265-c196-4c98-bdeb-b60e2dd9eb89 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.051479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90086265-c196-4c98-bdeb-b60e2dd9eb89 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.052034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90086265-c196-4c98-bdeb-b60e2dd9eb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.063156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6316d-1a93-46be-864d-ac3a520f1ba5 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.074465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.075220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.075577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.076050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.078317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.083925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.092413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-90086265-c196-4c98-bdeb-b60e2dd9eb89 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:32.094522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-90086265-c196-4c98-bdeb-b60e2dd9eb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:32.155150 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2719/5430] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:32.160004 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2712/5431] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:32 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:32.160777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90086265-c196-4c98-bdeb-b60e2dd9eb89 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.163817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.208630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90086265-c196-4c98-bdeb-b60e2dd9eb89 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.242589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:32.305230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.306398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.307037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.307623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.309985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:32.310489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.312649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.318331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f41be0-1ecd-4c96-a56d-0b306c215189 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.337830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.338578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.338925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.339362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.340176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.341551 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2720/5432] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:32.342348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.342510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.354692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348b92b-555e-4786-8dee-8a0ba4e2d1fb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.359930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.360563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.360839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.361314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.362399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.364457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.396208 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2713/5433] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:32 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:32.400311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.412662 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2721/5434] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:32 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:32.417901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.421654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d854e-cb29-4327-892c-5ffdbce73711 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.514974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.515450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.515911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.516243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.516535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7801-eca1-49d8-b43b-b4ef8489a2d6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.516905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.519388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.524499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.525241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.525587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.525902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.526738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.531156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.537016 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2714/5435] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:12:32.542005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.558132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da820ae-98d5-438d-84fb-d2f98d0b90f3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.566338 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2722/5436] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:32 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:32.566580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.566580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.566580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.566580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] PATH_INFO: `/v3/projects/677f9030a37e4d6c82c5b9b9e43fece3/users/2cc8125d1fb24485a0a9d41f1ff5fe61/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.568580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.582642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb121d-4bb0-4e1c-9cd9-e9eedbe57ed4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.588641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.588911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=677f9030a37e4d6c82c5b9b9e43fece3, user_id=2cc8125d1fb24485a0a9d41f1ff5fe61, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.589876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.590325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.590765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.591589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.592092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.593729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.594999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:32.611164 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2723/5437] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:32.612404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.625575 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2715/5438] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:32 2022] PUT /identity/v3/projects/677f9030a37e4d6c82c5b9b9e43fece3/users/2cc8125d1fb24485a0a9d41f1ff5fe61/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:32.627028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.639001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda6059-4e31-4eec-bae4-727326e676fa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.719850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.720464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.720739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.721022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.721675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.723288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.740343 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2724/5439] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:32.741262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.758782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a99029-26b6-40f0-82fb-6a28420b11b8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.765406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f82c52-ddbc-4063-9f5d-2f9cb1e321e3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.766260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.767156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.767439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.768249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.768911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.770599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.783862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2716/5440] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:32.787760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cb976-39f1-4fca-b76c-dea1601af84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.788453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cb976-39f1-4fca-b76c-dea1601af84d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.788732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cb976-39f1-4fca-b76c-dea1601af84d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.789005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cb976-39f1-4fca-b76c-dea1601af84d admin admin] PATH_INFO: `/v3/projects/a7158dd65f8541798e8fbcaa62aa2725/users/68181fb50df646f88dac4f4d2e4f29b3/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.790115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:32.802105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034900-9105-4de1-bc52-87d8e2cd95c5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:32.809631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b209fa-0717-41c7-92ae-6d5ffb7811bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:32.810416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b209fa-0717-41c7-92ae-6d5ffb7811bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.810891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b209fa-0717-41c7-92ae-6d5ffb7811bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.811167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b209fa-0717-41c7-92ae-6d5ffb7811bd admin admin] PATH_INFO: `/v3/projects/677f9030a37e4d6c82c5b9b9e43fece3/users/2cc8125d1fb24485a0a9d41f1ff5fe61/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.813354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b209fa-0717-41c7-92ae-6d5ffb7811bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=677f9030a37e4d6c82c5b9b9e43fece3, user_id=2cc8125d1fb24485a0a9d41f1ff5fe61, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.816023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:32.816177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b209fa-0717-41c7-92ae-6d5ffb7811bd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.829450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cb976-39f1-4fca-b76c-dea1601af84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7158dd65f8541798e8fbcaa62aa2725, user_id=68181fb50df646f88dac4f4d2e4f29b3, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:32.833853 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2717/5441] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:32 2022] PUT /identity/v3/projects/677f9030a37e4d6c82c5b9b9e43fece3/users/2cc8125d1fb24485a0a9d41f1ff5fe61/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:32.834043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cb976-39f1-4fca-b76c-dea1601af84d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:32.844611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93e6f8-2b62-4f72-9cc7-a94d9b3b81b7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.844946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93e6f8-2b62-4f72-9cc7-a94d9b3b81b7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.845222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93e6f8-2b62-4f72-9cc7-a94d9b3b81b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.865708 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2725/5442] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:32 2022] PUT /identity/v3/projects/a7158dd65f8541798e8fbcaa62aa2725/users/68181fb50df646f88dac4f4d2e4f29b3/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:32.876664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065e0c1-fb66-418c-a192-68ac75304a8d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:32.876947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065e0c1-fb66-418c-a192-68ac75304a8d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:32.877219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065e0c1-fb66-418c-a192-68ac75304a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:32.908994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bb93e6f8-2b62-4f72-9cc7-a94d9b3b81b7 None None] MFA Rules not processed for user `2cc8125d1fb24485a0a9d41f1ff5fe61`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:32.910271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bb93e6f8-2b62-4f72-9cc7-a94d9b3b81b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:32.958205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8065e0c1-fb66-418c-a192-68ac75304a8d None None] MFA Rules not processed for user `68181fb50df646f88dac4f4d2e4f29b3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:32.961545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8065e0c1-fb66-418c-a192-68ac75304a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:33.131289 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2718/5443] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:33.136295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93e6f8-2b62-4f72-9cc7-a94d9b3b81b7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.157204 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2726/5444] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:12:33.158458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93e6f8-2b62-4f72-9cc7-a94d9b3b81b7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.161246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065e0c1-fb66-418c-a192-68ac75304a8d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.173357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:33.174165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.174447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.174957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.175638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:33.182632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:33.219081 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2719/5445] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/roles => generated 234 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:33.220195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.244445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c406091-7aea-4d53-a27d-291e6c348338 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.266357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:33.267357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.269203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.269579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.271222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:33.273811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:33.303903 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2720/5446] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/roles => generated 235 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:33.308593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.314940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065e0c1-fb66-418c-a192-68ac75304a8d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.326665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:33.330206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.331429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.333434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.334133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:33.338306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6e0d6-6993-4fa5-b0f4-e61cf6564b1c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.339673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:33.358181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:33.359442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.362922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.364735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.365580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:33.374189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:33.415025 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2721/5447] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/roles => generated 234 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:33.420133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.426039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:33.431221 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2727/5448] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/projects => generated 369 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:33.432234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.456995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c32c49-ed69-48b9-9a08-ff8cf8f73b31 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.488992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:33.571495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:33.572484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.572611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.572803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] PATH_INFO: `/v3/projects/27a2296cdbb84eada80ed0322c8c680a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.574115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a2296cdbb84eada80ed0322c8c680a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:33.576537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:33.621044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1157c-a000-440b-ae2c-8e22abac6aaa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.639164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:33.639830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.641350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.641786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.648105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:33.652471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:33.681811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:33.703156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2728/5449] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/domains => generated 275 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:33.704336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.724947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a2296cdbb84eada80ed0322c8c680a', 'request_id': 'req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:33.740625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a2296cdbb84eada80ed0322c8c680a', 'request_id': 'req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:33.776721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a2296cdbb84eada80ed0322c8c680a', 'request_id': 'req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:33.800343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e558f0-3352-438e-8634-b7be8c8dfd34 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.809352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] The token cache is being invalidate because project 27a2296cdbb84eada80ed0322c8c680a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:33.811844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:33.814495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:33.824520 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2722/5450] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:33 2022] DELETE /identity/v3/projects/27a2296cdbb84eada80ed0322c8c680a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:33.826184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.830271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:33.833719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91518711-7624-4a25-8fae-c7fb873972b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:33.834364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518711-7624-4a25-8fae-c7fb873972b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.835943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518711-7624-4a25-8fae-c7fb873972b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.865596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518711-7624-4a25-8fae-c7fb873972b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.866454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91518711-7624-4a25-8fae-c7fb873972b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:33.879159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91518711-7624-4a25-8fae-c7fb873972b8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:33.920344 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2729/5451] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:33.921855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:33.922278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:33.922989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:33.947952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:33.948967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc44a8-d816-41ec-adbe-fdd23da5c6e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:33.949421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:33.965196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2730/5452] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:33.969222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:33.971345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:34.048583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.049222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.049527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.049808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.061707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.063968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.076217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:34.139776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee401ce-59b7-4d13-b36c-3f71c11b0067 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.152069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba110a1f-e099-46bb-a47d-ae2331026bc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.153693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110a1f-e099-46bb-a47d-ae2331026bc9 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.154269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110a1f-e099-46bb-a47d-ae2331026bc9 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.155603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110a1f-e099-46bb-a47d-ae2331026bc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.166918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ba110a1f-e099-46bb-a47d-ae2331026bc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:34.177606 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2723/5453] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:33 2022] POST /identity/v3/projects => generated 400 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:34.182634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.186243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:34.271382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba110a1f-e099-46bb-a47d-ae2331026bc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.273155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba110a1f-e099-46bb-a47d-ae2331026bc9 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.286947 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2731/5454] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:33 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:34.289425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00647763-e3b2-4710-a5c7-db10974ac8ef None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.290118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00647763-e3b2-4710-a5c7-db10974ac8ef None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.290498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00647763-e3b2-4710-a5c7-db10974ac8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.292880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15799e-2721-45a7-9f18-75cb414d33d6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.307097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.307738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.308019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.308347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.309010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.310719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.317620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-00647763-e3b2-4710-a5c7-db10974ac8ef None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:34.318857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-00647763-e3b2-4710-a5c7-db10974ac8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:34.345693 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2724/5455] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:34 2022] GET /identity/v3/roles => generated 2096 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:34.346553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.369672 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2732/5456] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:34.369672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55cfe1-efa3-446b-acd0-0292cc79cd96 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.369672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55cfe1-efa3-446b-acd0-0292cc79cd96 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.369672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55cfe1-efa3-446b-acd0-0292cc79cd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.414588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ab55cfe1-efa3-446b-acd0-0292cc79cd96 None None] MFA Rules not processed for user `caa2c2f4848e497f80d4a439c785c2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:34.421936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ab55cfe1-efa3-446b-acd0-0292cc79cd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:34.428395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:34.466276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2f275-9c53-4556-8d57-6c566adda22f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.472803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.473528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.473744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.474024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/groups` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.474669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.477645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.504787 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2725/5457] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:12:34 2022] POST /identity/v3/groups => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:34.504787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.510346 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2733/5458] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:34.512067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55cfe1-efa3-446b-acd0-0292cc79cd96 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.520655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5b3b9-e63d-4c2d-a2f6-8bad22fd73f2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.527083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.528241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.528609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.528873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] PATH_INFO: `/v3/projects/05f135241b5243a195fa3cf603041d25/users/2d35db83e16646f5a34a0af723ce671a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.529303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55cfe1-efa3-446b-acd0-0292cc79cd96 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.539672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.540313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.540580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.540846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.541552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:34.541854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.544266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.552750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f135241b5243a195fa3cf603041d25, user_id=2d35db83e16646f5a34a0af723ce671a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.558113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.564690 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2734/5459] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:34.564690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.577723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a3f07-e763-4100-a5aa-7de2974e2cb0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.584067 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2726/5460] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:34 2022] PUT /identity/v3/projects/05f135241b5243a195fa3cf603041d25/users/2d35db83e16646f5a34a0af723ce671a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:34.585353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.658125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:34.674782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.677504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.677753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.678074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.681768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.684038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.719705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d80b4b-75af-4075-8eb0-a1f318e8ad13 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.733396 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2735/5461] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:34 2022] POST /identity/v3/roles => generated 235 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:34.734186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.735268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.736536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.736814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.737081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.737755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.746648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.769610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47765cc5-5f41-4e61-9fda-7743760b398b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:34.776990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:34.777667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:34.777945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:34.778218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:34.778872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:34.782130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:34.812642 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2727/5462] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:34 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:34.812642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.813314 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2736/5463] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:34 2022] GET /identity/v3/roles => generated 2322 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:34.818810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:34.844263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71386e8-1b60-49b9-8307-71c10a29361e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.013642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c085829-ba90-4ca4-8ec1-3fe6f034caba admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.022937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.023564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.032508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.034119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.035174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.035174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.035413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.035924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.037718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.038454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.041125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.041775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.115398 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2737/5464] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:34 2022] POST /identity/v3/users => generated 290 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:35.116502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.122199 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2728/5465] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:34 2022] POST /identity/v3/users => generated 442 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:35.122199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.142470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3e943-4fa7-48a0-b7c0-59fcceae7f81 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.147109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b417ff-8206-4e3d-82bc-ee8d9b64b1a0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.153306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.153619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.154265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.154523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.154792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.155074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.155447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.155794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.156565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] PATH_INFO: `/v3/projects/05f135241b5243a195fa3cf603041d25/users/2d35db83e16646f5a34a0af723ce671a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.161377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.162101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f135241b5243a195fa3cf603041d25, user_id=2d35db83e16646f5a34a0af723ce671a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.167653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.187759 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2729/5466] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:35 2022] GET /identity/v3/roles => generated 2322 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:35.188459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.213518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c5f10-76b0-41ff-9154-26f06421229d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.215122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2738/5467] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:35 2022] PUT /identity/v3/projects/05f135241b5243a195fa3cf603041d25/users/2d35db83e16646f5a34a0af723ce671a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:35.220256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.232702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:35.337710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f688b85-a5a7-4cfd-9243-1c6947b2b35f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.338662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f688b85-a5a7-4cfd-9243-1c6947b2b35f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.339266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f688b85-a5a7-4cfd-9243-1c6947b2b35f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.339619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f688b85-a5a7-4cfd-9243-1c6947b2b35f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.340302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f688b85-a5a7-4cfd-9243-1c6947b2b35f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.341767 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-0f688b85-a5a7-4cfd-9243-1c6947b2b35f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:35.341767 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:35.341767 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:35.342695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f688b85-a5a7-4cfd-9243-1c6947b2b35f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.352328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30fcf-22aa-494a-8859-56a739fcacdf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.361220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.362527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.362979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.363257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.363918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.365602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.367107 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2730/5468] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:35 2022] POST /identity/v3/roles => generated 270 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:35.368622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c2b59-9358-4681-9bd3-14774e9941ba None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.368812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c2b59-9358-4681-9bd3-14774e9941ba None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.369061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c2b59-9358-4681-9bd3-14774e9941ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.380537 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2739/5469] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:35 2022] GET /identity/v3/roles => generated 2322 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:35.384771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.398638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722d13a-310f-4b72-89b9-18976978e0cd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.405738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.406494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.406852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.407199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b29fa5fcfcf249f98a44433fa8bd24fb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.446720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=b29fa5fcfcf249f98a44433fa8bd24fb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.452912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.465439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b94c2b59-9358-4681-9bd3-14774e9941ba None None] MFA Rules not processed for user `2d35db83e16646f5a34a0af723ce671a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:35.467010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b94c2b59-9358-4681-9bd3-14774e9941ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:35.479030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2740/5470] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:35 2022] PUT /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b29fa5fcfcf249f98a44433fa8bd24fb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:35.481647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.512194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:35.580220 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2731/5471] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:35.581355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c2b59-9358-4681-9bd3-14774e9941ba None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.586185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e947572-b8b8-428c-a4d3-94ff6ab8149c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.592858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.594766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.595222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.595660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] PATH_INFO: `/v3/projects/2841af814e884a7e83a1e08aa1dd6133/users/a6bcb2ea4b03423cb720059fc73d24f4/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.600526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c2b59-9358-4681-9bd3-14774e9941ba None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.619617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2841af814e884a7e83a1e08aa1dd6133, user_id=a6bcb2ea4b03423cb720059fc73d24f4, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.623132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.649564 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2741/5472] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:35 2022] PUT /identity/v3/projects/2841af814e884a7e83a1e08aa1dd6133/users/a6bcb2ea4b03423cb720059fc73d24f4/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:35.650547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.699599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caee1d-ad0c-4b46-91e1-f88fcd091fae tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.706566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caee1d-ad0c-4b46-91e1-f88fcd091fae tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.707353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caee1d-ad0c-4b46-91e1-f88fcd091fae tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.707788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caee1d-ad0c-4b46-91e1-f88fcd091fae tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.708533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caee1d-ad0c-4b46-91e1-f88fcd091fae tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Adding query filter params (scope.project.id=963e5b531cf64e41b26bb29d69e0a735, user.id=faf54362aa9a4047bcb2c067f7b9b02e) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:35.708838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caee1d-ad0c-4b46-91e1-f88fcd091fae tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.712322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caee1d-ad0c-4b46-91e1-f88fcd091fae tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.749991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:35.756534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e5297-c545-419c-97c0-aafc4853dab8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.767912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.769073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.769356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.769691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.770739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:35.771997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:35.800136 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2732/5473] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:12:35 2022] GET /identity/v3/role_assignments?scope.project.id=963e5b531cf64e41b26bb29d69e0a735&user.id=faf54362aa9a4047bcb2c067f7b9b02e&effective => generated 228 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:35.801932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d26d0a-4815-46d7-976b-b5a4f62ecde9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.802680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d26d0a-4815-46d7-976b-b5a4f62ecde9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.802927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d26d0a-4815-46d7-976b-b5a4f62ecde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:35.816311 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2742/5474] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:12:35 2022] POST /identity/v3/services => generated 239 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:35.817442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:35.857987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e2d26d0a-4815-46d7-976b-b5a4f62ecde9 None None] MFA Rules not processed for user `a6bcb2ea4b03423cb720059fc73d24f4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:35.862223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e2d26d0a-4815-46d7-976b-b5a4f62ecde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:35.975861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fef98-1815-40dd-891b-cdc6f21b5c60 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:35.990701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:35.997389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:35.999236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:35.999722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b29fa5fcfcf249f98a44433fa8bd24fb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.007896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=b29fa5fcfcf249f98a44433fa8bd24fb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.013767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.017109 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2733/5475] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3388 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:36.018226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d26d0a-4815-46d7-976b-b5a4f62ecde9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:36.045983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:36.067438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d26d0a-4815-46d7-976b-b5a4f62ecde9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:36.067669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role b29fa5fcfcf249f98a44433fa8bd24fb was removed from user faf54362aa9a4047bcb2c067f7b9b02e on project 963e5b531cf64e41b26bb29d69e0a735. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:36.067669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.071218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.076829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:36.078058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:36.079128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:36.080218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/3c35d825ea714f18a084307e3142a0e7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.080926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:delete_service(service_id=3c35d825ea714f18a084307e3142a0e7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.084765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.100658 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2743/5476] 10.222.0.26 () {58 vars in 1467 bytes} [Sat Sep 17 21:12:35 2022] DELETE /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b29fa5fcfcf249f98a44433fa8bd24fb => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:36.105800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:36.107403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:36.135921 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2734/5477] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:36 2022] DELETE /identity/v3/services/3c35d825ea714f18a084307e3142a0e7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:36.139999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:36.144552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:36.178781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8eeae-d216-481b-8668-91e62ba8a696 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:36.254309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829000b-3323-40f9-9fc1-05c0461a3129 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:36.276289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:36.279815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:36.280741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:36.284773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:36.285766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:36.286300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:36.286580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.288256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b29fa5fcfcf249f98a44433fa8bd24fb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.288448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:36.290637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=b29fa5fcfcf249f98a44433fa8bd24fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.292746 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:36.292746 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:36.292746 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:36.294147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.381385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role b29fa5fcfcf249f98a44433fa8bd24fb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:36.384442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.385888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.396514 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2735/5478] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:36 2022] DELETE /identity/v3/roles/b29fa5fcfcf249f98a44433fa8bd24fb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:36.396514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:36.398785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:36.493097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.495941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.523253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:36.560323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2744/5479] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:36.561768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:36.574511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22893c-22ac-452a-bd7e-de11f40e1ec7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:36.585906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:36.586703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:36.588055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b08b5e-1003-44f3-ad7a-57762f61bea4 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:36.589025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:36.590139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.590740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.598994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.644409 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2736/5480] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:36 2022] POST /identity/v3/services => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:36.646035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:36.653041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:36.701808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:36.731788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:36.732476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:36.732755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:36.733034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b348ff808766462fbeb7304f6c5aae6d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.746569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=b348ff808766462fbeb7304f6c5aae6d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.752544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.784442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb9b3c-86fa-402b-97c9-4437778f35c3 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:36.812447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2745/5481] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:36 2022] PUT /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b348ff808766462fbeb7304f6c5aae6d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:36.816260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:36.847348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:36.848014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:36.848282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:36.848552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] PATH_INFO: `/v3/users/a6bcb2ea4b03423cb720059fc73d24f4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.849882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bcb2ea4b03423cb720059fc73d24f4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.851832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.942387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bcb2ea4b03423cb720059fc73d24f4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.944608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bcb2ea4b03423cb720059fc73d24f4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.959374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c332ce-93de-4efe-8fd2-e98aa6af9e56 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:36.961891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bcb2ea4b03423cb720059fc73d24f4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.963137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bcb2ea4b03423cb720059fc73d24f4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:36.972934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:36.974795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:36.974998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:36.975326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/9eecba2e375f48009ddb28814b595dcf` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:36.975920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:update_service(service_id=9eecba2e375f48009ddb28814b595dcf)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:36.979853 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:36.979853 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:36.979853 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:36.983313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:36.990581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bcb2ea4b03423cb720059fc73d24f4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:37.011951 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2746/5482] 10.222.0.26 () {60 vars in 1252 bytes} [Sat Sep 17 21:12:36 2022] PATCH /identity/v3/services/9eecba2e375f48009ddb28814b595dcf => generated 282 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:37.012716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.015345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:37.021286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2737/5483] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:36 2022] DELETE /identity/v3/users/a6bcb2ea4b03423cb720059fc73d24f4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:37.022324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.025258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:37.115721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:37.146031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83725786-64c1-4af3-9be4-3b3241a084a1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.166902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be003a2d-01d1-41a2-bd56-e143cc6caf66 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.167809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be003a2d-01d1-41a2-bd56-e143cc6caf66 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.167975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be003a2d-01d1-41a2-bd56-e143cc6caf66 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.168241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be003a2d-01d1-41a2-bd56-e143cc6caf66 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/9eecba2e375f48009ddb28814b595dcf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.168941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be003a2d-01d1-41a2-bd56-e143cc6caf66 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:get_service(service_id=9eecba2e375f48009ddb28814b595dcf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.169991 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-be003a2d-01d1-41a2-bd56-e143cc6caf66 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:37.169991 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:37.169991 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:37.170969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be003a2d-01d1-41a2-bd56-e143cc6caf66 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.178012 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2738/5484] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:12:37 2022] GET /identity/v3/services/9eecba2e375f48009ddb28814b595dcf => generated 282 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:37.178228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af83f2-ba63-4d59-a1a5-4c00a38ac290 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.179456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0a24-8506-4da2-ac47-3b6c7be49244 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.179745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0a24-8506-4da2-ac47-3b6c7be49244 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.180082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d0a24-8506-4da2-ac47-3b6c7be49244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.185772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:37.238289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4c2d0a24-8506-4da2-ac47-3b6c7be49244 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:37.239657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4c2d0a24-8506-4da2-ac47-3b6c7be49244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:37.310941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.313031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.313327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.313726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.316323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.317483 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:37.317483 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:37.317483 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:37.318410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.338115 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2739/5485] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3371 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:37.340276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0a24-8506-4da2-ac47-3b6c7be49244 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.355679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d0a24-8506-4da2-ac47-3b6c7be49244 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.376649 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2747/5486] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:37 2022] PUT /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 520 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:37.376649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.378816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:37.425036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.425721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.425994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.426263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/9eecba2e375f48009ddb28814b595dcf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.426868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:delete_service(service_id=9eecba2e375f48009ddb28814b595dcf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.428822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.457764 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2740/5487] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:37 2022] DELETE /identity/v3/services/9eecba2e375f48009ddb28814b595dcf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:37.461688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.461863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb33bd-a445-413c-b49a-66a75433b89f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.478074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.478849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.479229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.479693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.520854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:37.550684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.552434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.571957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06173618-1b25-4775-98ec-5d74c2ce03e8 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.576055 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2748/5488] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:37.577312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.592573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.593559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.593878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.594389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.595118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Adding query filter params (scope.project.id=963e5b531cf64e41b26bb29d69e0a735, user.id=faf54362aa9a4047bcb2c067f7b9b02e) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:37.595462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.596796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:37.599294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.602172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c7e29-fde8-4c20-bb28-c60b28d69b47 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.612542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.615197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.615523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.616446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/9eecba2e375f48009ddb28814b595dcf` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.617151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:get_service(service_id=9eecba2e375f48009ddb28814b595dcf)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.618221 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:37.618221 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:37.618221 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:37.619124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Could not find service: 9eecba2e375f48009ddb28814b595dcf.: keystone.exception.ServiceNotFound: Could not find service: 9eecba2e375f48009ddb28814b595dcf. Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 9eecba2e375f48009ddb28814b595dcf. Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:37.641922 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:12:37.643331 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 9eecba2e375f48009ddb28814b595dcf. Sep 17 21:12:37.644568 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:37.652018 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2749/5489] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:12:37 2022] GET /identity/v3/services/9eecba2e375f48009ddb28814b595dcf => generated 113 bytes in 71 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:37.653937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.668939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7340d9-3811-4868-96df-0b6c7e17eca6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.682569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.683434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.683716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.683983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.684756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.686437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.699612 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2741/5490] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:12:37 2022] GET /identity/v3/role_assignments?scope.project.id=963e5b531cf64e41b26bb29d69e0a735&user.id=faf54362aa9a4047bcb2c067f7b9b02e&effective => generated 1061 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:37.704725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.707172 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2750/5491] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:37 2022] POST /identity/v3/services => generated 275 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:37.711491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.736250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e929a-17c3-4425-8c7b-371ef5e9b59b tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.746945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3a3c3-cce7-4bc9-bd80-7a98eb98d928 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.753721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.754957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.754957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.754957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.756197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.758535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.762706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.765412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.765858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.769221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.773507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.776054 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:37.776054 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:37.776054 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:37.778321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.788858 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2751/5492] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:37 2022] POST /identity/v3/services => generated 273 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:37.790577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.814400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2742/5493] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:37 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:37.819914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.821747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cc216-036f-4b25-8293-7b8c9c97ece6 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.900126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.901231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.902073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.902562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.903567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.906832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:37.927301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c6ae-cbbf-4b7a-a580-4e9b8a201c9d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:37.933038 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2752/5494] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:37 2022] POST /identity/v3/services => generated 274 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:37.934561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:37.983174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:37.983867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:37.985514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:37.986704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:37.988412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Adding query filter params (scope.project.id=963e5b531cf64e41b26bb29d69e0a735, user.id=faf54362aa9a4047bcb2c067f7b9b02e) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:37.989392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:37.996420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.020248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0da93d-5a7b-4538-a5ba-5832abfab74e tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.046043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.046677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.048183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.048410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.049031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.060926 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:38.060926 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:38.060926 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:38.062473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.100700 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2753/5495] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:12:37 2022] GET /identity/v3/services => generated 2977 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:38.101627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.117698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:38.164646 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2743/5496] 10.222.0.26 () {58 vars in 1356 bytes} [Sat Sep 17 21:12:37 2022] GET /identity/v3/role_assignments?scope.project.id=963e5b531cf64e41b26bb29d69e0a735&user.id=faf54362aa9a4047bcb2c067f7b9b02e&effective => generated 577 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:38.166139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.189063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf6471-05a0-4b01-ae02-68a92db442ec tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.200430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.201069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.201351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.201648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.202851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1784217321) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:38.205257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.207619 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:38.207619 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:38.207619 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:38.208540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.222510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2744/5497] 10.222.0.26 () {58 vars in 1222 bytes} [Sat Sep 17 21:12:38 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1784217321 => generated 418 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:38.223409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.244194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad1d20-9c2c-4b89-9893-10c255080f48 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.245227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d78062-2d19-4290-b1ad-923e90558ccb tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.253392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.254084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.254364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.254639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] PATH_INFO: `/v3/projects/2841af814e884a7e83a1e08aa1dd6133` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.255954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.256620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.256897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.257171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.258366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2841af814e884a7e83a1e08aa1dd6133)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.260310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.263174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.264797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27.: keystone.exception.ImpliedRoleNotFound: b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27. Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = query.one() Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._iter().one() Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._only_one_row( Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:38.296462 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 17 21:12:38.301112 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 17 21:12:38.302392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 17 21:12:38.302392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 17 21:12:38.302392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 17 21:12:38.302392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27. Sep 17 21:12:38.302392 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:38.302392 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2745/5498] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:38 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 137 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:38.302392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.303586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:38.312276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:38.322170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3678a1-8779-483c-8a1c-7bdf52661810 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.349394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2841af814e884a7e83a1e08aa1dd6133', 'request_id': 'req-dbaae095-63c9-4859-b498-6eabdea0dc1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:38.365140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2841af814e884a7e83a1e08aa1dd6133', 'request_id': 'req-dbaae095-63c9-4859-b498-6eabdea0dc1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:38.375230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2841af814e884a7e83a1e08aa1dd6133', 'request_id': 'req-dbaae095-63c9-4859-b498-6eabdea0dc1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:38.393073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] The token cache is being invalidate because project 2841af814e884a7e83a1e08aa1dd6133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:38.393437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:38.394261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:38.399670 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2754/5499] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:38 2022] DELETE /identity/v3/projects/2841af814e884a7e83a1e08aa1dd6133 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:38.399670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.402789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:38.405946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabf31d-d0e5-4c9c-a55f-667388abc3db tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.406624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabf31d-d0e5-4c9c-a55f-667388abc3db tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.406910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabf31d-d0e5-4c9c-a55f-667388abc3db tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.407186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabf31d-d0e5-4c9c-a55f-667388abc3db tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.408163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabf31d-d0e5-4c9c-a55f-667388abc3db tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-358737642) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:38.408292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabf31d-d0e5-4c9c-a55f-667388abc3db tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.410081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabf31d-d0e5-4c9c-a55f-667388abc3db tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.417361 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2746/5500] 10.222.0.26 () {58 vars in 1220 bytes} [Sat Sep 17 21:12:38 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-358737642 => generated 415 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:38.421226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.421226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.421226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.444543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:38.446298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:38.541148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae095-63c9-4859-b498-6eabdea0dc1a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.542340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2747/5501] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3740 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:38.545023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.551580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:38.556203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.557272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.557859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.558436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b348ff808766462fbeb7304f6c5aae6d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.563287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=b348ff808766462fbeb7304f6c5aae6d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.570288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.579465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de904db-48f5-4bd5-a27b-950fd8e172c0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.585436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role b348ff808766462fbeb7304f6c5aae6d was removed from user faf54362aa9a4047bcb2c067f7b9b02e on project 963e5b531cf64e41b26bb29d69e0a735. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:38.586288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:38.588149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:38.594444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df124e53-dbc3-4e99-8c62-387d6e15ca65 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.595425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df124e53-dbc3-4e99-8c62-387d6e15ca65 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.595714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df124e53-dbc3-4e99-8c62-387d6e15ca65 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.595990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df124e53-dbc3-4e99-8c62-387d6e15ca65 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.596838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df124e53-dbc3-4e99-8c62-387d6e15ca65 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-678723027) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:38.597253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df124e53-dbc3-4e99-8c62-387d6e15ca65 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.599109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df124e53-dbc3-4e99-8c62-387d6e15ca65 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.610606 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2748/5502] 10.222.0.26 () {58 vars in 1220 bytes} [Sat Sep 17 21:12:38 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-678723027 => generated 416 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:38.611990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.612289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.612873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.627385 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2755/5503] 10.222.0.26 () {58 vars in 1467 bytes} [Sat Sep 17 21:12:38 2022] DELETE /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/b348ff808766462fbeb7304f6c5aae6d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:38.631324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.633717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:38.648433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:38.649809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:38.727700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465aca63-b111-4250-9def-c15eddf789ef tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.739040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.741236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.742595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.743757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/045316cf2848480b9431010646ae0c74` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.746458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:delete_service(service_id=045316cf2848480b9431010646ae0c74)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.749952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.781847 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2749/5504] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3740 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:38.783977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.787527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:38.807676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2756/5505] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:38 2022] DELETE /identity/v3/services/045316cf2848480b9431010646ae0c74 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:38.808627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.871469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1ed72-1fcc-446a-866c-62fed2fc5e11 tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.884895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:38.889434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.890070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.890592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.890760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.891437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:38.891740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.895989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.916866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:38.932350 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2757/5506] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:38.934430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:38.938463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72233d-433b-45a8-888b-bebb0964a26a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.968212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.969035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.970259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:38.970259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:38.970811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:38.974629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:38.985356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979944d8-1e70-4300-a805-7f9a7ad0977f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:38.998504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:38.999524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:38.999799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.000071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/0de8e58a1262429c8d1753c54930456a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.000663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:delete_service(service_id=0de8e58a1262429c8d1753c54930456a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.002751 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2750/5507] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:38 2022] POST /identity/v3/roles => generated 272 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:39.004076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.005158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.031789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a0ca-5b82-4173-9da5-f603f2d18c64 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.043201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.043893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.044265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.044764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.047672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.049037 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2758/5508] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:38 2022] DELETE /identity/v3/services/0de8e58a1262429c8d1753c54930456a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:39.049923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.052477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.085783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9375-044e-4fa6-b18b-a70a655fb6bf tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.086466 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2751/5509] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:39 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:39.087630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.188804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.189440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.189758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.190331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.191093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Adding query filter params (domain_id=a1cadb4bc7f643c396c8d84d79be82ff) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:39.191455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.192665 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:39.192665 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:39.192665 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:39.193521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.196685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d01f8e-c1aa-4511-99da-f56b1b0909ac admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.213543 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2759/5510] 10.222.0.26 () {58 vars in 1207 bytes} [Sat Sep 17 21:12:39 2022] GET /identity/v3/roles?domain_id=a1cadb4bc7f643c396c8d84d79be82ff => generated 409 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:39.214739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.232755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d35db83e16646f5a34a0af723ce671a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05f135241b5243a195fa3cf603041d25', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.233684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.233875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.234149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] PATH_INFO: `/v3/services/f1e36e810cff46829031d328f4980fcf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.234742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorizing `identity:delete_service(service_id=f1e36e810cff46829031d328f4980fcf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.236444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.294011 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2752/5511] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:39 2022] DELETE /identity/v3/services/f1e36e810cff46829031d328f4980fcf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:39.296900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.332286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3497b3-2db9-462b-b761-2fc06b5d211c tempest-ServicesTestJSON-1013441303 tempest-ServicesTestJSON-1013441303-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.337549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.338409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.338699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.338974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/3d1f20c5886a452b9e668fd1cb14ce98` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.340171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ad3b4-ebf7-4af7-a9b8-af87dd54203c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.340579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=3d1f20c5886a452b9e668fd1cb14ce98)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.351045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.367865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.368741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.369247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.369657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.370552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.373310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.407210 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2760/5512] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:39 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:39.408133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.409898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:39.429348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724715-1ede-426a-83a8-af5656379636 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.436372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.439087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.439401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.439752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] PATH_INFO: `/v3/users/68181fb50df646f88dac4f4d2e4f29b3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.440206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role 3d1f20c5886a452b9e668fd1cb14ce98 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:39.440670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:39.443235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68181fb50df646f88dac4f4d2e4f29b3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.445893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.450475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:39.458295 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2753/5513] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:39 2022] DELETE /identity/v3/roles/3d1f20c5886a452b9e668fd1cb14ce98 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:39.459976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.465069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:39.484459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:39.543667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:39.559519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68181fb50df646f88dac4f4d2e4f29b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:39.596874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68181fb50df646f88dac4f4d2e4f29b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:39.613222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68181fb50df646f88dac4f4d2e4f29b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:39.614337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68181fb50df646f88dac4f4d2e4f29b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:39.615126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68181fb50df646f88dac4f4d2e4f29b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:39.638698 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2761/5514] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:39 2022] DELETE /identity/v3/users/68181fb50df646f88dac4f4d2e4f29b3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:39.639307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b809d-97bd-4531-bb04-5112b59cd284 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.640589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.643156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:39.758045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ed708-17fc-4fdc-bf4d-a4777894b631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.758602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ed708-17fc-4fdc-bf4d-a4777894b631 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.758890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ed708-17fc-4fdc-bf4d-a4777894b631 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.759161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ed708-17fc-4fdc-bf4d-a4777894b631 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.759821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ed708-17fc-4fdc-bf4d-a4777894b631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.776498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ed708-17fc-4fdc-bf4d-a4777894b631 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.798564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2754/5515] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:39 2022] GET /identity/v3/roles => generated 2322 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:39.798564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fba95-d215-4380-91d6-1e7a36714182 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.798564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fba95-d215-4380-91d6-1e7a36714182 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.798564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fba95-d215-4380-91d6-1e7a36714182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.806159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f491286-4206-4bd1-8f0f-36b08c16ad14 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.812979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.814048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.814564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.815646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.818278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Adding query filter params (domain_id=a1cadb4bc7f643c396c8d84d79be82ff) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:39.818937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.822748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.836721 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2762/5516] 10.222.0.26 () {58 vars in 1207 bytes} [Sat Sep 17 21:12:39 2022] GET /identity/v3/roles?domain_id=a1cadb4bc7f643c396c8d84d79be82ff => generated 148 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:39.837524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.839769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:39.854326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-463fba95-d215-4380-91d6-1e7a36714182 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:39.856138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-463fba95-d215-4380-91d6-1e7a36714182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:39.869685 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2755/5517] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:39.870780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fba95-d215-4380-91d6-1e7a36714182 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.883943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fba95-d215-4380-91d6-1e7a36714182 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.889012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.889831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7a3b8-db17-43d3-955c-c93aa4114c84 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:39.890228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.890609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.907085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/3d1f20c5886a452b9e668fd1cb14ce98` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.917904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:39.922592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:39.922872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:39.923286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:39.923421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:12:39.925894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Could not find role: 3d1f20c5886a452b9e668fd1cb14ce98.: keystone.exception.RoleNotFound: Could not find role: 3d1f20c5886a452b9e668fd1cb14ce98. Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Sep 17 21:12:39.932411 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:12:39.933799 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 3d1f20c5886a452b9e668fd1cb14ce98. Sep 17 21:12:39.935062 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:39.936020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:39.937315 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2756/5518] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:39 2022] DELETE /identity/v3/roles/3d1f20c5886a452b9e668fd1cb14ce98 => generated 110 bytes in 67 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:39.938521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:39.942613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:39.984951 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2763/5519] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:39 2022] GET /identity/v3/roles => generated 2322 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:39.986715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.014679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1949930-6c09-43e5-9665-4afbd3a0aa4a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.023936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.025616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.025616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.027565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.038878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.042701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.084302 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2764/5520] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:39 2022] PUT /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:40.085431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.130433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45577dae-88d6-4734-90dc-0de975ecc102 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.216671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5e590-252f-41fa-893a-fc929a7b33dd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.222894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeafd7-9898-4c62-8e92-99f2877ab7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.223747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeafd7-9898-4c62-8e92-99f2877ab7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.224255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeafd7-9898-4c62-8e92-99f2877ab7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.224687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeafd7-9898-4c62-8e92-99f2877ab7fe admin admin] PATH_INFO: `/v3/projects/7a2458735038406092c81b60273d5f52/users/5b51dbc1bdc54503bb7371e7491fa43d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.237488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.238149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.238426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.238703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.245524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:40.246999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:40.252577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.254750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeafd7-9898-4c62-8e92-99f2877ab7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a2458735038406092c81b60273d5f52, user_id=5b51dbc1bdc54503bb7371e7491fa43d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.255200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.257914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeafd7-9898-4c62-8e92-99f2877ab7fe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.274133 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2757/5521] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:40.275189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.290910 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2765/5522] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:40 2022] PUT /identity/v3/projects/7a2458735038406092c81b60273d5f52/users/5b51dbc1bdc54503bb7371e7491fa43d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:40.302930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.307074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.307409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.380899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] MFA Rules not processed for user `5b51dbc1bdc54503bb7371e7491fa43d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:40.383218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:40.475246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e3be-5c38-4d43-836e-bd4142df3a19 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.568011 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2766/5523] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:40.581881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.583725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:40.625494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.626362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.626897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.627180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.629289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, group_id=304008188b134566b4b04d1e26c07946)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.631994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.672585 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2758/5524] 10.222.0.26 () {58 vars in 1365 bytes} [Sat Sep 17 21:12:40 2022] GET /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles => generated 410 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:40.678340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.695161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ae6d-1fa5-4254-a5b2-322ab050babd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.699745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.700433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.700720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.700990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.711556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.713164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae15362-72bc-482b-9771-80496080aa58 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.713369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.724328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.725071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.725525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.725898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.726513 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2767/5525] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:40.728599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.733246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.745964 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2759/5526] 10.222.0.26 () {58 vars in 1465 bytes} [Sat Sep 17 21:12:40 2022] HEAD /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:40.751873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.766779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a39d72-6492-4c14-81d8-ff37b93b8f07 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.774682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.775541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.775970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.776743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.780014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.784879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.792681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role bfedd871715c4f25a287d0c531c724f3 was removed from group 304008188b134566b4b04d1e26c07946 on domain a1cadb4bc7f643c396c8d84d79be82ff. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:40.793125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:40.794304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:40.837498 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2768/5527] 10.222.0.26 () {58 vars in 1467 bytes} [Sat Sep 17 21:12:40 2022] DELETE /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:40.839329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.843142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:40.861981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75be81-c237-453e-8075-4bd584a996c8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:40.949695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:40.950722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:40.950722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:40.951135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:40.953855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:40.956951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:40.972698 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2760/5528] 10.222.0.26 () {60 vars in 1486 bytes} [Sat Sep 17 21:12:40 2022] PUT /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:40.980098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.980341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:40.982190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:40.993182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d3ba-96e0-4ca1-a410-fed5ea63f419 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:41.011624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:41.095031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:41.095031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.095031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.095827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.098914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_grants(project_id=963e5b531cf64e41b26bb29d69e0a735, group_id=304008188b134566b4b04d1e26c07946)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:41.103333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:41.112423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:41.133282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb6b4d-b932-4f40-9707-3195ccdb2712 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:41.139150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:41.139919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.140358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.140731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] PATH_INFO: `/v3/projects/a7158dd65f8541798e8fbcaa62aa2725` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.142939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7158dd65f8541798e8fbcaa62aa2725)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:41.147480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2769/5529] 10.222.0.26 () {58 vars in 1368 bytes} [Sat Sep 17 21:12:40 2022] GET /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:41.150117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:41.151654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:41.178621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d024164-13e0-4805-b375-709cdfc5d8cb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:41.191627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70cf28-f6b4-4546-b67c-5f798f0801c9 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:41.193501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70cf28-f6b4-4546-b67c-5f798f0801c9 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.193848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70cf28-f6b4-4546-b67c-5f798f0801c9 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.194181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70cf28-f6b4-4546-b67c-5f798f0801c9 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/groups/304008188b134566b4b04d1e26c07946/users/faf54362aa9a4047bcb2c067f7b9b02e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.196620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70cf28-f6b4-4546-b67c-5f798f0801c9 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=304008188b134566b4b04d1e26c07946, user_id=faf54362aa9a4047bcb2c067f7b9b02e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:41.199031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70cf28-f6b4-4546-b67c-5f798f0801c9 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:41.266626 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2770/5530] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:12:41 2022] PUT /identity/v3/groups/304008188b134566b4b04d1e26c07946/users/faf54362aa9a4047bcb2c067f7b9b02e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:41.270138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.270595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.270986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.293256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7158dd65f8541798e8fbcaa62aa2725', 'request_id': 'req-e012dc2f-41be-49ce-a9bb-b1897a8f1709'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.307960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7158dd65f8541798e8fbcaa62aa2725', 'request_id': 'req-e012dc2f-41be-49ce-a9bb-b1897a8f1709'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.336843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7158dd65f8541798e8fbcaa62aa2725', 'request_id': 'req-e012dc2f-41be-49ce-a9bb-b1897a8f1709'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.342647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] MFA Rules not processed for user `faf54362aa9a4047bcb2c067f7b9b02e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:41.346952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:41.351154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] The token cache is being invalidate because project a7158dd65f8541798e8fbcaa62aa2725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:41.351718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.352466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.354233 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2761/5531] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:40 2022] DELETE /identity/v3/projects/a7158dd65f8541798e8fbcaa62aa2725 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:41.358604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:41.360277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:41.464432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012dc2f-41be-49ce-a9bb-b1897a8f1709 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:41.472208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:41.472868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.473160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.473415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] PATH_INFO: `/v3/users/2d35db83e16646f5a34a0af723ce671a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.474740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d35db83e16646f5a34a0af723ce671a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:41.476759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:41.525157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2771/5532] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:41.529400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:41.531937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:41.545691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d35db83e16646f5a34a0af723ce671a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.546513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d35db83e16646f5a34a0af723ce671a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.550198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:41.561339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d35db83e16646f5a34a0af723ce671a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.563118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d35db83e16646f5a34a0af723ce671a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.599653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d35db83e16646f5a34a0af723ce671a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.621779 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2762/5533] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:41 2022] DELETE /identity/v3/users/2d35db83e16646f5a34a0af723ce671a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:41.624858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80646cae-d204-48bd-be65-35f78b4b10a3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:41.687726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:41.715668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:41.716378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.716648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.717713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.719886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:check_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:41.723375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:41.731339 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2772/5534] 10.222.0.26 () {58 vars in 1468 bytes} [Sat Sep 17 21:12:41 2022] HEAD /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:41.735622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:41.737161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:41.751358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98e8fb-b487-4fb1-90f0-db3a76dbc243 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:41.760216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:41.761495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.761495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.761738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.764127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:41.769201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:41.775486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role bfedd871715c4f25a287d0c531c724f3 was removed from group 304008188b134566b4b04d1e26c07946 on project 963e5b531cf64e41b26bb29d69e0a735. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:41.775786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.776643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:41.820220 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2763/5535] 10.222.0.26 () {58 vars in 1470 bytes} [Sat Sep 17 21:12:41 2022] DELETE /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:41.825132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:41.827354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:41.842293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8611ded-d7e2-47e8-a9b9-bf9b4489a089 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:41.980582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:41.981400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:41.981861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:41.982234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/groups/304008188b134566b4b04d1e26c07946/users/faf54362aa9a4047bcb2c067f7b9b02e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:41.984235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=304008188b134566b4b04d1e26c07946, user_id=faf54362aa9a4047bcb2c067f7b9b02e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:41.987719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:41.999719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'faf54362aa9a4047bcb2c067f7b9b02e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.029997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2773/5536] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:12:41 2022] DELETE /identity/v3/groups/304008188b134566b4b04d1e26c07946/users/faf54362aa9a4047bcb2c067f7b9b02e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.035595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.055280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e53bf8-dcc9-4a1e-8d4a-8579095506e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.128396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.128958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.129238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.129546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.131296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.132352 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:42.132352 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:42.132352 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:42.133247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.144101 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2764/5537] 10.222.0.26 () {60 vars in 1381 bytes} [Sat Sep 17 21:12:42 2022] PUT /identity/v3/system/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.148759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.164732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68124f-21ee-4113-8515-e03d60a4bce1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.181703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.181703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.181703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.182147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/groups/304008188b134566b4b04d1e26c07946/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.182147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=304008188b134566b4b04d1e26c07946)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.182147 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:42.182147 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:42.182147 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:42.199384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.217880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2774/5538] 10.222.0.26 () {58 vars in 1263 bytes} [Sat Sep 17 21:12:42 2022] GET /identity/v3/system/groups/304008188b134566b4b04d1e26c07946/roles => generated 376 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:42.221789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.238727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c374ab2-f152-494f-aa6f-88cca25d1710 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.245355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.246523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.247304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.247304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.250573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.251702 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:42.251702 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:42.251702 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:42.252945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.264184 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2765/5539] 10.222.0.26 () {58 vars in 1363 bytes} [Sat Sep 17 21:12:42 2022] HEAD /identity/v3/system/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.265253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.265387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.267466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:42.282230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fb1ea-77db-4a23-85dc-3c6e37078f1a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.289227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.291407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.291782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.292067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.295104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=304008188b134566b4b04d1e26c07946, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.301816 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:42.301816 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:42.301816 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:42.302815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.321110 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2766/5540] 10.222.0.26 () {58 vars in 1365 bytes} [Sat Sep 17 21:12:42 2022] DELETE /identity/v3/system/groups/304008188b134566b4b04d1e26c07946/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.326621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.347824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751d997-ab7f-4d19-ab88-0e4ce50e4821 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.362152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.362838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.363121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.363398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.371233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.374668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.413134 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2767/5541] 10.222.0.26 () {60 vars in 1480 bytes} [Sat Sep 17 21:12:42 2022] PUT /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.418166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.499535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eb2e8-3075-4701-9284-625df4f23e88 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.548164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be75e23-4d47-4aa6-beca-af5e27335b08 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.560656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.561333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.561643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.565576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.569147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, user_id=faf54362aa9a4047bcb2c067f7b9b02e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.572522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.594163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:42.599620 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2768/5542] 10.222.0.26 () {58 vars in 1362 bytes} [Sat Sep 17 21:12:42 2022] GET /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles => generated 409 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:42.603475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.615183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.616272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.616967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.617482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] PATH_INFO: `/v3/projects/05f135241b5243a195fa3cf603041d25` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.619258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f135241b5243a195fa3cf603041d25)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.620634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb8c86-adc2-4c80-94b5-ad7ed34c2a23 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.621922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.627304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.628289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.628643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.628924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.632361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.635709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.640235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:42.644679 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2769/5543] 10.222.0.26 () {58 vars in 1462 bytes} [Sat Sep 17 21:12:42 2022] HEAD /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:42.650188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.679249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc7af-2046-4ff7-a425-c4cf17f5d0cf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.695267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.696037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.696642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.697514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.700407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a1cadb4bc7f643c396c8d84d79be82ff, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.703266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.711669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role bfedd871715c4f25a287d0c531c724f3 was removed from user faf54362aa9a4047bcb2c067f7b9b02e on domain a1cadb4bc7f643c396c8d84d79be82ff. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:42.711997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.712693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.724843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f135241b5243a195fa3cf603041d25', 'request_id': 'req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.741362 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2770/5544] 10.222.0.26 () {58 vars in 1464 bytes} [Sat Sep 17 21:12:42 2022] DELETE /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.743113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.745350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:42.747961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f135241b5243a195fa3cf603041d25', 'request_id': 'req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.759506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f135241b5243a195fa3cf603041d25', 'request_id': 'req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.775898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] The token cache is being invalidate because project 05f135241b5243a195fa3cf603041d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:42.776374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.777599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-10648a2f-9987-4d22-ad8c-dc98dd07b3a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:42.784355 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2775/5545] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:42 2022] DELETE /identity/v3/projects/05f135241b5243a195fa3cf603041d25 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.787178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf20e9-e4e3-4d67-9f80-df6c066a0baf None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.787178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf20e9-e4e3-4d67-9f80-df6c066a0baf None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.787178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf20e9-e4e3-4d67-9f80-df6c066a0baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.813273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b1bf20e9-e4e3-4d67-9f80-df6c066a0baf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:42.814961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b1bf20e9-e4e3-4d67-9f80-df6c066a0baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:42.901046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfa467-db84-4675-a56e-5241f8e55e3c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:42.907431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:42.908142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.908465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.908683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:42.912226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:42.915691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:42.936295 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2771/5546] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:42 2022] PUT /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:42.944609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:42.950577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:42.965727 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2776/5547] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:42.977964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce7d9b-0264-457d-84e3-afefbe93cfa0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:42.979215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce7d9b-0264-457d-84e3-afefbe93cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:42.979587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce7d9b-0264-457d-84e3-afefbe93cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.022696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c8ce7d9b-0264-457d-84e3-afefbe93cfa0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:43.024589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c8ce7d9b-0264-457d-84e3-afefbe93cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:43.061179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d8540-0d1c-49ef-9521-13799bc7a88a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.068606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.069282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.069597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.069875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.072367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_grants(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.076111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.096040 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2772/5548] 10.222.0.26 () {58 vars in 1365 bytes} [Sat Sep 17 21:12:42 2022] GET /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles => generated 410 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:43.107979 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2777/5549] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:43.109313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.111851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce7d9b-0264-457d-84e3-afefbe93cfa0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.124129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce7d9b-0264-457d-84e3-afefbe93cfa0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.130799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b75e0-3c5d-4d6c-88bc-252a4bcc3224 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.131847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.132775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.133271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.133889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.134755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:43.135739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.137777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.138463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.138593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.138927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.139283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.144959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:check_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.148230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.156197 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2773/5550] 10.222.0.26 () {58 vars in 1465 bytes} [Sat Sep 17 21:12:43 2022] HEAD /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:43.159890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.172756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a102e2-92d4-4298-b6fd-f40c18d4d62c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.180229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.181248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.181590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.182114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.182496 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2778/5551] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:43.186780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.190375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=963e5b531cf64e41b26bb29d69e0a735, user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.193926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.204374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role bfedd871715c4f25a287d0c531c724f3 was removed from user faf54362aa9a4047bcb2c067f7b9b02e on project 963e5b531cf64e41b26bb29d69e0a735. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:43.205084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:43.206212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:43.213922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc0f69-31e2-4531-9cdc-69ffcf1a8d5e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.226933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.227556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.227972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.228388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.229025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.231047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.231693 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2774/5552] 10.222.0.26 () {58 vars in 1467 bytes} [Sat Sep 17 21:12:43 2022] DELETE /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:43.236468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.238529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:43.257895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:43.298345 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2779/5553] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:43 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:43.302616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.304577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:43.308347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:43.358202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2f42a-2a59-41e8-baa4-dbd1353f1ccc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.458021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55205af3-6d0f-4f88-bbb5-b4ef91243f64 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.468778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.469700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.469833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.470096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.470965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.473138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.495590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.497947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.499049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.500622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.503969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.505780 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:43.505780 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:43.505780 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:43.509512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.509676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2780/5554] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:43 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:43.513968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.526109 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2775/5555] 10.222.0.26 () {60 vars in 1378 bytes} [Sat Sep 17 21:12:43 2022] PUT /identity/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:43.530279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f2dc-9926-46d6-a172-f0bebddf9ad8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.530453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.535732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.536408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.536695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.537086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.537785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.539752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.547254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97024c-e9ff-4979-8cf9-72dc43369a95 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.557403 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2781/5556] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:43 2022] GET /identity/v3/roles => generated 2322 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:43.558508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.565011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.565011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.565011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.566200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.568763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=faf54362aa9a4047bcb2c067f7b9b02e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.569286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a3bc0-c5d1-4ae2-a006-767d8db3e525 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.569830 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:43.569830 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:43.569830 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:43.572970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.574845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.575588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.576324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.576686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.577622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.579472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.596118 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2776/5557] 10.222.0.26 () {58 vars in 1260 bytes} [Sat Sep 17 21:12:43 2022] GET /identity/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles => generated 375 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:43.597744 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2782/5558] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:43 2022] GET /identity/v3/roles => generated 2322 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:43.599689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.600805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.611553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d24de-c563-4198-bc05-fac6a7e6a581 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.614140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70129fb-5b84-45e9-b68b-347f2cecb1ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.617721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59e23c-6110-482c-b409-6104c7f47b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.618636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e23c-6110-482c-b409-6104c7f47b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.619072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e23c-6110-482c-b409-6104c7f47b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.619523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e23c-6110-482c-b409-6104c7f47b2c admin admin] PATH_INFO: `/v3/projects/f3bc2a51f0f04ce2bdcefb403244a912/users/12de706808bb460e892dfeff76d60313/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.620882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.624432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.624432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.624432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.624432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.625507 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:43.625507 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:43.625507 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:43.627137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.636293 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2777/5559] 10.222.0.26 () {58 vars in 1360 bytes} [Sat Sep 17 21:12:43 2022] HEAD /identity/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:43.640523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.646312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59e23c-6110-482c-b409-6104c7f47b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bc2a51f0f04ce2bdcefb403244a912, user_id=12de706808bb460e892dfeff76d60313, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.650435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59e23c-6110-482c-b409-6104c7f47b2c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.654457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac4f1d-80ab-449b-8f86-b02b941bb955 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.663583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.664431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.664819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.665754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.670605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=faf54362aa9a4047bcb2c067f7b9b02e, role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.672757 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:43.672757 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:43.672757 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:43.674377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.677551 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2783/5560] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:43 2022] PUT /identity/v3/projects/f3bc2a51f0f04ce2bdcefb403244a912/users/12de706808bb460e892dfeff76d60313/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:43.687551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2778/5561] 10.222.0.26 () {58 vars in 1362 bytes} [Sat Sep 17 21:12:43 2022] DELETE /identity/v3/system/users/faf54362aa9a4047bcb2c067f7b9b02e/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:43.688912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219818-8792-44fc-9090-f0949a87198b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.689395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219818-8792-44fc-9090-f0949a87198b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.689721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219818-8792-44fc-9090-f0949a87198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.692514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.712800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac60640-45f5-4fec-b235-58203bdd6192 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.732824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e4219818-8792-44fc-9090-f0949a87198b None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:43.735560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e4219818-8792-44fc-9090-f0949a87198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:43.817862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.818650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.819035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.819391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.820129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.822261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.848944 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2779/5562] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:43 2022] POST /identity/v3/roles => generated 268 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:43.850453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.858794 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2784/5563] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:12:43.860298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4219818-8792-44fc-9090-f0949a87198b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.865649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d80d4-698c-4e2c-9801-ee28cdc55111 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.872984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4219818-8792-44fc-9090-f0949a87198b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.874680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.875605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.875956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.876573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.878150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.880485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.899861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e390dce-e5cd-4fc6-9dbf-f8d35b53387d tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.901664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e390dce-e5cd-4fc6-9dbf-f8d35b53387d tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.902075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e390dce-e5cd-4fc6-9dbf-f8d35b53387d tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.902426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e390dce-e5cd-4fc6-9dbf-f8d35b53387d tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.910933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2780/5564] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:43 2022] POST /identity/v3/roles => generated 270 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:43.927979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:43.945454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24272e-c2b3-40f1-b8aa-75a9db7ffdbf tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:43.972892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:43.973894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:43.974710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:43.975011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:43.975299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/12261eed7f554c828898cfc6f1455f8b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:43.981102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=4b473f4deedd481394bb7dbd5a897f59, implied_role_id=12261eed7f554c828898cfc6f1455f8b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:43.982239 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:43.982239 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:43.982239 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:43.983285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:43.989905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0e390dce-e5cd-4fc6-9dbf-f8d35b53387d tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:44.004103 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2785/5565] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:12:43 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:44.016706 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2781/5566] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:43 2022] PUT /identity/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/12261eed7f554c828898cfc6f1455f8b => generated 529 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:44.020617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e390dce-e5cd-4fc6-9dbf-f8d35b53387d tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.036341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e390dce-e5cd-4fc6-9dbf-f8d35b53387d tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.114717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.116744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.117039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.117313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.118034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.119661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.144987 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2786/5567] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:12:44 2022] POST /identity/v3/domains => generated 276 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:44.146284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.158306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a9070-95f1-478d-acd7-b804926131a2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.163650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:44.228069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.229211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.229525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.229870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.230633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.232565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.256575 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2782/5568] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:44 2022] POST /identity/v3/roles => generated 269 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:44.260277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.271992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9cf97-0670-4da3-92a2-b0923b8606f4 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.276812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.277778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.278064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.278272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/24fb5ffa088644798b92044bce924f03` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.280213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=4b473f4deedd481394bb7dbd5a897f59, implied_role_id=24fb5ffa088644798b92044bce924f03)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.281859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.311481 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2787/5569] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:44 2022] PUT /identity/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/24fb5ffa088644798b92044bce924f03 => generated 528 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:44.325931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.339692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390e9b-577d-4871-9c13-d454b7538fba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.419513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.427711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.428390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.429209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.432269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=4b473f4deedd481394bb7dbd5a897f59, implied_role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.435393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.486464 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2783/5570] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:44 2022] PUT /identity/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/bfedd871715c4f25a287d0c531c724f3 => generated 524 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:44.488083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.505942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01578d7b-a297-4ebe-8b0c-9e746fb4bf2a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.628290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.628975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.629259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.629602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/bfedd871715c4f25a287d0c531c724f3/implies/4b473f4deedd481394bb7dbd5a897f59` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.633823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=bfedd871715c4f25a287d0c531c724f3, implied_role_id=4b473f4deedd481394bb7dbd5a897f59)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.635358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] 4b473f4deedd481394bb7dbd5a897f59 cannot be an implied roles.: keystone.exception.InvalidImpliedRole: 4b473f4deedd481394bb7dbd5a897f59 cannot be an implied roles. Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Sep 17 21:12:44.659124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Sep 17 21:12:44.660131 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:44.660131 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:44.660131 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Sep 17 21:12:44.660131 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Sep 17 21:12:44.660131 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: 4b473f4deedd481394bb7dbd5a897f59 cannot be an implied roles. Sep 17 21:12:44.660131 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:12:44.661380 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2788/5571] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:44 2022] PUT /identity/v3/roles/bfedd871715c4f25a287d0c531c724f3/implies/4b473f4deedd481394bb7dbd5a897f59 => generated 116 bytes in 173 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:12:44.665812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.684624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c9d74-9d7b-4375-80da-41f5fe2b8e37 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.690619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.691291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.691571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.691845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.693552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=4b473f4deedd481394bb7dbd5a897f59, implied_role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.695232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.709865 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2784/5572] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:44 2022] DELETE /identity/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:44.711626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.722789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf1e0e-a66b-4da2-8c32-090536493a7c tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.776649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:44.797645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.798365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.798657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.798934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/24fb5ffa088644798b92044bce924f03` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.800639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=4b473f4deedd481394bb7dbd5a897f59, implied_role_id=24fb5ffa088644798b92044bce924f03)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.801726 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:44.801726 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:44.801726 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:44.802823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.815107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2789/5573] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:44 2022] DELETE /identity/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/24fb5ffa088644798b92044bce924f03 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:44.819507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.831101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f840a-fa58-4e9c-b919-d81aa1978734 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:44.907709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:44.908363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:44.908641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:44.908930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/24fb5ffa088644798b92044bce924f03` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:44.910307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=24fb5ffa088644798b92044bce924f03)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:44.912123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:44.925540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:44.960380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role 24fb5ffa088644798b92044bce924f03 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:44.960961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:44.962420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a9577ce-91fb-4f2d-bec5-aeffde45848d tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:44.969767 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2785/5574] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:44 2022] DELETE /identity/v3/roles/24fb5ffa088644798b92044bce924f03 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:44.973050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:44.975457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:45.009827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6462cd-b582-46b4-8047-e48488f129cf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.010226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6462cd-b582-46b4-8047-e48488f129cf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.010717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6462cd-b582-46b4-8047-e48488f129cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.024604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b100-5131-4099-a997-e4954e7b230f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:45.055713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ec6462cd-b582-46b4-8047-e48488f129cf None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:45.056765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ec6462cd-b582-46b4-8047-e48488f129cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:45.061292 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2786/5575] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:45 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:12:45.068244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d3b3-684e-4299-94a9-80f70f5594c6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.068629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d3b3-684e-4299-94a9-80f70f5594c6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.068990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4d3b3-684e-4299-94a9-80f70f5594c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.094718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9ea4d3b3-684e-4299-94a9-80f70f5594c6 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:45.096352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9ea4d3b3-684e-4299-94a9-80f70f5594c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:45.143628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:45.144872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.145420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.146183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/41753e96bd994ae88568c5a5f3af4f3d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.163230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=41753e96bd994ae88568c5a5f3af4f3d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:45.165183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:45.180902 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2787/5576] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:45.184930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d3b3-684e-4299-94a9-80f70f5594c6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:45.195855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4d3b3-684e-4299-94a9-80f70f5594c6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:45.197604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.198676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.202433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:45.203979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.204540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.205059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.246011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.246852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.255974 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2790/5577] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:12:44 2022] PATCH /identity/v3/domains/41753e96bd994ae88568c5a5f3af4f3d => generated 277 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:45.257253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:45.260498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:45.285950 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 17 21:12:45.287197 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:45.290511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2788/5578] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:12:45 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 188 bytes in 106 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 17 21:12:45.294471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:45.297301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:45.314007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4abac-3650-4b5d-9c41-e7d188f84bcd tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:45.345226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000873a3-4a6b-4173-ba9b-63aa00da3f5e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:45.350427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:45.351470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.351828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.352177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/41753e96bd994ae88568c5a5f3af4f3d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.367225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=41753e96bd994ae88568c5a5f3af4f3d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:45.368996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:45.397579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.406951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dd545-3a11-4c84-894a-1a1ae377e7a0 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:45.407947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dd545-3a11-4c84-894a-1a1ae377e7a0 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.408359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dd545-3a11-4c84-894a-1a1ae377e7a0 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.408704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dd545-3a11-4c84-894a-1a1ae377e7a0 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.410568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.413154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-248dd545-3a11-4c84-894a-1a1ae377e7a0 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:45.464126 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 17 21:12:45.465330 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:45.467789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d', 'request_id': 'req-e7773e65-ba0d-407b-b001-e658acc5d1ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.468329 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2789/5579] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:12:45 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 188 bytes in 174 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 17 21:12:45.473521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dd545-3a11-4c84-894a-1a1ae377e7a0 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:45.485292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d', 'request_id': 'req-e7773e65-ba0d-407b-b001-e658acc5d1ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.496459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dd545-3a11-4c84-894a-1a1ae377e7a0 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:45.501882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d', 'request_id': 'req-e7773e65-ba0d-407b-b001-e658acc5d1ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.525498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] The token cache is being invalidate because project 41753e96bd994ae88568c5a5f3af4f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:45.526807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.527893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.551470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d', 'request_id': 'req-e7773e65-ba0d-407b-b001-e658acc5d1ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.556138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d', 'request_id': 'req-e7773e65-ba0d-407b-b001-e658acc5d1ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.558804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '41753e96bd994ae88568c5a5f3af4f3d', 'request_id': 'req-e7773e65-ba0d-407b-b001-e658acc5d1ac'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.571900 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2791/5580] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:12:45 2022] DELETE /identity/v3/domains/41753e96bd994ae88568c5a5f3af4f3d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:45.577663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:45.579909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:45.610205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:45.648652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb46c3-3727-4df5-bea6-83986fa9799b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:45.649322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb46c3-3727-4df5-bea6-83986fa9799b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.649634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb46c3-3727-4df5-bea6-83986fa9799b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.649908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb46c3-3727-4df5-bea6-83986fa9799b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.733050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7773e65-ba0d-407b-b001-e658acc5d1ac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:45.738169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:45.738840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.739125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.739408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/12261eed7f554c828898cfc6f1455f8b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.746402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-81cb46c3-3727-4df5-bea6-83986fa9799b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:45.753421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=4b473f4deedd481394bb7dbd5a897f59, implied_role_id=12261eed7f554c828898cfc6f1455f8b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:45.755412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:45.767737 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2790/5581] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:12:45 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:45.774794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb46c3-3727-4df5-bea6-83986fa9799b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:45.775283 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2792/5582] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:45 2022] DELETE /identity/v3/roles/4b473f4deedd481394bb7dbd5a897f59/implies/12261eed7f554c828898cfc6f1455f8b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:45.811961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb46c3-3727-4df5-bea6-83986fa9799b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:45.941303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:45.942046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:45.942336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:45.943202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/12261eed7f554c828898cfc6f1455f8b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:45.945031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=12261eed7f554c828898cfc6f1455f8b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:45.946350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.018337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role 12261eed7f554c828898cfc6f1455f8b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:46.018734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:46.020593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:46.028463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:46.031139 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2791/5583] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:45 2022] DELETE /identity/v3/roles/12261eed7f554c828898cfc6f1455f8b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:46.031679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:46.047554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5ef8f-940d-47ba-9137-06666717a357 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:46.132213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:46.133076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.133676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.134051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/4b473f4deedd481394bb7dbd5a897f59` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.135651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=4b473f4deedd481394bb7dbd5a897f59)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.136953 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:46.136953 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:46.136953 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:46.138339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.159187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:46.196858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role 4b473f4deedd481394bb7dbd5a897f59 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:46.197413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:46.201771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:46.209694 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2793/5584] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:46 2022] DELETE /identity/v3/roles/4b473f4deedd481394bb7dbd5a897f59 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:46.212780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:46.225233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8e7c6-4003-43d2-8718-5a41a3c3c591 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:46.333448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:46.334542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.335018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.335801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.338618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.341370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.385868 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2792/5585] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:46 2022] PUT /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 520 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:46.387995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:46.389620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:46.415298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf89ec7-fd8e-4517-b908-9f5ee3064dc0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:46.505829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:46.506662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.506874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.507116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.509320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.510721 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:46.510721 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:46.510721 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:46.511978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.513034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.514004 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:46.514004 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:46.514004 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:46.515200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.526398 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2794/5586] 10.222.0.26 () {58 vars in 1345 bytes} [Sat Sep 17 21:12:46 2022] HEAD /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:46.528247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:46.547685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af663fdf-2c99-43dc-968d-158e4ca664e6 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:46.562980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:46.564092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.566021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.566647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.571732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.574572 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:46.574572 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:46.574572 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:46.577008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.581345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:46.592777 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2793/5587] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:12:46 2022] GET /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 520 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:12:46.594463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:46.621708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa8800-8d6c-48c0-862f-b62cebc25633 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:46.639455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb592e8a-1070-4f83-97d9-f7cb04f6ded7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:46.640569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb592e8a-1070-4f83-97d9-f7cb04f6ded7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.641268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb592e8a-1070-4f83-97d9-f7cb04f6ded7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.641762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb592e8a-1070-4f83-97d9-f7cb04f6ded7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.643990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb592e8a-1070-4f83-97d9-f7cb04f6ded7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.646529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb592e8a-1070-4f83-97d9-f7cb04f6ded7 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.668671 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2795/5588] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:46 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:46.673058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:46.700853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18ec1d-1ae0-4ad2-963d-f11492234dfa tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:46.772646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a303cc-8523-4def-a32a-5c90ac01e733 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.773202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a303cc-8523-4def-a32a-5c90ac01e733 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.774686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a303cc-8523-4def-a32a-5c90ac01e733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.818037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:46.818440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-59a303cc-8523-4def-a32a-5c90ac01e733 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:46.819325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.819718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-59a303cc-8523-4def-a32a-5c90ac01e733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:46.820683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.821344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.823818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.830952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2796/5589] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:46 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 21:12:46.832006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da35024-f559-4ab6-a8aa-a5377ed423b6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.832297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da35024-f559-4ab6-a8aa-a5377ed423b6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.832603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.835356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da35024-f559-4ab6-a8aa-a5377ed423b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27.: keystone.exception.ImpliedRoleNotFound: b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27. Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = query.one() Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._iter().one() Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._only_one_row( Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:46.849232 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 17 21:12:46.850659 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 17 21:12:46.851728 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 17 21:12:46.851728 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27. Sep 17 21:12:46.851728 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:46.856117 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2794/5590] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:12:46 2022] GET /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 137 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:12:46.860992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:46.882401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7da35024-f559-4ab6-a8aa-a5377ed423b6 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:46.883830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7da35024-f559-4ab6-a8aa-a5377ed423b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:46.913276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7921f5f-ec1f-423c-abf4-9b43cbb927dc tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:46.925730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:46.928450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:46.929367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:46.929897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:46.934281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:46.938536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27.: keystone.exception.ImpliedRoleNotFound: b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27. Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = query.one() Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._iter().one() Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._only_one_row( Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:46.964140 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 17 21:12:46.965625 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 17 21:12:46.966884 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 17 21:12:46.966884 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 17 21:12:46.966884 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 17 21:12:46.966884 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: b348ff808766462fbeb7304f6c5aae6d does not imply 205287c074c9443fa434921e00b9ab27. Sep 17 21:12:46.966884 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:46.979767 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2795/5591] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:46 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 137 bytes in 119 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:12:46.992307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.011223 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2797/5592] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:47.012965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da35024-f559-4ab6-a8aa-a5377ed423b6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.021510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2d4b1-a933-4265-bb54-41b2088d68c0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:47.032950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:47.035085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da35024-f559-4ab6-a8aa-a5377ed423b6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:47.035577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:47.035577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:47.036825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:47.043235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34472bc-6320-452e-8139-f4723fdcbfd8 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:47.044155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34472bc-6320-452e-8139-f4723fdcbfd8 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:47.044342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:47.044574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34472bc-6320-452e-8139-f4723fdcbfd8 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:47.045150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34472bc-6320-452e-8139-f4723fdcbfd8 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:47.047832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:47.115104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e34472bc-6320-452e-8139-f4723fdcbfd8 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:47.121522 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2796/5593] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:46 2022] PUT /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 520 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:47.127025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.133494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2798/5594] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:12:47 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:47.242154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06e361-13d4-4282-980e-f863c4177154 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:47.248357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:47.250270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:47.250545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:47.251111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:47.253394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=20cb997598bb4f4d807e93030d71778a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:47.255483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:47.308488 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2797/5595] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:47 2022] PUT /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a => generated 519 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:47.314210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34472bc-6320-452e-8139-f4723fdcbfd8 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.328681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34472bc-6320-452e-8139-f4723fdcbfd8 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:47.407910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:47.448298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:47.449619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:47.449619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:47.450046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:47.452407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=20cb997598bb4f4d807e93030d71778a, implied_role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:47.459645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:47.499176 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2799/5596] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:47 2022] PUT /identity/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3 => generated 520 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:47.504053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.519910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e8821-f89c-4e82-b778-812046ffe86a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:47.616296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:47.645914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:47.647487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:47.647710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:47.647953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:47.649559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:47.651640 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:47.651640 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:47.651640 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:47.652701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:47.679691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2798/5597] 10.222.0.26 () {58 vars in 1152 bytes} [Sat Sep 17 21:12:47 2022] GET /identity/v3/role_inferences => generated 1605 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:47.684198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.699613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965d413-9df0-40d0-a443-8bb12056606f tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:47.709007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:47.709007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:47.709007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:47.709443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:47.712309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=20cb997598bb4f4d807e93030d71778a, implied_role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:47.714325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:47.730569 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2800/5598] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:47 2022] DELETE /identity/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:47.732652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.748469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5f877-b2ce-4b1d-bdc6-dd33a6e56b1e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:47.846001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:47.846947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:47.847462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:47.847836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:47.850722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=20cb997598bb4f4d807e93030d71778a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:47.852617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:47.866871 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2799/5599] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:47 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:47.871631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:47.896485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae2647-10c5-4afc-8203-801f4b72e604 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.031500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b94ca0-8b63-44f0-9b75-b8b1307fdbac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.032976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b94ca0-8b63-44f0-9b75-b8b1307fdbac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.033290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b94ca0-8b63-44f0-9b75-b8b1307fdbac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.033715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b94ca0-8b63-44f0-9b75-b8b1307fdbac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.036274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b94ca0-8b63-44f0-9b75-b8b1307fdbac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.038196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b94ca0-8b63-44f0-9b75-b8b1307fdbac tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.059500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2801/5600] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:47 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:48.060965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.084552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586693ab-5af5-49e0-836b-a6eba4bcb1ab tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.139410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568028d-042f-4b88-8ecf-97bd5edf7c09 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.139827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568028d-042f-4b88-8ecf-97bd5edf7c09 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.140607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568028d-042f-4b88-8ecf-97bd5edf7c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.183615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d568028d-042f-4b88-8ecf-97bd5edf7c09 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:48.185067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d568028d-042f-4b88-8ecf-97bd5edf7c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:48.193614 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2802/5601] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:48 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:12:48.195233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b58cb-d1c9-409c-bacd-726ff0a052ea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.196717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b58cb-d1c9-409c-bacd-726ff0a052ea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.197101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b58cb-d1c9-409c-bacd-726ff0a052ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.201319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.202197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.202486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.202761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.203656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.205634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.228690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-710b58cb-d1c9-409c-bacd-726ff0a052ea None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:48.231480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-710b58cb-d1c9-409c-bacd-726ff0a052ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:48.258059 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2800/5602] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:48 2022] GET /identity/v3/roles => generated 2322 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:48.269057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.285124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8fd2f-ccda-4562-96a3-f192e33e1c0a tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.291612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.292296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.292671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.293226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.294350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.296040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.330603 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2801/5603] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:12:48 2022] POST /identity/v3/roles => generated 235 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:48.337376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.348901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368390f-2278-4da6-97c2-cbb58ae1bf53 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.361373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.362264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.362554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.362836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b1b6df7f417046a9a9a5cde84a346144` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.364337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:update_role(role_id=b1b6df7f417046a9a9a5cde84a346144)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.365753 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:48.365753 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:48.365753 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:48.366646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.371719 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2803/5604] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:48.382281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b58cb-d1c9-409c-bacd-726ff0a052ea None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.404282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b58cb-d1c9-409c-bacd-726ff0a052ea None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.411179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7a02-82f6-4c3b-8feb-36302b05fd40 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.413219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7a02-82f6-4c3b-8feb-36302b05fd40 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.413219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7a02-82f6-4c3b-8feb-36302b05fd40 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.413219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7a02-82f6-4c3b-8feb-36302b05fd40 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.432091 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2802/5605] 10.222.0.26 () {60 vars in 1243 bytes} [Sat Sep 17 21:12:48 2022] PATCH /identity/v3/roles/b1b6df7f417046a9a9a5cde84a346144 => generated 238 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:48.438322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.451400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0ef73-2eda-4b64-bcf2-830cf4d24f04 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.461816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.462994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.463485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.463916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b1b6df7f417046a9a9a5cde84a346144` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.468543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-59be7a02-82f6-4c3b-8feb-36302b05fd40 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:48.472331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:48.478749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:get_role(role_id=b1b6df7f417046a9a9a5cde84a346144)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.483539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.485369 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2804/5606] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:12:48 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:48.491120 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2803/5607] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:12:48 2022] GET /identity/v3/roles/b1b6df7f417046a9a9a5cde84a346144 => generated 238 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:48.491334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7a02-82f6-4c3b-8feb-36302b05fd40 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.508230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7a02-82f6-4c3b-8feb-36302b05fd40 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.515021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.515642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.516724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.516724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.517300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.519659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.537213 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2805/5608] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:48 2022] GET /identity/v3/roles => generated 2551 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:48.539789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.553865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bcb7d-c7a6-4df9-9bc5-b45fa53ab829 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.560231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.561423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.561722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.562499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b1b6df7f417046a9a9a5cde84a346144` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.565451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b1b6df7f417046a9a9a5cde84a346144)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.565451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.580576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:48.638635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role b1b6df7f417046a9a9a5cde84a346144 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:48.639200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:48.639901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:48.647401 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2804/5609] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:48 2022] DELETE /identity/v3/roles/b1b6df7f417046a9a9a5cde84a346144 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:48.649217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.651131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:48.683083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52d2be-d9af-4510-b423-bca990d3f956 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.787343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.788754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.789311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.789765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.792009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.794313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:48.845370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2806/5610] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:48 2022] PUT /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 520 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:48.849704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:48.865441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dbf5d-c357-46aa-82ea-ded190abb26e tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:48.949757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:48.951154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:48.960470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:48.960950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:48.963187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=20cb997598bb4f4d807e93030d71778a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:48.965091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:49.011525 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2805/5611] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:48 2022] PUT /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a => generated 519 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:49.016467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.030783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830a7bf-7539-4f36-91f2-3de646dc80d2 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:49.120564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:49.121305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.121618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.121921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.124174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=20cb997598bb4f4d807e93030d71778a, implied_role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:49.128679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:49.174392 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2807/5612] 10.222.0.26 () {60 vars in 1363 bytes} [Sat Sep 17 21:12:49 2022] PUT /identity/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3 => generated 520 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:49.178663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.201667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429690d5-56d4-470a-8701-0f976598c4f0 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:49.292633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:49.295006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.295309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.295721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/20cb997598bb4f4d807e93030d71778a/implies` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.296831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=20cb997598bb4f4d807e93030d71778a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:49.297981 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:49.297981 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:12:49.297981 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:12:49.299090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:49.306341 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2806/5613] 10.222.0.26 () {58 vars in 1245 bytes} [Sat Sep 17 21:12:49 2022] GET /identity/v3/roles/20cb997598bb4f4d807e93030d71778a/implies => generated 489 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:49.311440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.340063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57e5a1-92bc-4b3f-b9b6-f11478e34c73 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:49.345161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:49.345899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.346253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.347872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.348794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=b348ff808766462fbeb7304f6c5aae6d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:49.350456 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:12:49.350456 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:12:49.350456 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:12:49.351979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:49.360855 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2808/5614] 10.222.0.26 () {58 vars in 1245 bytes} [Sat Sep 17 21:12:49 2022] GET /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies => generated 659 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:49.365221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.376425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12119d34-0b7d-4f72-a1bf-bb63a4196af1 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:49.381358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429917a-d6a8-4b5f-8b69-6e875a189771 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:49.382364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429917a-d6a8-4b5f-8b69-6e875a189771 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.382364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429917a-d6a8-4b5f-8b69-6e875a189771 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.382776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429917a-d6a8-4b5f-8b69-6e875a189771 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.385627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429917a-d6a8-4b5f-8b69-6e875a189771 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=20cb997598bb4f4d807e93030d71778a, implied_role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:49.387224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429917a-d6a8-4b5f-8b69-6e875a189771 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:49.401531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2807/5615] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:49 2022] DELETE /identity/v3/roles/20cb997598bb4f4d807e93030d71778a/implies/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:49.405745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.416943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b6d20-1c22-4782-abc6-5c785a3c7ef8 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:49.492866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081b03-9f46-444c-9ddc-65d3119794c0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.493256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081b03-9f46-444c-9ddc-65d3119794c0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.493658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081b03-9f46-444c-9ddc-65d3119794c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.536948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-23081b03-9f46-444c-9ddc-65d3119794c0 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:49.539192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-23081b03-9f46-444c-9ddc-65d3119794c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:49.548398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2808/5616] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:49 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 21:12:49.550919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51664bc-04e0-4e7f-8ef0-2aa5a216ef37 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.551390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51664bc-04e0-4e7f-8ef0-2aa5a216ef37 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.552016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51664bc-04e0-4e7f-8ef0-2aa5a216ef37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.584289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a51664bc-04e0-4e7f-8ef0-2aa5a216ef37 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:49.586067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a51664bc-04e0-4e7f-8ef0-2aa5a216ef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:49.640474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:49.657492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:49.694257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:49.697016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.699323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.699959 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2809/5617] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:49.700304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.704384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51664bc-04e0-4e7f-8ef0-2aa5a216ef37 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.706054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=20cb997598bb4f4d807e93030d71778a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:49.708279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:49.717432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51664bc-04e0-4e7f-8ef0-2aa5a216ef37 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:49.724112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ee1e-c7ad-4391-a160-b048bbae1abb tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:49.725557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ee1e-c7ad-4391-a160-b048bbae1abb tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.725958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ee1e-c7ad-4391-a160-b048bbae1abb tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.726503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ee1e-c7ad-4391-a160-b048bbae1abb tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.727385 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2809/5618] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:49 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/20cb997598bb4f4d807e93030d71778a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:49.731899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.810933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-29a1ee1e-c7ad-4391-a160-b048bbae1abb tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:49.846396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2810/5619] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:12:49 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:49.925550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6964d-98a6-48c0-835b-4def578255ba tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:49.943436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:49.944367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:49.944717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:49.945179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:49.947500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=b348ff808766462fbeb7304f6c5aae6d, implied_role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:49.949366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:49.977291 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2810/5620] 10.222.0.26 () {58 vars in 1347 bytes} [Sat Sep 17 21:12:49 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d/implies/205287c074c9443fa434921e00b9ab27 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:49.978159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ee1e-c7ad-4391-a160-b048bbae1abb tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:49.995755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ee1e-c7ad-4391-a160-b048bbae1abb tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:50.170734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:50.171861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:50.172630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:50.172630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/41753e96bd994ae88568c5a5f3af4f3d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:50.186307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Could not find domain: 41753e96bd994ae88568c5a5f3af4f3d.: keystone.exception.DomainNotFound: Could not find domain: 41753e96bd994ae88568c5a5f3af4f3d. Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 41753e96bd994ae88568c5a5f3af4f3d. Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:12:50.204353 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:12:50.205828 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 41753e96bd994ae88568c5a5f3af4f3d. Sep 17 21:12:50.207803 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:12:50.210393 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2811/5621] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:12:49 2022] PATCH /identity/v3/domains/41753e96bd994ae88568c5a5f3af4f3d => generated 112 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:12:50.214412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:50.229284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e508f-0ade-49af-a2d6-d0b9998dc512 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:50.236649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:50.237491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:50.237909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:50.238370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/users/faf54362aa9a4047bcb2c067f7b9b02e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:50.239695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_user(user_id=faf54362aa9a4047bcb2c067f7b9b02e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:50.244005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:50.319214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf54362aa9a4047bcb2c067f7b9b02e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:50.348081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf54362aa9a4047bcb2c067f7b9b02e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:50.366857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf54362aa9a4047bcb2c067f7b9b02e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:50.368972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf54362aa9a4047bcb2c067f7b9b02e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:50.369834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf54362aa9a4047bcb2c067f7b9b02e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:50.390233 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2811/5622] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:50 2022] DELETE /identity/v3/users/faf54362aa9a4047bcb2c067f7b9b02e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:50.394655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:50.398225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:50.414446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7243e3-7477-4a39-b5e8-ca30c0993216 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:50.528292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:50.528953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:50.529232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:50.529525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/bfedd871715c4f25a287d0c531c724f3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:50.531155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bfedd871715c4f25a287d0c531c724f3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:50.533272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:50.606524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role bfedd871715c4f25a287d0c531c724f3 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:50.606524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:50.606524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-437e39c8-f7b0-473e-b9d4-edcfe6af4874 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:50.612381 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2812/5623] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:50 2022] DELETE /identity/v3/roles/bfedd871715c4f25a287d0c531c724f3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:50.616912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:50.651545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f7a59-d8c4-4755-b31e-11cb74e8b896 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:50.767451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:50.799517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:50.802260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:50.802831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:50.803401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/groups/304008188b134566b4b04d1e26c07946` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:50.808126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_group(group_id=304008188b134566b4b04d1e26c07946)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:50.811248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:50.852519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:50.853846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:50.856382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:50.903678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:50.906879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:50.913402 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2813/5624] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:50 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:12:50.920731 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2812/5625] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:12:50 2022] DELETE /identity/v3/groups/304008188b134566b4b04d1e26c07946 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:50.924716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:50.926686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:50.942582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:51.072805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe89e1-34a7-4252-9f1c-8c4157678d09 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:51.080170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:51.081308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:51.081730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:51.082373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/projects/963e5b531cf64e41b26bb29d69e0a735` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:51.085326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_project(project_id=963e5b531cf64e41b26bb29d69e0a735)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:51.087598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:51.198224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963e5b531cf64e41b26bb29d69e0a735', 'request_id': 'req-ac35185a-2f3a-491f-87aa-d630983c1f82'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.209098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963e5b531cf64e41b26bb29d69e0a735', 'request_id': 'req-ac35185a-2f3a-491f-87aa-d630983c1f82'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.226157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963e5b531cf64e41b26bb29d69e0a735', 'request_id': 'req-ac35185a-2f3a-491f-87aa-d630983c1f82'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.252441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] The token cache is being invalidate because project 963e5b531cf64e41b26bb29d69e0a735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:51.252908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.253949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.256121 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2814/5626] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:50 2022] DELETE /identity/v3/projects/963e5b531cf64e41b26bb29d69e0a735 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:51.260530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:51.264336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:51.282063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571b58-5686-4911-8e01-d82fc2d24d80 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:51.387543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:51.389075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:51.389746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:51.390900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:51.392891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a1cadb4bc7f643c396c8d84d79be82ff)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:51.394664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:51.427952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.428806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.472723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.473877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.479121 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2813/5627] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:12:51 2022] PATCH /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff => generated 276 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:51.483265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:51.499602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35185a-2f3a-491f-87aa-d630983c1f82 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:51.598436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:51.599362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:51.600057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:51.600057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:51.616164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a1cadb4bc7f643c396c8d84d79be82ff)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:51.619948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:51.653773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.661386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.669546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:51.748259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff', 'request_id': 'req-397fdd56-94e9-4326-9524-a2fba3c004fd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.766239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff', 'request_id': 'req-397fdd56-94e9-4326-9524-a2fba3c004fd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.783420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff', 'request_id': 'req-397fdd56-94e9-4326-9524-a2fba3c004fd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.806688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] The token cache is being invalidate because project a1cadb4bc7f643c396c8d84d79be82ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:51.807014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.810452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.819545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff', 'request_id': 'req-397fdd56-94e9-4326-9524-a2fba3c004fd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.823751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff', 'request_id': 'req-397fdd56-94e9-4326-9524-a2fba3c004fd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.826591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-397fdd56-94e9-4326-9524-a2fba3c004fd tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a1cadb4bc7f643c396c8d84d79be82ff', 'request_id': 'req-397fdd56-94e9-4326-9524-a2fba3c004fd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.842494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2815/5628] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:12:51 2022] DELETE /identity/v3/domains/a1cadb4bc7f643c396c8d84d79be82ff => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:51.846482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:51.847969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:51.866344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf9df4-dc54-41aa-b0bb-5aef1ee851bb tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:51.920257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178f37f-d4d3-4316-8d8a-99185fc0d81e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:51.920866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178f37f-d4d3-4316-8d8a-99185fc0d81e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:51.921368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178f37f-d4d3-4316-8d8a-99185fc0d81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:51.958134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7178f37f-d4d3-4316-8d8a-99185fc0d81e None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:51.960961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7178f37f-d4d3-4316-8d8a-99185fc0d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.001808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.003077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.003077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.004078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/20cb997598bb4f4d807e93030d71778a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.006145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_role(role_id=20cb997598bb4f4d807e93030d71778a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.008951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.039052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:52.095879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role 20cb997598bb4f4d807e93030d71778a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:52.096352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.097407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.103203 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2814/5629] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:51 2022] DELETE /identity/v3/roles/20cb997598bb4f4d807e93030d71778a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:52.104589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.106793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.138579 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2816/5630] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:52.146389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62d6da-3667-47d0-9930-68da4f69eb50 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.148314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62d6da-3667-47d0-9930-68da4f69eb50 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.148686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62d6da-3667-47d0-9930-68da4f69eb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.186389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-af62d6da-3667-47d0-9930-68da4f69eb50 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:52.190998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af62d6da-3667-47d0-9930-68da4f69eb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.203969 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2817/5631] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:12:52.221203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4879f-3a4b-47ae-91fc-b90bcbe622ba None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.221864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4879f-3a4b-47ae-91fc-b90bcbe622ba None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.222292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4879f-3a4b-47ae-91fc-b90bcbe622ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.252214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e6ba4-6284-4bfc-9cb4-40719bb4c454 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.263694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.264754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.264923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.266379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/205287c074c9443fa434921e00b9ab27` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.268910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_role(role_id=205287c074c9443fa434921e00b9ab27)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.270742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.339789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role 205287c074c9443fa434921e00b9ab27 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:52.340762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.341973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.345102 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2815/5632] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:52 2022] DELETE /identity/v3/roles/205287c074c9443fa434921e00b9ab27 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:52.349064 ncdv-master-1 devstack@keystone.service[99167]: WARNING keystone.server.flask.application [None req-eea4879f-3a4b-47ae-91fc-b90bcbe622ba None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 17 21:12:52.350885 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2818/5633] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 144 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Sep 17 21:12:52.354755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.355041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e051-ff16-4c5e-8d77-b38eab3cb038 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.355360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e051-ff16-4c5e-8d77-b38eab3cb038 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.355641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5e051-ff16-4c5e-8d77-b38eab3cb038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.357707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.447109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ffabf-f53c-4f1c-a8c8-1fe4700b0e33 tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.450826 ncdv-master-1 devstack@keystone.service[99167]: WARNING keystone.server.flask.application [None req-80d5e051-ff16-4c5e-8d77-b38eab3cb038 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 17 21:12:52.453157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cc8125d1fb24485a0a9d41f1ff5fe61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '677f9030a37e4d6c82c5b9b9e43fece3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.453687 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2819/5634] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 100 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Sep 17 21:12:52.453971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.454590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.454590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] PATH_INFO: `/v3/roles/b348ff808766462fbeb7304f6c5aae6d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.456469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b348ff808766462fbeb7304f6c5aae6d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.458408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.458805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772a3f-dacf-4433-a1bf-182dcfa9880c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.459351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772a3f-dacf-4433-a1bf-182dcfa9880c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.460531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772a3f-dacf-4433-a1bf-182dcfa9880c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.507879 ncdv-master-1 devstack@keystone.service[99167]: WARNING keystone.server.flask.application [None req-14772a3f-dacf-4433-a1bf-182dcfa9880c None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 17 21:12:52.511866 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2820/5635] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 54 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 17 21:12:52.541333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invalidating the token cache because role b348ff808766462fbeb7304f6c5aae6d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:52.541874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.543160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7e753b5-bda3-4b1c-ba08-d0c9391346ee tempest-RolesV3TestJSON-1710027648 tempest-RolesV3TestJSON-1710027648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.546508 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2816/5636] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:52 2022] DELETE /identity/v3/roles/b348ff808766462fbeb7304f6c5aae6d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:52.551038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772a3f-dacf-4433-a1bf-182dcfa9880c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.557491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-14772a3f-dacf-4433-a1bf-182dcfa9880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.624919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=1,keystone:UPDATE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:52.741723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772a3f-dacf-4433-a1bf-182dcfa9880c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.747973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.749185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.749391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.749675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] PATH_INFO: `/v3/users/bfafce367b684265bd8263f39da04048` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.750864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfafce367b684265bd8263f39da04048)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.752834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.829173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfafce367b684265bd8263f39da04048'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.830141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfafce367b684265bd8263f39da04048'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.852338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfafce367b684265bd8263f39da04048'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.854023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfafce367b684265bd8263f39da04048'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.879854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfafce367b684265bd8263f39da04048'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.921000 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2821/5637] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:52 2022] DELETE /identity/v3/users/bfafce367b684265bd8263f39da04048 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:52.925777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.926097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.926352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.942174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:52.962169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:52.965874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:53.077365 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2817/5638] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:53.095454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:53.096971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:53.218564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99d90e-7135-4ee0-83fb-c5419d04ae20 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:53.226275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:53.226275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:53.226275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:53.226275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:53.244575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:53.247001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:53.262648 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2822/5639] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:54.084891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:54.102370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:54.118785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796627c-a18d-42ce-846f-b68a2c436542 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:54.133956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:54.134728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:54.135100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:54.135382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] PATH_INFO: `/v3/users/2d249eb9b0124c339d8e730c96649171` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:54.137741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d249eb9b0124c339d8e730c96649171)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:54.140625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:54.213384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d249eb9b0124c339d8e730c96649171'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.232073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d249eb9b0124c339d8e730c96649171'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.241900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d249eb9b0124c339d8e730c96649171'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.243073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d249eb9b0124c339d8e730c96649171'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.243688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d249eb9b0124c339d8e730c96649171'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.276278 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2818/5640] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:54 2022] DELETE /identity/v3/users/2d249eb9b0124c339d8e730c96649171 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:54.538597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:54.540775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:54.556229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2d442-265c-4281-8ab6-873a4e7e2cbc service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:54.652185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:54.682722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:54.683916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:54.684320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:54.684736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] PATH_INFO: `/v3/projects/ad85b9c7350e45a3b4598302209b3aa5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:54.686190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad85b9c7350e45a3b4598302209b3aa5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:54.688276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:54.787141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad85b9c7350e45a3b4598302209b3aa5', 'request_id': 'req-5e761071-4396-4e68-acd7-4108aee63cd7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.792044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:54.794486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:54.800039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad85b9c7350e45a3b4598302209b3aa5', 'request_id': 'req-5e761071-4396-4e68-acd7-4108aee63cd7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.825233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad85b9c7350e45a3b4598302209b3aa5', 'request_id': 'req-5e761071-4396-4e68-acd7-4108aee63cd7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.832711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640825b6-2698-4f47-8f5d-c55b7bf3db8c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:54.842688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] The token cache is being invalidate because project ad85b9c7350e45a3b4598302209b3aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:54.842688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.843444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:54.845346 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2823/5641] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:54 2022] DELETE /identity/v3/projects/ad85b9c7350e45a3b4598302209b3aa5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:54.846494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:54.848215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:54.956978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:54.958000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:54.958427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:54.958934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] PATH_INFO: `/v3/projects/3723f4e7f0bb4a71ac7aaf46def6b4f6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:54.964345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3723f4e7f0bb4a71ac7aaf46def6b4f6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:54.966946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:54.978391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e761071-4396-4e68-acd7-4108aee63cd7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:54.989072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:54.992161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:54.992463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:54.992795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] PATH_INFO: `/v3/users/5b51dbc1bdc54503bb7371e7491fa43d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:54.994824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b51dbc1bdc54503bb7371e7491fa43d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:54.995701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:55.002314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:55.058830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3723f4e7f0bb4a71ac7aaf46def6b4f6', 'request_id': 'req-92cb2abd-b9f1-4d56-b3d4-c65382f90411'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.083820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3723f4e7f0bb4a71ac7aaf46def6b4f6', 'request_id': 'req-92cb2abd-b9f1-4d56-b3d4-c65382f90411'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.093559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b51dbc1bdc54503bb7371e7491fa43d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.094553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b51dbc1bdc54503bb7371e7491fa43d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.096607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3723f4e7f0bb4a71ac7aaf46def6b4f6', 'request_id': 'req-92cb2abd-b9f1-4d56-b3d4-c65382f90411'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.113702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] The token cache is being invalidate because project 3723f4e7f0bb4a71ac7aaf46def6b4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:55.114007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.114839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.116592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b51dbc1bdc54503bb7371e7491fa43d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.118290 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2819/5642] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:54 2022] DELETE /identity/v3/projects/3723f4e7f0bb4a71ac7aaf46def6b4f6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:55.119721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:55.124376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:55.129292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b51dbc1bdc54503bb7371e7491fa43d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.170042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6f25aee1-e07d-456e-85ab-c6d4f70c6a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b51dbc1bdc54503bb7371e7491fa43d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.181905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:55.205552 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2824/5643] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:54 2022] DELETE /identity/v3/users/5b51dbc1bdc54503bb7371e7491fa43d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:55.208066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640ceb1-588b-4c14-962f-925946a65bb9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.209427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640ceb1-588b-4c14-962f-925946a65bb9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.210127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640ceb1-588b-4c14-962f-925946a65bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.249770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4640ceb1-588b-4c14-962f-925946a65bb9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:55.251406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4640ceb1-588b-4c14-962f-925946a65bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:55.253096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb2abd-b9f1-4d56-b3d4-c65382f90411 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:55.354457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:55.357560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.358206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.358635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] PATH_INFO: `/v3/users/2cc8125d1fb24485a0a9d41f1ff5fe61` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.361195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc8125d1fb24485a0a9d41f1ff5fe61)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:55.368285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:55.381080 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2825/5644] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:55.383835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee7930-5f09-4761-982b-6102578433b8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.383835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee7930-5f09-4761-982b-6102578433b8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.383835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee7930-5f09-4761-982b-6102578433b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.422254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1eee7930-5f09-4761-982b-6102578433b8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:55.423721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1eee7930-5f09-4761-982b-6102578433b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:55.434426 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2826/5645] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:55.436367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.436748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.437293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.463945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:55.466682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:55.477500 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2827/5646] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:55.479365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:55.482324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:55.497882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc8125d1fb24485a0a9d41f1ff5fe61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.514896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:55.528166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc8125d1fb24485a0a9d41f1ff5fe61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.544064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc8125d1fb24485a0a9d41f1ff5fe61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.544949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc8125d1fb24485a0a9d41f1ff5fe61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.546489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc8125d1fb24485a0a9d41f1ff5fe61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:55.570935 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2820/5647] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:55 2022] DELETE /identity/v3/users/2cc8125d1fb24485a0a9d41f1ff5fe61 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:55.572516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:55.575141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:55.599384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119b478-b588-4676-a15b-bbf67576afe6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:55.704832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:55.705152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0728b6-702d-46ec-b834-362f2ccab24c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:55.707138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.707420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.707696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.709761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:55.719237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:55.720179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.720522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.720848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.722153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:55.722468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:55.724364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:55.730947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:55.734411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:55.741891 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2821/5648] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:55.750119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:55.760396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2828/5649] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:55.768323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f64c4-e23c-4f0e-95d1-639838ab6230 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:55.786583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:55.788081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.788551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.788981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.789815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:55.792711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:55.860383 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2822/5650] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:55 2022] POST /identity/v3/projects => generated 395 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:55.865924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:55.880788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6aa7-ba06-465c-83ce-8bc28c459020 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:55.947564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:55.992864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:55.994068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:55.995239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:55.996055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:55.998299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.004023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.050623 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2829/5651] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:55 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:56.055640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:56.071436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fcf0b-cd40-4bf4-9ec9-3c642bbefac5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:56.081515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:56.083095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.083687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.084028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.085156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.095092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.108551 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2823/5652] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:56.113699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:56.130412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f17564-3eb7-44a2-886b-d2f5f4658543 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:56.141084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddc162-865b-4d68-b51c-ae65c6ff13e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:56.141914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddc162-865b-4d68-b51c-ae65c6ff13e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.142394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddc162-865b-4d68-b51c-ae65c6ff13e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.142874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddc162-865b-4d68-b51c-ae65c6ff13e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.144061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddc162-865b-4d68-b51c-ae65c6ff13e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.146697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddc162-865b-4d68-b51c-ae65c6ff13e9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.166051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:56.175220 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2830/5653] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:56.180965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a56804-7439-4d5f-9146-6441345f80ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:56.186507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:56.187373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.187740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.188107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] PATH_INFO: `/v3/projects/ee1481a8ca9c4dd590cd6c7c53179df8/users/2db8955202fe42d2a4c86a15b193787f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.210028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1481a8ca9c4dd590cd6c7c53179df8, user_id=2db8955202fe42d2a4c86a15b193787f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.214481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.235573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2824/5654] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:56 2022] PUT /identity/v3/projects/ee1481a8ca9c4dd590cd6c7c53179df8/users/2db8955202fe42d2a4c86a15b193787f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:56.241129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5100d-3821-4821-a498-e29e39262255 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.241428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5100d-3821-4821-a498-e29e39262255 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.241742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5100d-3821-4821-a498-e29e39262255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.245306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:56.248425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.269574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bde5100d-3821-4821-a498-e29e39262255 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:56.270880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bde5100d-3821-4821-a498-e29e39262255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.357903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e986-8faf-4abf-997a-b4e86847a3d6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:56.364867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:56.366036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.366325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.366622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] PATH_INFO: `/v3/users/adfe77cd9de149508efbea44af748876` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.367937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adfe77cd9de149508efbea44af748876)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.371165 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2831/5655] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:56.375075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.380223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5100d-3821-4821-a498-e29e39262255 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:56.382269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bde5100d-3821-4821-a498-e29e39262255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.399632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:56.469173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adfe77cd9de149508efbea44af748876'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.488915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adfe77cd9de149508efbea44af748876'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.490453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5100d-3821-4821-a498-e29e39262255 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:56.498411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:56.498993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adfe77cd9de149508efbea44af748876'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.499573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adfe77cd9de149508efbea44af748876'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.499784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.500379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adfe77cd9de149508efbea44af748876'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.500982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.501374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.504579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.517535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2825/5656] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:56 2022] DELETE /identity/v3/users/adfe77cd9de149508efbea44af748876 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:12:56.522037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:56.523634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.523785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.525289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.543332 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2832/5657] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:12:56.545731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:56.548323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.564692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5eb73c-a793-448a-bf8b-d2071e287bb5 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:56.654095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:56.654785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.654988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.656138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] PATH_INFO: `/v3/projects/677f9030a37e4d6c82c5b9b9e43fece3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.657745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=677f9030a37e4d6c82c5b9b9e43fece3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.660258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.663509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:56.686422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d180d57-abf1-4fdf-a58e-7a354ff78691 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:56.693066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:56.694094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.694380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.694656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] PATH_INFO: `/v3/projects/7a2458735038406092c81b60273d5f52` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.696016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a2458735038406092c81b60273d5f52)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:56.698794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:56.785763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '677f9030a37e4d6c82c5b9b9e43fece3', 'request_id': 'req-64136e31-c377-4de7-b582-60f157705841'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.799895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '677f9030a37e4d6c82c5b9b9e43fece3', 'request_id': 'req-64136e31-c377-4de7-b582-60f157705841'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.802427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a2458735038406092c81b60273d5f52', 'request_id': 'req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.812809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '677f9030a37e4d6c82c5b9b9e43fece3', 'request_id': 'req-64136e31-c377-4de7-b582-60f157705841'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.817442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a2458735038406092c81b60273d5f52', 'request_id': 'req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.827147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a2458735038406092c81b60273d5f52', 'request_id': 'req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.829877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] The token cache is being invalidate because project 677f9030a37e4d6c82c5b9b9e43fece3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:56.830761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.831288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-64136e31-c377-4de7-b582-60f157705841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.837756 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2833/5658] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:56 2022] DELETE /identity/v3/projects/677f9030a37e4d6c82c5b9b9e43fece3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:56.840573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011c6d4-172a-4184-b129-02ff74381d89 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.840963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011c6d4-172a-4184-b129-02ff74381d89 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.841665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011c6d4-172a-4184-b129-02ff74381d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.845243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] The token cache is being invalidate because project 7a2458735038406092c81b60273d5f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:56.845626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.846494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c2b9af6-84b4-4f09-bbb1-a847dc0ca193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:56.848625 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2826/5659] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:56 2022] DELETE /identity/v3/projects/7a2458735038406092c81b60273d5f52 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:56.854133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8e72f-f757-4313-be33-485fb7b33a67 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.854468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8e72f-f757-4313-be33-485fb7b33a67 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.854759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8e72f-f757-4313-be33-485fb7b33a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:56.872343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2011c6d4-172a-4184-b129-02ff74381d89 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:56.874286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2011c6d4-172a-4184-b129-02ff74381d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.885403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-12d8e72f-f757-4313-be33-485fb7b33a67 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:56.887239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-12d8e72f-f757-4313-be33-485fb7b33a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:56.903735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:56.976797 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2834/5660] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:56.978602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2511b5e-f2f7-44a1-a86b-3f9b2a7d07e6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:56.978939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2511b5e-f2f7-44a1-a86b-3f9b2a7d07e6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:56.980142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2511b5e-f2f7-44a1-a86b-3f9b2a7d07e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.009880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2827/5661] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:12:57.014847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a2511b5e-f2f7-44a1-a86b-3f9b2a7d07e6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:57.021029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a2511b5e-f2f7-44a1-a86b-3f9b2a7d07e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.037659 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2835/5662] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:57.039660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2511b5e-f2f7-44a1-a86b-3f9b2a7d07e6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.040129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b4f91-b215-46ae-9c1e-d9067b4c8953 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.040613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b4f91-b215-46ae-9c1e-d9067b4c8953 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.041350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b4f91-b215-46ae-9c1e-d9067b4c8953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.056445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2511b5e-f2f7-44a1-a86b-3f9b2a7d07e6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.063606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.064425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.064983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.065370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.081822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:57.082195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.083937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.090548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0a3b4f91-b215-46ae-9c1e-d9067b4c8953 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:57.092496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0a3b4f91-b215-46ae-9c1e-d9067b4c8953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.101579 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2836/5663] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:57.101579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.101579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.110424 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2828/5664] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:57.111407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b4f91-b215-46ae-9c1e-d9067b4c8953 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.127697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b4f91-b215-46ae-9c1e-d9067b4c8953 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.132522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90e0c2-a62b-4a92-815f-452ff62ce35d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.139703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.140372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.140647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.140967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] PATH_INFO: `/v3/projects/bbb68270a6474c41b16e24cae657407c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.142177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.142925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb68270a6474c41b16e24cae657407c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.143155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.143366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.143646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.144449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.145046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.146419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.197073 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2829/5665] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:57.198706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.239254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb68270a6474c41b16e24cae657407c', 'request_id': 'req-3556907e-824e-4a74-be27-8340bafb1a90'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.248319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb68270a6474c41b16e24cae657407c', 'request_id': 'req-3556907e-824e-4a74-be27-8340bafb1a90'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.273337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb68270a6474c41b16e24cae657407c', 'request_id': 'req-3556907e-824e-4a74-be27-8340bafb1a90'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.279933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:57.288177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] The token cache is being invalidate because project bbb68270a6474c41b16e24cae657407c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:57.288884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.289548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.291784 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2837/5666] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:57 2022] DELETE /identity/v3/projects/bbb68270a6474c41b16e24cae657407c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:57.295488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.298091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.310515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0b480-982b-41d6-922c-003444419363 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.388352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556907e-824e-4a74-be27-8340bafb1a90 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.403607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.405069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.405706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.406087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.407098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.408066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.408200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.408649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.409196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.411292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:57.411806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.413188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.414568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.444345 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2830/5667] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:57.445263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.447534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.454758 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2838/5668] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:57.456644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.461192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.467006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3677631-e69b-44b5-a4d1-99caa3e291eb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.473054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.473890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.474118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.474590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] PATH_INFO: `/v3/users/b2704d86338949ba9bb87bf60ee24146` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.476214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2704d86338949ba9bb87bf60ee24146)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.479032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.484512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb7367-7da3-40de-b5c5-ec852a43e2c3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.494540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.495562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.496510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.497412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.498405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.503875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.517693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:57.555078 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2839/5669] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:57.560719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.579989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2704d86338949ba9bb87bf60ee24146'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.624820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2704d86338949ba9bb87bf60ee24146'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.635135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156daa5-4111-4dbc-ba61-db66d4d40d0d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.641506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2704d86338949ba9bb87bf60ee24146'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.642328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2704d86338949ba9bb87bf60ee24146'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.643066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2704d86338949ba9bb87bf60ee24146'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:57.646972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.650045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.650389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.651769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.652780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.655633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.663609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2831/5670] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:57 2022] DELETE /identity/v3/users/b2704d86338949ba9bb87bf60ee24146 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:57.664289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.678148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.699754 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2840/5671] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:57.704252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.708955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.739764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dedb3-ae12-49e3-a52a-cedc3c0880ed admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.813706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaca08e-c9d7-4927-b135-04a86435a4a3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.821412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.822253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.822628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.822954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.823882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.826902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.860375 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2832/5672] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/users => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:57.861742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.862437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.862713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.862989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.864477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.864981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.867902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.870132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:57.882046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c31fd-ed2b-4835-b3d3-db5c308f8381 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.888595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.889379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.889818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.890185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.890944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.892771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.899674 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2841/5673] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:57.901600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.906247 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2833/5674] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:57.910321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.934989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf81a5f-21ef-4f6e-a37d-64d477375e04 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.941813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.942074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30ab9f-1bac-43ae-918f-df95fe96069c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.942974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.943476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.943959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.944859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.947065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.962880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2834/5675] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:57.967194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.967896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.968600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.968883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.969160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] PATH_INFO: `/v3/projects/17c435ee210a4e68bf13205656e1b3f9/users/6d25098862934bb688c1bf1aaf22e079/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.981553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5566add-733d-421d-82ef-2ca613ec1432 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.986985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e88e8-6af1-490c-9e3e-263cda64c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.988133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e88e8-6af1-490c-9e3e-263cda64c6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.988781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e88e8-6af1-490c-9e3e-263cda64c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.989272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e88e8-6af1-490c-9e3e-263cda64c6b8 admin admin] PATH_INFO: `/v3/projects/09e45a9e6ff54bc7906dc9f7c60a4cfc/users/ce0278bcbdf940eeb53c6bb58865a4de/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.016875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c435ee210a4e68bf13205656e1b3f9, user_id=6d25098862934bb688c1bf1aaf22e079, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.021000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.026357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e88e8-6af1-490c-9e3e-263cda64c6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e45a9e6ff54bc7906dc9f7c60a4cfc, user_id=ce0278bcbdf940eeb53c6bb58865a4de, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.035468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e88e8-6af1-490c-9e3e-263cda64c6b8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.052663 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2842/5676] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:57 2022] PUT /identity/v3/projects/17c435ee210a4e68bf13205656e1b3f9/users/6d25098862934bb688c1bf1aaf22e079/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:58.052663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:58.066914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.072917 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2835/5677] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:57 2022] PUT /identity/v3/projects/09e45a9e6ff54bc7906dc9f7c60a4cfc/users/ce0278bcbdf940eeb53c6bb58865a4de/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:58.077344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa53c-a9aa-4524-9f0e-80563ab0dba7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.077782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa53c-a9aa-4524-9f0e-80563ab0dba7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.078152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa53c-a9aa-4524-9f0e-80563ab0dba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.153325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-21caa53c-a9aa-4524-9f0e-80563ab0dba7 None None] MFA Rules not processed for user `6d25098862934bb688c1bf1aaf22e079`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:58.155205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-21caa53c-a9aa-4524-9f0e-80563ab0dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.210072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1b6d3-0267-4a52-bfbd-bbacba2ac048 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:58.267251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:58.355762 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2836/5678] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:58.357146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beb1b6-f68e-4875-b4f5-b7ddfdaf382d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.357436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beb1b6-f68e-4875-b4f5-b7ddfdaf382d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.357738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beb1b6-f68e-4875-b4f5-b7ddfdaf382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.368116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:58.372055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.372745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.373155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] PATH_INFO: `/v3/projects/81efda2512d54722b767bdf07a566236` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.376689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81efda2512d54722b767bdf07a566236)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.383247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.419434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-77beb1b6-f68e-4875-b4f5-b7ddfdaf382d None None] MFA Rules not processed for user `ce0278bcbdf940eeb53c6bb58865a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:58.420639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-77beb1b6-f68e-4875-b4f5-b7ddfdaf382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.475024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81efda2512d54722b767bdf07a566236', 'request_id': 'req-2f227023-10a4-408c-ab91-5fdaabe98ce6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.482317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81efda2512d54722b767bdf07a566236', 'request_id': 'req-2f227023-10a4-408c-ab91-5fdaabe98ce6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.496348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81efda2512d54722b767bdf07a566236', 'request_id': 'req-2f227023-10a4-408c-ab91-5fdaabe98ce6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.516013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] The token cache is being invalidate because project 81efda2512d54722b767bdf07a566236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:58.516510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.518073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.521152 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2843/5679] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:58 2022] DELETE /identity/v3/projects/81efda2512d54722b767bdf07a566236 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:58.522158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:58.532011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.547565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:58.567703 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2837/5680] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:58.570227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.570707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.571860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.618778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:58.620020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.645283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f227023-10a4-408c-ab91-5fdaabe98ce6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:58.652561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d25098862934bb688c1bf1aaf22e079', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17c435ee210a4e68bf13205656e1b3f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:58.653637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.654010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.654378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] PATH_INFO: `/v3/auth/catalog` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.655272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.658447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.660573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2844/5681] 10.222.0.26 () {58 vars in 1143 bytes} [Sat Sep 17 21:12:58 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:58.661409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:58.665821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.761070 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2838/5682] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:58.761962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:58.764742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.784979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0a8ef-5587-4cb2-8f16-3f72aef47dd1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:58.790186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:58.790922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.791291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.791673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] PATH_INFO: `/v3/users/6d25098862934bb688c1bf1aaf22e079` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.793089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d25098862934bb688c1bf1aaf22e079)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.795806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.838597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d43875-f5af-4477-a0fa-3901e4261a3e tempest-IdentityCatalogTest-915177429 tempest-IdentityCatalogTest-915177429-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:58.846003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4daa2-5953-4408-8637-f844ea292057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:58.850370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4daa2-5953-4408-8637-f844ea292057 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.850757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4daa2-5953-4408-8637-f844ea292057 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.851049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4daa2-5953-4408-8637-f844ea292057 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.851805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4daa2-5953-4408-8637-f844ea292057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.854419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4daa2-5953-4408-8637-f844ea292057 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.891897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d25098862934bb688c1bf1aaf22e079'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.906598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:58.909003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d25098862934bb688c1bf1aaf22e079'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.912498 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2845/5683] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:58 2022] POST /identity/v3/projects => generated 405 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:58.914130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b95b7-1b73-4ca6-b52c-ee125865a759 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.914416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b95b7-1b73-4ca6-b52c-ee125865a759 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.914774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b95b7-1b73-4ca6-b52c-ee125865a759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.920949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d25098862934bb688c1bf1aaf22e079'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.921795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d25098862934bb688c1bf1aaf22e079'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.922853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d25098862934bb688c1bf1aaf22e079'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:58.939278 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2839/5684] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:58 2022] DELETE /identity/v3/users/6d25098862934bb688c1bf1aaf22e079 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:58.940253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:58.941806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:58.945347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-db0b95b7-1b73-4ca6-b52c-ee125865a759 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:58.947770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-db0b95b7-1b73-4ca6-b52c-ee125865a759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:59.035087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a24fb-510f-4b74-b557-a8ef9ab7bbdf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.040322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04c691-7a4a-46f2-946c-165f52656d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.041055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04c691-7a4a-46f2-946c-165f52656d78 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.041412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04c691-7a4a-46f2-946c-165f52656d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.041765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04c691-7a4a-46f2-946c-165f52656d78 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.042642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04c691-7a4a-46f2-946c-165f52656d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.045135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04c691-7a4a-46f2-946c-165f52656d78 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.079281 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2840/5685] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:58 2022] POST /identity/v3/users => generated 309 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:59.079281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fd2b1-3191-4c6a-aa88-8f42c05a57cd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.079281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fd2b1-3191-4c6a-aa88-8f42c05a57cd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.079786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fd2b1-3191-4c6a-aa88-8f42c05a57cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.096074 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2846/5686] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:59.097258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b95b7-1b73-4ca6-b52c-ee125865a759 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.107121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-583fd2b1-3191-4c6a-aa88-8f42c05a57cd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:59.108762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-583fd2b1-3191-4c6a-aa88-8f42c05a57cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:59.117575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b95b7-1b73-4ca6-b52c-ee125865a759 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.121826 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2841/5687] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:59.122414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.123114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.123390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.124001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.124650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.125259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fd2b1-3191-4c6a-aa88-8f42c05a57cd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.126362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.142050 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2847/5688] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:59.143015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.151342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fd2b1-3191-4c6a-aa88-8f42c05a57cd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.153797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac637-67a7-4086-b930-afdd78faf4dd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.158734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.159480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.159762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.160032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] PATH_INFO: `/v3/projects/de09b8784fd04be0bee0843df58a3ca3/users/25c8979b479c4c7ba588fac9b0ada3c2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.170164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.170908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.171859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.172204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.173244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:59.173636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.178308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.179439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de09b8784fd04be0bee0843df58a3ca3, user_id=25c8979b479c4c7ba588fac9b0ada3c2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.182234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.202239 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2848/5689] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:59 2022] PUT /identity/v3/projects/de09b8784fd04be0bee0843df58a3ca3/users/25c8979b479c4c7ba588fac9b0ada3c2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:59.203252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.210932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2842/5690] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:59.214182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:59.215463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.246984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c450b-9c97-4ae3-a095-4452780be3da admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.302579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e24c19-b7f7-4d6d-9598-224e1cb1ec77 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.309176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.310204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.310820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.311320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.312185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.314110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.333606 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2849/5691] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:59.333606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.335990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:59.352832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.353756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.354101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.354440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.355328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.357820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.394161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2843/5692] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:59 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:59.395535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.401521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f3217-98f5-4f7f-bead-1cee7c275028 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.447508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:59.460069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251932-64cd-4941-b8a7-bf0a8c6e21ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.460686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251932-64cd-4941-b8a7-bf0a8c6e21ad service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.460966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251932-64cd-4941-b8a7-bf0a8c6e21ad service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.461235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251932-64cd-4941-b8a7-bf0a8c6e21ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.480752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37560ae-0f38-4179-8c2b-69b349327f6a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.485799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.486678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.486965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.487289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] PATH_INFO: `/v3/projects/de09b8784fd04be0bee0843df58a3ca3/users/25c8979b479c4c7ba588fac9b0ada3c2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.490178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de09b8784fd04be0bee0843df58a3ca3, user_id=25c8979b479c4c7ba588fac9b0ada3c2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.493181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.512168 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2844/5693] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:59 2022] PUT /identity/v3/projects/de09b8784fd04be0bee0843df58a3ca3/users/25c8979b479c4c7ba588fac9b0ada3c2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:59.512168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.534555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81251932-64cd-4941-b8a7-bf0a8c6e21ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.536259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81251932-64cd-4941-b8a7-bf0a8c6e21ad service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.547281 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2850/5694] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:59.548544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.548832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.549112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.572543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:59.584291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] MFA Rules not processed for user `25c8979b479c4c7ba588fac9b0ada3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:59.585710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:59.601521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054b1d2-0a78-417a-a115-a5d287fc6c52 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.610292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.610972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.611260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.611679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.612401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.615737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.652200 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2845/5695] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:59 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:59.656230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.674714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db632-e372-42d9-b31c-4f6754743e53 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.682811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.683471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.683605 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2851/5696] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:59.684156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.684156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.684787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.692266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.695130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.698110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:59.708348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2846/5697] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:59.710435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.725644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a04f66-e29f-4142-9b86-3eaf22a43d3e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.732143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.732997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.733427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.734171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.735035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.737753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.758802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2847/5698] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:59.759740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:59.773155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acae85e-519c-41fb-8a66-e7b9a9d2fa2c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.780285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.781794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.782139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.782447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] PATH_INFO: `/v3/projects/896f5e22b177497eb73cabac0c9ab6c4/users/201c6cfb271b43ba931ad686f62f25df/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.800034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72c5d-e3b9-4592-9785-690f39e63d52 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:59.803579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896f5e22b177497eb73cabac0c9ab6c4, user_id=201c6cfb271b43ba931ad686f62f25df, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.805666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594ba32-4c51-46eb-ae63-6fcdfa0a6b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:59.806361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594ba32-4c51-46eb-ae63-6fcdfa0a6b56 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.807065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594ba32-4c51-46eb-ae63-6fcdfa0a6b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.807065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594ba32-4c51-46eb-ae63-6fcdfa0a6b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.807965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.819816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594ba32-4c51-46eb-ae63-6fcdfa0a6b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:59.821598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594ba32-4c51-46eb-ae63-6fcdfa0a6b56 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:59.842145 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2852/5700] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:59 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:12:59.842687 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2848/5700] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:59 2022] PUT /identity/v3/projects/896f5e22b177497eb73cabac0c9ab6c4/users/201c6cfb271b43ba931ad686f62f25df/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:59.844302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155a9f9-f18b-4269-9ff2-e9eeace6d8d2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:59.844723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155a9f9-f18b-4269-9ff2-e9eeace6d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:59.845054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155a9f9-f18b-4269-9ff2-e9eeace6d8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:59.876795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f155a9f9-f18b-4269-9ff2-e9eeace6d8d2 None None] MFA Rules not processed for user `201c6cfb271b43ba931ad686f62f25df`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:59.878161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f155a9f9-f18b-4269-9ff2-e9eeace6d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:59.965508 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2853/5701] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:12:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:59.969486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.001888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155a9f9-f18b-4269-9ff2-e9eeace6d8d2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.023038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:00.053704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3461-648d-4846-8db6-63c37f526ae5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.058985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.060189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.060784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.061627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.062367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.064725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.078707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155a9f9-f18b-4269-9ff2-e9eeace6d8d2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.102607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.103967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.104615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.105326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] PATH_INFO: `/v3/users/ce0278bcbdf940eeb53c6bb58865a4de` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.107255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce0278bcbdf940eeb53c6bb58865a4de)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.107406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:00.111327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.120943 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2849/5702] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:59 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:00.122667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.125707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.174785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce0278bcbdf940eeb53c6bb58865a4de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.175637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce0278bcbdf940eeb53c6bb58865a4de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.197785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dff26-e32c-4ef9-8dad-0e66002dc980 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.203638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce0278bcbdf940eeb53c6bb58865a4de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.204564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.204947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce0278bcbdf940eeb53c6bb58865a4de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.205491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.205967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.206311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] PATH_INFO: `/v3/projects/17c435ee210a4e68bf13205656e1b3f9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.208131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c435ee210a4e68bf13205656e1b3f9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.210292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.224738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce0278bcbdf940eeb53c6bb58865a4de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.240460 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2854/5703] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:00 2022] DELETE /identity/v3/users/ce0278bcbdf940eeb53c6bb58865a4de => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:00.241360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.242732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.271169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c435ee210a4e68bf13205656e1b3f9', 'request_id': 'req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.281309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c435ee210a4e68bf13205656e1b3f9', 'request_id': 'req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.289944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c435ee210a4e68bf13205656e1b3f9', 'request_id': 'req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.304278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] The token cache is being invalidate because project 17c435ee210a4e68bf13205656e1b3f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:00.304525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.305653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f33c9cb7-7254-4d8f-8528-60afd2bb7cc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:00.307602 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2850/5704] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:00 2022] DELETE /identity/v3/projects/17c435ee210a4e68bf13205656e1b3f9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:00.308965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf30a1-2303-429d-b4e4-ba93c4250df3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.309257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf30a1-2303-429d-b4e4-ba93c4250df3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.309566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf30a1-2303-429d-b4e4-ba93c4250df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.321872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6460de65-902a-4007-a03c-1264fe9f9e48 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.332329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-13cf30a1-2303-429d-b4e4-ba93c4250df3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:00.334158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-13cf30a1-2303-429d-b4e4-ba93c4250df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.378478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:00.400219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.400855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.401193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.401494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.402331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.404907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.415173 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2851/5705] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:00.416732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.417075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.417510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.433793 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2855/5706] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:00.434852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.436851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.440943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:00.442180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.459181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2852/5707] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:00.459181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.460537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.473516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead38525-1ca6-4e22-a32a-978fc4442e8f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.479345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1594473-7753-4195-ad33-84a3dc278642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.480032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594473-7753-4195-ad33-84a3dc278642 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.480305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594473-7753-4195-ad33-84a3dc278642 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.480578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594473-7753-4195-ad33-84a3dc278642 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.481236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1594473-7753-4195-ad33-84a3dc278642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.482859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1594473-7753-4195-ad33-84a3dc278642 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.495002 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2853/5708] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:00.496682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6eda3-0c51-4ebe-b3e3-baf0542a9499 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.497218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6eda3-0c51-4ebe-b3e3-baf0542a9499 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.497717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6eda3-0c51-4ebe-b3e3-baf0542a9499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.514293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bc634-c148-439c-9d78-d2acb9f1ddd9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.518957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8ab6eda3-0c51-4ebe-b3e3-baf0542a9499 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:00.520200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8ab6eda3-0c51-4ebe-b3e3-baf0542a9499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.521501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.522408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.522870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.523298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.528352 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2854/5709] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:00.529174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6eda3-0c51-4ebe-b3e3-baf0542a9499 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.535582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.537250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.537783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:00.542377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6eda3-0c51-4ebe-b3e3-baf0542a9499 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.549434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2856/5710] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:00.550027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.554265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.557060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.557060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.557304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] PATH_INFO: `/v3/projects/6571dcaf068445f184188c21a59a4c7c/users/4d575d1e82324e249e4a2102372f8c1d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.561319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58625-fa4b-4efd-8118-f8069a6d572e service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.566367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.567020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.567299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.567652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.568587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:00.568964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.570773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.579347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6571dcaf068445f184188c21a59a4c7c, user_id=4d575d1e82324e249e4a2102372f8c1d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.582472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.587262 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2857/5711] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:00.592885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.603632 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2855/5712] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:00 2022] PUT /identity/v3/projects/6571dcaf068445f184188c21a59a4c7c/users/4d575d1e82324e249e4a2102372f8c1d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:00.605292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f3acc-42e6-49e9-88a2-ea01c22cc0e2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.607836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.690850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59998d9-6e65-46fa-a772-5c779aad7c63 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.702124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.703648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.704228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.705139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.705794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.706738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.706738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.706984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.707319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.707949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.709894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.710269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.728884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2856/5713] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:00.732460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.744607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608a634-4116-482d-8fc5-fb23c3e80513 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.750716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea84967-a794-4f01-95fc-ecce513481b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.751156 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2858/5714] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/projects => generated 359 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:00.751823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea84967-a794-4f01-95fc-ecce513481b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.751952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.752276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea84967-a794-4f01-95fc-ecce513481b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.752662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea84967-a794-4f01-95fc-ecce513481b9 admin admin] PATH_INFO: `/v3/projects/6571dcaf068445f184188c21a59a4c7c/users/4d575d1e82324e249e4a2102372f8c1d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.754691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea84967-a794-4f01-95fc-ecce513481b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6571dcaf068445f184188c21a59a4c7c, user_id=4d575d1e82324e249e4a2102372f8c1d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.757453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea84967-a794-4f01-95fc-ecce513481b9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.768802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2857/5715] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:00 2022] PUT /identity/v3/projects/6571dcaf068445f184188c21a59a4c7c/users/4d575d1e82324e249e4a2102372f8c1d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:00.779123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984727ad-ad11-4723-812c-6f868b06c693 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.779408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984727ad-ad11-4723-812c-6f868b06c693 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.779681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984727ad-ad11-4723-812c-6f868b06c693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.808692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:00.809602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-984727ad-ad11-4723-812c-6f868b06c693 None None] MFA Rules not processed for user `4d575d1e82324e249e4a2102372f8c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:00.810822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-984727ad-ad11-4723-812c-6f868b06c693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:00.826331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0b985-971d-4838-b6ab-2961a9eaa316 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.889872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.890555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.890829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.891212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.891740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.893745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.894194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2858/5716] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:00.898449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984727ad-ad11-4723-812c-6f868b06c693 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.910368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984727ad-ad11-4723-812c-6f868b06c693 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.916175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.917133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.917444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.917737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.918780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.920344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.925544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2859/5717] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/users => generated 280 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:00.928310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.943533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c493a-ba91-4ff4-92fe-202bb2ba0aa3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.950520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:00.951373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:00.951804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:00.952225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:00.953141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:00.954664 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2859/5718] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/projects => generated 350 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:00.955782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:00.959354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.970180 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2860/5719] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:00.974236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:00.986824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66458188-2187-4244-b2a7-097c90e34ed2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:00.993486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:01.036020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972928e-6357-4959-b386-ab6254e6f86a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:01.041985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:01.042709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.042994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.043445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.044232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:01.046445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:01.087637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:01.088351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.088613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.089595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.090375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:01.092220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:01.105338 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2861/5720] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:01.106410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:01.108162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:01.109803 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2860/5721] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:00 2022] POST /identity/v3/projects => generated 350 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:01.111218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:01.125735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e03f64-8753-4387-9ff8-f66099636590 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:01.229433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:01.230853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.231235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.232765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] PATH_INFO: `/v3/projects/09e45a9e6ff54bc7906dc9f7c60a4cfc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.235409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e45a9e6ff54bc7906dc9f7c60a4cfc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:01.239736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:01.254081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:01.297376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd10ce-4a20-4e99-af85-bb2340153df6 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:01.322676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e45a9e6ff54bc7906dc9f7c60a4cfc', 'request_id': 'req-13277f37-9b37-4fb5-9270-33af85666478'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.332273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e45a9e6ff54bc7906dc9f7c60a4cfc', 'request_id': 'req-13277f37-9b37-4fb5-9270-33af85666478'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.348480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e45a9e6ff54bc7906dc9f7c60a4cfc', 'request_id': 'req-13277f37-9b37-4fb5-9270-33af85666478'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.363294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] The token cache is being invalidate because project 09e45a9e6ff54bc7906dc9f7c60a4cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:01.364083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.364861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.366834 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2862/5722] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:01 2022] DELETE /identity/v3/projects/09e45a9e6ff54bc7906dc9f7c60a4cfc => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:01.370422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:01.374766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:01.406402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:01.408161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.408161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.408161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] PATH_INFO: `/v3/projects/49da478c23eb41498957705bf7a6286f/users/09de6b6b4ce345fd9886b163e96e0a26/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.427969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49da478c23eb41498957705bf7a6286f, user_id=09de6b6b4ce345fd9886b163e96e0a26, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:01.431513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:01.456991 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2861/5723] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:01 2022] PUT /identity/v3/projects/49da478c23eb41498957705bf7a6286f/users/09de6b6b4ce345fd9886b163e96e0a26/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:01.456991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:01.459650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:01.488788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277f37-9b37-4fb5-9270-33af85666478 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:01.513176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:01.547641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1857250-a68d-46ee-8420-ca086616b23f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:01.553854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:01.555175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.555613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.556186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] PATH_INFO: `/v3/users/25c8979b479c4c7ba588fac9b0ada3c2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.557928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c8979b479c4c7ba588fac9b0ada3c2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:01.560233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:01.563183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcef11-4b1b-432e-b493-85842ed597f5 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:01.563847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcef11-4b1b-432e-b493-85842ed597f5 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.564131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcef11-4b1b-432e-b493-85842ed597f5 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.564408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcef11-4b1b-432e-b493-85842ed597f5 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/users/201c6cfb271b43ba931ad686f62f25df` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.566357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcef11-4b1b-432e-b493-85842ed597f5 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:get_user(user_id=201c6cfb271b43ba931ad686f62f25df)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:01.570662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcef11-4b1b-432e-b493-85842ed597f5 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:01.577665 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2863/5724] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:13:01 2022] GET /identity/v3/users/201c6cfb271b43ba931ad686f62f25df => generated 288 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:01.579781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba593a-be94-4a74-97c6-33289a5c7385 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.580070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba593a-be94-4a74-97c6-33289a5c7385 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.580349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba593a-be94-4a74-97c6-33289a5c7385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.618990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ebba593a-be94-4a74-97c6-33289a5c7385 None None] MFA Rules not processed for user `09de6b6b4ce345fd9886b163e96e0a26`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:01.620636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ebba593a-be94-4a74-97c6-33289a5c7385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:01.642558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c8979b479c4c7ba588fac9b0ada3c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.658825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c8979b479c4c7ba588fac9b0ada3c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.671080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c8979b479c4c7ba588fac9b0ada3c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.674173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c8979b479c4c7ba588fac9b0ada3c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.674651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c8979b479c4c7ba588fac9b0ada3c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:01.681452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:01.709676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2862/5725] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:01 2022] DELETE /identity/v3/users/25c8979b479c4c7ba588fac9b0ada3c2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:01.710864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:01.713702 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2864/5726] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:01.717638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:01.718454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca254843-cd86-4c1e-abf3-6b0767cb2ef6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.718812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca254843-cd86-4c1e-abf3-6b0767cb2ef6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.719155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca254843-cd86-4c1e-abf3-6b0767cb2ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.742169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ca254843-cd86-4c1e-abf3-6b0767cb2ef6 None None] MFA Rules not processed for user `09de6b6b4ce345fd9886b163e96e0a26`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:01.743549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca254843-cd86-4c1e-abf3-6b0767cb2ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:01.749885 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2865/5727] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:01 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:01.755770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3e880-7622-44b8-a326-ec0a87b8cd58 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.756389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3e880-7622-44b8-a326-ec0a87b8cd58 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.757112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3e880-7622-44b8-a326-ec0a87b8cd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.806253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3ec3e880-7622-44b8-a326-ec0a87b8cd58 None None] MFA Rules not processed for user `09de6b6b4ce345fd9886b163e96e0a26`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:01.811717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3ec3e880-7622-44b8-a326-ec0a87b8cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:01.838155 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2866/5728] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:01 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:01.849507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3e880-7622-44b8-a326-ec0a87b8cd58 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:01.951984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cadff-6f72-4c91-a468-cf90695d8162 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:01.972712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:01.975980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:01.976325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:01.976608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:01.979314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:01.983359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.034991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3e880-7622-44b8-a326-ec0a87b8cd58 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.043924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09de6b6b4ce345fd9886b163e96e0a26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49da478c23eb41498957705bf7a6286f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.044835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.045304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.045828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.058504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2863/5729] 10.222.0.26 () {60 vars in 1162 bytes} [Sat Sep 17 21:13:01 2022] POST /identity/v3/credentials => generated 429 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:02.062695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorizing `identity:check_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.064884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.065577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.083761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961afbc-2b7f-44ea-8c3f-094b92236f8e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.085353 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2867/5730] 10.222.0.26 () {56 vars in 1264 bytes} [Sat Sep 17 21:13:01 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Sep 17 21:13:02.092186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.102144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.103648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.104834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.105332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials/88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.108930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.110775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.114891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.116823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.127659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c428d8-7aa5-4dad-954a-5750d6961102 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.138526 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2864/5731] 10.222.0.26 () {60 vars in 1358 bytes} [Sat Sep 17 21:13:02 2022] PATCH /identity/v3/credentials/88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a => generated 437 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:02.139581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.141127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09de6b6b4ce345fd9886b163e96e0a26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49da478c23eb41498957705bf7a6286f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.141946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.142326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.143265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.154570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f875c4-301d-4448-b66c-9c1e18faa7b8 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.158257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.163367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.169507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.173621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.175871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.181360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials/88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.184953 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2868/5732] 10.222.0.26 () {56 vars in 1263 bytes} [Sat Sep 17 21:13:02 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 21:13:02.187946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.190188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.192915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.194979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.208490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2865/5733] 10.222.0.26 () {58 vars in 1335 bytes} [Sat Sep 17 21:13:02 2022] GET /identity/v3/credentials/88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a => generated 437 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:02.209812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.242048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566ca3d-80e9-4158-a019-143951486d9a tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.247525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09de6b6b4ce345fd9886b163e96e0a26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49da478c23eb41498957705bf7a6286f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.249119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.250315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.250588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.260562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.262366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.283848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2866/5734] 10.222.0.26 () {56 vars in 1266 bytes} [Sat Sep 17 21:13:02 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:02.286161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.300043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be90be-18c2-4e63-ba04-f5210c396c78 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.306669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.307433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.307854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.308230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials/88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.310850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.312987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.341286 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2867/5735] 10.222.0.26 () {58 vars in 1338 bytes} [Sat Sep 17 21:13:02 2022] DELETE /identity/v3/credentials/88f2f141696f45c2ff718f92c3a76740866fcff96edc8a355c82d3fd254c2c3a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:02.342211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.354925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:02.362354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee61d2-5ce8-4670-9ad7-206a9af67caf tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.369749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09de6b6b4ce345fd9886b163e96e0a26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49da478c23eb41498957705bf7a6286f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.370491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] REQUEST_METHOD: `HEAD` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.374488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.374772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.378595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.409275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ae579-fbf4-4006-8a50-2bd6447165f7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.415944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.417367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.418099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.418358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] PATH_INFO: `/v3/projects/de09b8784fd04be0bee0843df58a3ca3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.420725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de09b8784fd04be0bee0843df58a3ca3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.423461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Sep 17 21:13:02.438022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 17 21:13:02.440263 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2868/5736] 10.222.0.26 () {56 vars in 1264 bytes} [Sat Sep 17 21:13:02 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 98 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) Sep 17 21:13:02.443952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.474451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4720-4d60-4d0d-8ae1-eb71fc80b9bd tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.484890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be04f15-e0ac-49b7-b33a-c543bb1eeaca tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.486211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be04f15-e0ac-49b7-b33a-c543bb1eeaca tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.486573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be04f15-e0ac-49b7-b33a-c543bb1eeaca tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.486946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be04f15-e0ac-49b7-b33a-c543bb1eeaca tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.487871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be04f15-e0ac-49b7-b33a-c543bb1eeaca tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.492218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be04f15-e0ac-49b7-b33a-c543bb1eeaca tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.534132 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2869/5737] 10.222.0.26 () {60 vars in 1162 bytes} [Sat Sep 17 21:13:02 2022] POST /identity/v3/credentials => generated 429 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:02.540021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171a0a3-bc93-4654-a530-a6eb968150d0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.540290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171a0a3-bc93-4654-a530-a6eb968150d0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.542544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171a0a3-bc93-4654-a530-a6eb968150d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.594790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3171a0a3-bc93-4654-a530-a6eb968150d0 None None] MFA Rules not processed for user `09de6b6b4ce345fd9886b163e96e0a26`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:02.596760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3171a0a3-bc93-4654-a530-a6eb968150d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.603433 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2870/5738] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:02 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:02.604912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171a0a3-bc93-4654-a530-a6eb968150d0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.615427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de09b8784fd04be0bee0843df58a3ca3', 'request_id': 'req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:02.634066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de09b8784fd04be0bee0843df58a3ca3', 'request_id': 'req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:02.638133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:02.654657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de09b8784fd04be0bee0843df58a3ca3', 'request_id': 'req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:02.680649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] The token cache is being invalidate because project de09b8784fd04be0bee0843df58a3ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:02.681087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:02.682148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:02.688465 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2869/5739] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:02 2022] DELETE /identity/v3/projects/de09b8784fd04be0bee0843df58a3ca3 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:02.688465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.690041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.804988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4c698-b57c-46e7-a4a4-68c9a42c9c3e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.813691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bb9f3-1711-419f-b146-9f49c03d1d5e tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09de6b6b4ce345fd9886b163e96e0a26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49da478c23eb41498957705bf7a6286f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.815649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bb9f3-1711-419f-b146-9f49c03d1d5e tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.816141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bb9f3-1711-419f-b146-9f49c03d1d5e tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.816966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bb9f3-1711-419f-b146-9f49c03d1d5e tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.818697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171a0a3-bc93-4654-a530-a6eb968150d0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.820337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-540bb9f3-1711-419f-b146-9f49c03d1d5e tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.825331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.826280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.826780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.827586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.828709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.831161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.835331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bb9f3-1711-419f-b146-9f49c03d1d5e tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.837495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bb9f3-1711-419f-b146-9f49c03d1d5e tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.845400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2870/5740] 10.222.0.26 () {56 vars in 1263 bytes} [Sat Sep 17 21:13:02 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 21:13:02.847651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7954-7ca0-4a0b-a779-edc0c34479e2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.847942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7954-7ca0-4a0b-a779-edc0c34479e2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.848367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7954-7ca0-4a0b-a779-edc0c34479e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.860252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2871/5741] 10.222.0.26 () {60 vars in 1162 bytes} [Sat Sep 17 21:13:02 2022] POST /identity/v3/credentials => generated 429 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:02.862096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.876230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9b477-872c-4efe-bd86-f1c6ee6908a7 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.878307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-276e7954-7ca0-4a0b-a779-edc0c34479e2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:02.879546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-276e7954-7ca0-4a0b-a779-edc0c34479e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.882202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09de6b6b4ce345fd9886b163e96e0a26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49da478c23eb41498957705bf7a6286f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.882818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.883097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.883369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.891921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.893583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.915978 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2872/5742] 10.222.0.26 () {56 vars in 1266 bytes} [Sat Sep 17 21:13:02 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:02.917322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.922068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.938313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a90ae-a65e-4c3d-877c-d2219b1afdfb tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.943683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.944391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.944733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.945004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.945656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.947210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.954905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.956582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.958154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.959724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.962198 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2873/5743] 10.222.0.26 () {58 vars in 1140 bytes} [Sat Sep 17 21:13:02 2022] GET /identity/v3/credentials => generated 943 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:02.962761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.969190 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2871/5744] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:02.970263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7954-7ca0-4a0b-a779-edc0c34479e2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.983030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e30e72-1ff7-4bac-9f66-ac4e593ce304 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.989667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7954-7ca0-4a0b-a779-edc0c34479e2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.994634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902175d-09db-48d2-8c13-2b07a38acfa7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09de6b6b4ce345fd9886b163e96e0a26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '49da478c23eb41498957705bf7a6286f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.995311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902175d-09db-48d2-8c13-2b07a38acfa7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.996046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902175d-09db-48d2-8c13-2b07a38acfa7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.996046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902175d-09db-48d2-8c13-2b07a38acfa7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.000321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.000905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.001188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.001505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials/94d1013728714f46dbd4fa95403012f4dc2317e7f2016cdd333d92a24eca3ca3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.001745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b902175d-09db-48d2-8c13-2b07a38acfa7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.003725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=94d1013728714f46dbd4fa95403012f4dc2317e7f2016cdd333d92a24eca3ca3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.005855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-b902175d-09db-48d2-8c13-2b07a38acfa7 tempest-TokensV3Test-503816514 tempest-TokensV3Test-503816514-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 17 21:13:03.016681 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 17 21:13:03.018117 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 17 21:13:03.019393 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:03.019393 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:03.019393 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 17 21:13:03.019393 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 17 21:13:03.019393 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 17 21:13:03.019393 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:03.022754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2874/5745] 10.222.0.26 () {56 vars in 1263 bytes} [Sat Sep 17 21:13:02 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 60 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 21:13:03.024121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7ba67-ce7c-4a18-ac75-d2df4eb33d57 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.024400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7ba67-ce7c-4a18-ac75-d2df4eb33d57 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.024669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7ba67-ce7c-4a18-ac75-d2df4eb33d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.034317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2872/5746] 10.222.0.26 () {58 vars in 1338 bytes} [Sat Sep 17 21:13:02 2022] DELETE /identity/v3/credentials/94d1013728714f46dbd4fa95403012f4dc2317e7f2016cdd333d92a24eca3ca3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:03.035546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.037802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.054104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-10b7ba67-ce7c-4a18-ac75-d2df4eb33d57 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:03.054988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f1463-d3c2-4f9f-827b-9440e0d54e6e tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.056085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-10b7ba67-ce7c-4a18-ac75-d2df4eb33d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.061147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.062716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.063064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.063428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] PATH_INFO: `/v3/users/09de6b6b4ce345fd9886b163e96e0a26` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.064832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09de6b6b4ce345fd9886b163e96e0a26)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.067348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2875/5747] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:03.068522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7ba67-ce7c-4a18-ac75-d2df4eb33d57 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.071154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.082892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7ba67-ce7c-4a18-ac75-d2df4eb33d57 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.089227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.090096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.090393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.091151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/credentials/13b2dfa4b920f75279b8f157009c518345b5c51b27c7b5bd14294122db592434` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.095246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=13b2dfa4b920f75279b8f157009c518345b5c51b27c7b5bd14294122db592434)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.097056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.116801 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2876/5748] 10.222.0.26 () {58 vars in 1338 bytes} [Sat Sep 17 21:13:03 2022] DELETE /identity/v3/credentials/13b2dfa4b920f75279b8f157009c518345b5c51b27c7b5bd14294122db592434 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:03.117680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.133752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38880-0cac-4249-a1b9-ea90d696e990 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.139067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.139429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09de6b6b4ce345fd9886b163e96e0a26'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.139679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.139954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.140101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09de6b6b4ce345fd9886b163e96e0a26'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.140215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.140858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:03.141159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.142819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.150560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09de6b6b4ce345fd9886b163e96e0a26'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.151478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09de6b6b4ce345fd9886b163e96e0a26'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.157105 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2877/5749] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:03.158189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.160092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.164297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:03.174477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09de6b6b4ce345fd9886b163e96e0a26'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.180140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdc77a-7c72-45ce-ad5a-ddd51da1ce85 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.185412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.186088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.186371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.186793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/projects/2047079495bf408890951f9a52815b38` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.187956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2047079495bf408890951f9a52815b38)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.189941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.200460 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2873/5750] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:03 2022] DELETE /identity/v3/users/09de6b6b4ce345fd9886b163e96e0a26 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:03.201879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.204053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.260433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2047079495bf408890951f9a52815b38', 'request_id': 'req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.263303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:03.286645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2047079495bf408890951f9a52815b38', 'request_id': 'req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.308101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2047079495bf408890951f9a52815b38', 'request_id': 'req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.320945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] The token cache is being invalidate because project 2047079495bf408890951f9a52815b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:03.321233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.321932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf2f8fc9-08e8-4963-9a8a-9cb757f2d017 tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.325558 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2878/5751] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:03 2022] DELETE /identity/v3/projects/2047079495bf408890951f9a52815b38 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:03.325558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.325558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.325890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.359653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:03.361607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.361969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71ae7a-75d8-4154-89ee-fde77e095a20 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.439833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.440381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.440658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.440927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.441762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.444683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.449812 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2879/5752] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:03.449812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.451811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.493245 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2874/5753] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:03 2022] POST /identity/v3/projects => generated 357 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:03.493245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.493583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.555405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06e4f4-fe17-4e2f-b08c-20da97ccf4e9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.581419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1a1d5-6956-489f-82ba-1b5bb2ed8204 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.585327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.586176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.586454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.586858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.614808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d575d1e82324e249e4a2102372f8c1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6571dcaf068445f184188c21a59a4c7c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.615563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.615913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.617334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] PATH_INFO: `/v3/projects/10550c5af4664467814db9bafe8754ea` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.617485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorizing `identity:delete_project(project_id=10550c5af4664467814db9bafe8754ea)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.619433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.620720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:03.659805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:03.664000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.665845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.680156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10550c5af4664467814db9bafe8754ea', 'request_id': 'req-19ac75bd-926e-407b-adbb-d0060337911b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.683802 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2875/5754] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:03.684547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.686922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.691714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10550c5af4664467814db9bafe8754ea', 'request_id': 'req-19ac75bd-926e-407b-adbb-d0060337911b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.705179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10550c5af4664467814db9bafe8754ea', 'request_id': 'req-19ac75bd-926e-407b-adbb-d0060337911b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.707225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba435-671e-4aca-b655-5896a82ba456 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.722278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] The token cache is being invalidate because project 10550c5af4664467814db9bafe8754ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:03.722625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.723409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-19ac75bd-926e-407b-adbb-d0060337911b tempest-CredentialsTestJSON-796507757 tempest-CredentialsTestJSON-796507757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:03.727953 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2880/5755] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:03 2022] DELETE /identity/v3/projects/10550c5af4664467814db9bafe8754ea => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:03.728147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.729140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.729868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.797499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa7cec-db5b-4c91-885c-636b4abd48a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.798197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa7cec-db5b-4c91-885c-636b4abd48a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.798469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa7cec-db5b-4c91-885c-636b4abd48a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.798735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa7cec-db5b-4c91-885c-636b4abd48a6 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.799663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa7cec-db5b-4c91-885c-636b4abd48a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.801482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa7cec-db5b-4c91-885c-636b4abd48a6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.830720 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2876/5756] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:03 2022] POST /identity/v3/users => generated 279 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:03.832268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.832549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.832886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.846253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:03.847645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.859523 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2881/5757] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:03 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:03.859523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:03.860147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.879029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] MFA Rules not processed for user `2db8955202fe42d2a4c86a15b193787f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:03.881353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:03.892732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e6a95-dcd3-431a-ba90-4086e30abf4c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:03.907173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:03.908067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:03.908614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:03.909073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] PATH_INFO: `/v3/users/201c6cfb271b43ba931ad686f62f25df` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:03.910520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201c6cfb271b43ba931ad686f62f25df)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:03.912803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:03.989175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201c6cfb271b43ba931ad686f62f25df'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.007226 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2877/5758] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:04.008125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.013423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.041707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4095-2e4c-48e4-9ac8-50c8c3b3de4c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.042551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201c6cfb271b43ba931ad686f62f25df'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.046990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.047897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.048275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.048716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.049639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.051635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.067012 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2878/5759] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:04.070082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.071047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.072221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201c6cfb271b43ba931ad686f62f25df'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.074362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201c6cfb271b43ba931ad686f62f25df'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.077025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fa7161e-5fdb-4d44-b365-e2db2fd6c8a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201c6cfb271b43ba931ad686f62f25df'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.106865 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2882/5760] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:03 2022] DELETE /identity/v3/users/201c6cfb271b43ba931ad686f62f25df => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:04.108475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.108761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.109033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.145023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:04.153310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] MFA Rules not processed for user `2db8955202fe42d2a4c86a15b193787f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:04.154689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.203504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ad36-12fc-4ff7-b92b-458813c2bc92 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.288937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec0249-b124-4cf2-834e-93007be8df31 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.289633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec0249-b124-4cf2-834e-93007be8df31 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.289936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec0249-b124-4cf2-834e-93007be8df31 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.290209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec0249-b124-4cf2-834e-93007be8df31 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.293616 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2883/5761] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:04.293616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.295261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.324118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:04.345241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1dec0249-b124-4cf2-834e-93007be8df31 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.354818 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2879/5762] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:04.356973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.357794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.358265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.377437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:04.378665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.380455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41a07-6f1b-4d0f-8d7e-da17d35fe957 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.384979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.385624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.385933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.386241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.386687 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2880/5763] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:04.387008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.388132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.388675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.389563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.402156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ddc34-5184-4138-9174-fa8cff2c9450 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.403585 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2884/5764] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:04.404303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.405970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.407266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.407890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.408161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.408426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] PATH_INFO: `/v3/projects/49da478c23eb41498957705bf7a6286f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.409544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49da478c23eb41498957705bf7a6286f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.411446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.462774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49da478c23eb41498957705bf7a6286f', 'request_id': 'req-1e44c8cb-5764-4200-b612-a4d72bda8503'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.469013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49da478c23eb41498957705bf7a6286f', 'request_id': 'req-1e44c8cb-5764-4200-b612-a4d72bda8503'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.477846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49da478c23eb41498957705bf7a6286f', 'request_id': 'req-1e44c8cb-5764-4200-b612-a4d72bda8503'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.489052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] The token cache is being invalidate because project 49da478c23eb41498957705bf7a6286f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:04.489339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.490002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:04.491628 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2881/5765] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:04 2022] DELETE /identity/v3/projects/49da478c23eb41498957705bf7a6286f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:04.496905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.496905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.498978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7371a2-e68a-4f1b-8ef4-2a64e8639f83 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.531252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:04.559441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c8cb-5764-4200-b612-a4d72bda8503 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.563252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.563895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.564289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.564599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.566265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40df273-101c-41c5-817f-732153c449e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.566420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.566905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df273-101c-41c5-817f-732153c449e1 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.567178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df273-101c-41c5-817f-732153c449e1 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.567449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df273-101c-41c5-817f-732153c449e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.569170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d40df273-101c-41c5-817f-732153c449e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.605561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:04.635494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.636541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40df273-101c-41c5-817f-732153c449e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.637029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.638517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40df273-101c-41c5-817f-732153c449e1 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.648351 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2882/5766] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:04.649245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.649842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2885/5767] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:04 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:04.650876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.659420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1fcf0-debc-4b3b-b008-b38e8f620990 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.660015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1fcf0-debc-4b3b-b008-b38e8f620990 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.660519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1fcf0-debc-4b3b-b008-b38e8f620990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.663569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2121ac-146c-491d-8905-feba11236dc1 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.670507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb307b3-c0d4-48cc-bcd3-1afff7569162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.671591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb307b3-c0d4-48cc-bcd3-1afff7569162 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.671878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb307b3-c0d4-48cc-bcd3-1afff7569162 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.672149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb307b3-c0d4-48cc-bcd3-1afff7569162 admin admin] PATH_INFO: `/v3/projects/9dd370c8e76749f69f6075678713bd39/users/a61d2ca32f8c48cdb67b10fdced8a721/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.693732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb307b3-c0d4-48cc-bcd3-1afff7569162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd370c8e76749f69f6075678713bd39, user_id=a61d2ca32f8c48cdb67b10fdced8a721, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.697301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb307b3-c0d4-48cc-bcd3-1afff7569162 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.703164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-afa1fcf0-debc-4b3b-b008-b38e8f620990 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:04.705635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-afa1fcf0-debc-4b3b-b008-b38e8f620990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.726777 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2883/5769] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:04 2022] PUT /identity/v3/projects/9dd370c8e76749f69f6075678713bd39/users/a61d2ca32f8c48cdb67b10fdced8a721/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:04.727242 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2886/5769] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:04.728677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bea94-2822-411e-9007-d43126e1e499 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.728971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bea94-2822-411e-9007-d43126e1e499 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.729251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bea94-2822-411e-9007-d43126e1e499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.733284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf7ca3-6531-4c97-bb70-610ef0ccc540 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.733607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf7ca3-6531-4c97-bb70-610ef0ccc540 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.735650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf7ca3-6531-4c97-bb70-610ef0ccc540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.756942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-284bea94-2822-411e-9007-d43126e1e499 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:04.758628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-284bea94-2822-411e-9007-d43126e1e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.778214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-17cf7ca3-6531-4c97-bb70-610ef0ccc540 None None] MFA Rules not processed for user `a61d2ca32f8c48cdb67b10fdced8a721`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:04.779423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17cf7ca3-6531-4c97-bb70-610ef0ccc540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:04.854341 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2884/5770] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:04.858510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bea94-2822-411e-9007-d43126e1e499 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.871958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bea94-2822-411e-9007-d43126e1e499 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.881915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.882549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.882831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.883100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.883756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:04.884104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.885958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.911285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2887/5771] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:04.916925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf7ca3-6531-4c97-bb70-610ef0ccc540 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.923476 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2885/5772] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:04.927218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.928736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf7ca3-6531-4c97-bb70-610ef0ccc540 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.933641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.934260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.934538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.934807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.935443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.938035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.942746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c21d3b-0ab5-41cd-bd56-321b78a7bb4a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:04.948752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:04.949970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:04.950304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:04.950577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:04.951247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:04.953178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:04.972929 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2888/5773] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:04.977081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:04.992527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2886/5774] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/projects => generated 359 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:04.993719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.029928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:05.064073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eeaa-0e96-451d-832e-9f41a4dfa4eb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.078016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7203c4-c5e5-4111-858c-8a0b32ec7e3e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.084428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.085143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.085491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.085914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.086617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.090516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.113991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:05.127493 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2887/5775] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/users => generated 280 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:05.133938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.140896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.141555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.141858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.142154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.142717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.145056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.152668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb275-b7cb-4951-9f91-56579d4034e3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.165025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.166689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.166689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.167049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.170060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.171476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.196109 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2889/5776] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:04 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:05.197624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.200648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:05.222456 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2888/5777] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:05.222456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.231552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39948e-20d2-400a-abef-ae9f5a3d4d41 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.235580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c071e0f-842a-4162-8b4e-01e647467634 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.242192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.243475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.243946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.244471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] PATH_INFO: `/v3/projects/896f5e22b177497eb73cabac0c9ab6c4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.246244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] RBAC: Authorizing `identity:delete_project(project_id=896f5e22b177497eb73cabac0c9ab6c4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.246818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.248321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.249596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.250296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.251797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.252304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.256254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.286247 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2889/5778] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:05.291035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.313309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a489f0c-6c66-4f6e-bb00-257e5ed34963 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.318793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.319781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.320259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.320717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.321749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.323688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.337476 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2890/5779] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:05.339639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.345627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896f5e22b177497eb73cabac0c9ab6c4', 'request_id': 'req-f474a5a6-ed32-437f-9175-efec3ac6d08a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.353054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4e81b-497f-4834-b4b2-14a367589b80 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.360127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896f5e22b177497eb73cabac0c9ab6c4', 'request_id': 'req-f474a5a6-ed32-437f-9175-efec3ac6d08a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.370967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896f5e22b177497eb73cabac0c9ab6c4', 'request_id': 'req-f474a5a6-ed32-437f-9175-efec3ac6d08a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.388291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] The token cache is being invalidate because project 896f5e22b177497eb73cabac0c9ab6c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:05.388674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.389421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.400103 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2890/5780] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:05 2022] DELETE /identity/v3/projects/896f5e22b177497eb73cabac0c9ab6c4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:05.401079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.402907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:05.444726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1284ec-c50b-4330-9fc9-47a0c68d0615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.445415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1284ec-c50b-4330-9fc9-47a0c68d0615 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.446561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1284ec-c50b-4330-9fc9-47a0c68d0615 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.446979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1284ec-c50b-4330-9fc9-47a0c68d0615 admin admin] PATH_INFO: `/v3/projects/f7c2bd2962c647748617610d30f56648/users/536fbff9200d4e74a74cdd9b1a67fa48/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.467807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1284ec-c50b-4330-9fc9-47a0c68d0615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c2bd2962c647748617610d30f56648, user_id=536fbff9200d4e74a74cdd9b1a67fa48, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.470992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1284ec-c50b-4330-9fc9-47a0c68d0615 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.491548 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2891/5781] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:05 2022] PUT /identity/v3/projects/f7c2bd2962c647748617610d30f56648/users/536fbff9200d4e74a74cdd9b1a67fa48/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:05.493041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.493393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.493879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.519105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:05.521496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:05.526181 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2892/5782] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:05 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:05.527126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.531356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:05.534233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a5a6-ed32-437f-9175-efec3ac6d08a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.632954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.634416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.634713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.635064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] PATH_INFO: `/v3/projects/0eb5b150988543af85be0d0b76400125/users/78114d4742f64d73a6639b4a3d27e110/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.644426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:05.646568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:05.653497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb5b150988543af85be0d0b76400125, user_id=78114d4742f64d73a6639b4a3d27e110, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.656171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd70f3b-bccf-4dfb-a9f3-9b21d5fa009a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.658967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.664039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.665064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.665734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.666220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] PATH_INFO: `/v3/users/4d575d1e82324e249e4a2102372f8c1d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.668291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d575d1e82324e249e4a2102372f8c1d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.679290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.702872 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2891/5783] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:05 2022] PUT /identity/v3/projects/0eb5b150988543af85be0d0b76400125/users/78114d4742f64d73a6639b4a3d27e110/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:05.703756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:05.713440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:05.742557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d575d1e82324e249e4a2102372f8c1d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.743306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d575d1e82324e249e4a2102372f8c1d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.753915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d575d1e82324e249e4a2102372f8c1d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.755088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d575d1e82324e249e4a2102372f8c1d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.771955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82c040a5-99dd-4d7f-afa2-d5ccf85ba5b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d575d1e82324e249e4a2102372f8c1d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:05.790255 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2893/5784] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:05 2022] DELETE /identity/v3/users/4d575d1e82324e249e4a2102372f8c1d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:05.791439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b9261-8b37-40ec-b6b2-e00854c16a8e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.791439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b9261-8b37-40ec-b6b2-e00854c16a8e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.791439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b9261-8b37-40ec-b6b2-e00854c16a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.799405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-065b9261-8b37-40ec-b6b2-e00854c16a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:05.821417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad60c23-5e89-477c-aec1-1375142a9368 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-065b9261-8b37-40ec-b6b2-e00854c16a8e None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Sep 17 21:13:05.914870 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:05.916485 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2894/5785] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:05 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 129 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 21:13:05.918170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5af7-8f78-4ed7-8e6e-451b4f4bc5b8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.919445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5af7-8f78-4ed7-8e6e-451b4f4bc5b8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.919445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5af7-8f78-4ed7-8e6e-451b4f4bc5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.946270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a8590-ff26-4793-8ed1-c89607033f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:05.946892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a8590-ff26-4793-8ed1-c89607033f55 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.947299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a8590-ff26-4793-8ed1-c89607033f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.947447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a8590-ff26-4793-8ed1-c89607033f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:05.948232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a8590-ff26-4793-8ed1-c89607033f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:05.949984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a8590-ff26-4793-8ed1-c89607033f55 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:05.963907 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2892/5786] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:05.965271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089665c-be31-494a-bec0-731da22c2256 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:05.965564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089665c-be31-494a-bec0-731da22c2256 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:05.965870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089665c-be31-494a-bec0-731da22c2256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.009812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-44bd5af7-8f78-4ed7-8e6e-451b4f4bc5b8 None None] MFA Rules not processed for user `78114d4742f64d73a6639b4a3d27e110`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:06.011246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-44bd5af7-8f78-4ed7-8e6e-451b4f4bc5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.028375 ncdv-master-1 devstack@keystone.service[99166]: WARNING keystone.server.flask.application [None req-9089665c-be31-494a-bec0-731da22c2256 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 17 21:13:06.033632 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2893/5787] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:05 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 69 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 17 21:13:06.034606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089665c-be31-494a-bec0-731da22c2256 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:06.036439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9089665c-be31-494a-bec0-731da22c2256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.053994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089665c-be31-494a-bec0-731da22c2256 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:06.059299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cabd7-5580-4b8c-b6ad-67f661db1a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:06.059991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cabd7-5580-4b8c-b6ad-67f661db1a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.060276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cabd7-5580-4b8c-b6ad-67f661db1a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.060554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cabd7-5580-4b8c-b6ad-67f661db1a8c admin admin] PATH_INFO: `/v3/projects/f7c2bd2962c647748617610d30f56648/users/536fbff9200d4e74a74cdd9b1a67fa48/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.063155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cabd7-5580-4b8c-b6ad-67f661db1a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c2bd2962c647748617610d30f56648, user_id=536fbff9200d4e74a74cdd9b1a67fa48, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:06.066874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cabd7-5580-4b8c-b6ad-67f661db1a8c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:06.094161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2894/5788] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:06 2022] PUT /identity/v3/projects/f7c2bd2962c647748617610d30f56648/users/536fbff9200d4e74a74cdd9b1a67fa48/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:06.097094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.098461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.098761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.149779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:06.157140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.171713 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2895/5789] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:06.186901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c158f4-7098-406d-9491-3592ac8d5174 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.188753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c158f4-7098-406d-9491-3592ac8d5174 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.189052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c158f4-7098-406d-9491-3592ac8d5174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.244084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:06.249198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d3c158f4-7098-406d-9491-3592ac8d5174 None None] MFA Rules not processed for user `536fbff9200d4e74a74cdd9b1a67fa48`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:06.250449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d3c158f4-7098-406d-9491-3592ac8d5174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.292119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:UPDATE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:06.338819 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2895/5790] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:06.340636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:06.342574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.384429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2896/5791] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:06.385525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c158f4-7098-406d-9491-3592ac8d5174 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:06.404387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c158f4-7098-406d-9491-3592ac8d5174 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:06.409580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:06.410300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.410579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.410890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.445174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2a7a5-c4e7-469b-8732-54ec26b3d6e7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:06.450234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:06.450962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.451318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.451676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] PATH_INFO: `/v3/users/78114d4742f64d73a6639b4a3d27e110` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.453445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] RBAC: Authorizing `identity:delete_user(user_id=78114d4742f64d73a6639b4a3d27e110)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:06.455848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:06.478634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.490636 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2897/5792] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:13:06 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:06.493183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:06.495578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.510851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1e7ac-65a7-4034-8d40-babe40dc994b tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:06.516066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:06.516884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.517258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.517685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] PATH_INFO: `/v3/projects/6571dcaf068445f184188c21a59a4c7c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.519003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6571dcaf068445f184188c21a59a4c7c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:06.521068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:06.554588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78114d4742f64d73a6639b4a3d27e110'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.582839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78114d4742f64d73a6639b4a3d27e110'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.596274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6571dcaf068445f184188c21a59a4c7c', 'request_id': 'req-4b20353c-4d8f-4a19-8dee-d9b9011d622d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.602640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78114d4742f64d73a6639b4a3d27e110'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.603274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78114d4742f64d73a6639b4a3d27e110'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.603946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78114d4742f64d73a6639b4a3d27e110'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.618071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6571dcaf068445f184188c21a59a4c7c', 'request_id': 'req-4b20353c-4d8f-4a19-8dee-d9b9011d622d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.625274 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2896/5793] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:06 2022] DELETE /identity/v3/users/78114d4742f64d73a6639b4a3d27e110 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:06.626175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:06.639921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.641904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6571dcaf068445f184188c21a59a4c7c', 'request_id': 'req-4b20353c-4d8f-4a19-8dee-d9b9011d622d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.669075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] The token cache is being invalidate because project 6571dcaf068445f184188c21a59a4c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:06.669075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.670566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4b20353c-4d8f-4a19-8dee-d9b9011d622d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:06.672984 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2898/5794] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:06 2022] DELETE /identity/v3/projects/6571dcaf068445f184188c21a59a4c7c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:06.674484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880fc3f-486c-4f7c-b6aa-28d2b016fa58 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.674858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880fc3f-486c-4f7c-b6aa-28d2b016fa58 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.675224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880fc3f-486c-4f7c-b6aa-28d2b016fa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.705830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8880fc3f-486c-4f7c-b6aa-28d2b016fa58 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:06.707868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8880fc3f-486c-4f7c-b6aa-28d2b016fa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.792860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa21169-0000-4b79-9830-8c5f1c90e7af admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:06.796848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b284791-d256-42cd-bd4a-a6bdfc638431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:06.797494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b284791-d256-42cd-bd4a-a6bdfc638431 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.797852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b284791-d256-42cd-bd4a-a6bdfc638431 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.798053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b284791-d256-42cd-bd4a-a6bdfc638431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.801394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6b284791-d256-42cd-bd4a-a6bdfc638431 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.817515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:06.834417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:06.845044 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2899/5795] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:06.846547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.846832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.847105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.867592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:06.868836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.877651 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2900/5796] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:06.878347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:06.879584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.898595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b284791-d256-42cd-bd4a-a6bdfc638431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:06.900273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b284791-d256-42cd-bd4a-a6bdfc638431 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:06.911407 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2897/5797] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:06.912881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cc87d-a9e8-4f55-a392-1bb46832d4f3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.913266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cc87d-a9e8-4f55-a392-1bb46832d4f3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.913660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cc87d-a9e8-4f55-a392-1bb46832d4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.935781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-953cc87d-a9e8-4f55-a392-1bb46832d4f3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:06.937326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-953cc87d-a9e8-4f55-a392-1bb46832d4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:06.946000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95afd-07e6-45d1-9674-fad9095c14b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:06.949857 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2898/5798] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:06.950103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:06.950878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.951161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.951431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.960967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cc87d-a9e8-4f55-a392-1bb46832d4f3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:06.964189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:06.965698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:06.976268 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2901/5799] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:06.977804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cc87d-a9e8-4f55-a392-1bb46832d4f3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:06.988400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:06.989267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:06.989613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:06.989887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:06.990715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:06.991016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:06.992679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.012318 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2899/5800] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:07.012525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.032555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaa3ad-2aaa-45bf-be7f-573b6a90326f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:07.053303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:07.054584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.055074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.055661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.056503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:07.059264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.066348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.068457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:07.087413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54de80-4bd6-4d9d-89c2-d53575e89535 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:07.118979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:07.121237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.121889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.122191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.128673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:07.136190 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2902/5801] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:07 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:07.138423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.299582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:07.301057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.320506 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2900/5802] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:07.337725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a94e3e-1beb-4dce-9d5c-10b229af30c8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:07.355970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:07.357101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.357628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.358153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.359022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:07.362238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.410499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2903/5803] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:07 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:07.414773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.430806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca4b64-b365-4e21-902c-3e0fffeecf79 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:07.438285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c4cabf-8495-45ca-9bfd-9c527d5f11ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:07.439181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4cabf-8495-45ca-9bfd-9c527d5f11ee admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.440227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4cabf-8495-45ca-9bfd-9c527d5f11ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.440501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4cabf-8495-45ca-9bfd-9c527d5f11ee admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.442292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4cabf-8495-45ca-9bfd-9c527d5f11ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:07.445141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c4cabf-8495-45ca-9bfd-9c527d5f11ee admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.466201 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2901/5804] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:07.473078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.496149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.496418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.496706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.497285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9750c-0ed6-4e5f-868d-b5c402a9f7d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:07.521041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c267-7f27-4ce9-ba16-06099dcae26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:07.523090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c267-7f27-4ce9-ba16-06099dcae26c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.523090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c267-7f27-4ce9-ba16-06099dcae26c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.523090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c267-7f27-4ce9-ba16-06099dcae26c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.525893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c267-7f27-4ce9-ba16-06099dcae26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:07.536249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:07.537819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:07.540964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c267-7f27-4ce9-ba16-06099dcae26c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.542338 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2902/5805] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:07 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:07.544124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.551469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:07.561282 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2904/5806] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:07.562575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41ceb-e491-469e-9c45-38618affc841 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.562853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41ceb-e491-469e-9c45-38618affc841 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.563122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41ceb-e491-469e-9c45-38618affc841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.583819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08312987-2ed6-47ad-8289-e877dbfaf313 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:07.609155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:07.609846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.610126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.610400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] PATH_INFO: `/v3/projects/0eb5b150988543af85be0d0b76400125` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.615609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb5b150988543af85be0d0b76400125)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:07.617903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.621647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-85e41ceb-e491-469e-9c45-38618affc841 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:07.623048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85e41ceb-e491-469e-9c45-38618affc841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:07.710998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:07.731001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb5b150988543af85be0d0b76400125', 'request_id': 'req-97dee7dc-778e-4171-8684-21d40fd3b6aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:07.745192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb5b150988543af85be0d0b76400125', 'request_id': 'req-97dee7dc-778e-4171-8684-21d40fd3b6aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:07.755245 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2905/5807] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:07.756222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41ceb-e491-469e-9c45-38618affc841 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.759644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb5b150988543af85be0d0b76400125', 'request_id': 'req-97dee7dc-778e-4171-8684-21d40fd3b6aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:07.773239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41ceb-e491-469e-9c45-38618affc841 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:07.787336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] The token cache is being invalidate because project 0eb5b150988543af85be0d0b76400125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:07.787630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:07.790864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:07.804344 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2903/5808] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:07 2022] DELETE /identity/v3/projects/0eb5b150988543af85be0d0b76400125 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:07.804344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:07.809205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:07.847710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:07.879480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbdc6e-4c31-4e9f-b100-124464b4cbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:07.880175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbdc6e-4c31-4e9f-b100-124464b4cbda admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.880452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbdc6e-4c31-4e9f-b100-124464b4cbda admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.880724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbdc6e-4c31-4e9f-b100-124464b4cbda admin admin] PATH_INFO: `/v3/projects/42783e4d01654ae7a43b8819a8a91b66/users/063583139dca4c2d94ac19baf096c122/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.903448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbdc6e-4c31-4e9f-b100-124464b4cbda admin admin] RBAC: Authorizing `identity:create_grant(project_id=42783e4d01654ae7a43b8819a8a91b66, user_id=063583139dca4c2d94ac19baf096c122, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:07.909449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbdc6e-4c31-4e9f-b100-124464b4cbda admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:07.939044 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2906/5809] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:07 2022] PUT /identity/v3/projects/42783e4d01654ae7a43b8819a8a91b66/users/063583139dca4c2d94ac19baf096c122/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:07.940135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7393c44-ace5-4acc-b29a-fd8857c51655 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:07.940423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7393c44-ace5-4acc-b29a-fd8857c51655 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:07.940694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7393c44-ace5-4acc-b29a-fd8857c51655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:07.980081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d7393c44-ace5-4acc-b29a-fd8857c51655 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:07.981506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d7393c44-ace5-4acc-b29a-fd8857c51655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:07.984296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dee7dc-778e-4171-8684-21d40fd3b6aa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.087259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c76834-0254-46e3-8c63-4950dc20c5e9 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.087928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c76834-0254-46e3-8c63-4950dc20c5e9 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.088209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c76834-0254-46e3-8c63-4950dc20c5e9 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.088487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c76834-0254-46e3-8c63-4950dc20c5e9 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.129162 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2907/5810] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:08.132037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.132529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.132983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.181035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] MFA Rules not processed for user `063583139dca4c2d94ac19baf096c122`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:08.186712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85c76834-0254-46e3-8c63-4950dc20c5e9 tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:08.187223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:08.218490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2904/5811] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:13:07 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:08.223564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55597-2e67-4ea0-bed9-8ddd8301fd64 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.224099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55597-2e67-4ea0-bed9-8ddd8301fd64 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.224400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a55597-2e67-4ea0-bed9-8ddd8301fd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.263994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b4a55597-2e67-4ea0-bed9-8ddd8301fd64 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:08.265260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b4a55597-2e67-4ea0-bed9-8ddd8301fd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:08.281403 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2905/5812] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:08.285712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a55597-2e67-4ea0-bed9-8ddd8301fd64 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.302016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a55597-2e67-4ea0-bed9-8ddd8301fd64 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.312181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.313048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.313367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.313795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.314652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:08.314945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.316788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.334603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:08.335554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2906/5813] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:13:08.339182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.354028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf0cc8-c4ef-4a67-9e2d-2809f82adbd7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.365546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.367188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.367530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.367841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.368476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.371824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.374138 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2908/5814] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:08.378667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.380220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:08.413264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4243a-c67f-42d1-af60-35a5c9187b8e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.420894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.421643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.422136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.422417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.423480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.427423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.434041 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2907/5815] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:08 2022] POST /identity/v3/projects => generated 389 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:08.435251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.467692 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2909/5816] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:08 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:08.472055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.508404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:08.524577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87fcd3-7288-454f-b95b-e5a73e4e2f69 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.579523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43293a98-bb4a-40be-9ba4-8a430ecc7b4e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.585434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.586653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.586934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.587252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.587900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.589981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.608650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.609296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.609742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.610028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.610768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.612773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.626697 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2910/5817] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:08 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:08.630900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.645229 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2908/5818] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:08 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:08.645498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6c118-5208-4d54-baf0-fb0aa6844a3c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.646878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.648428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.649046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.649374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.649688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.650342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.652053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.659325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05368b6-ea22-43d7-bf19-6bbd945d7413 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.666617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.668072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.668485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.668859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.669659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.671972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.673409 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2911/5819] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:08.680109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.706021 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2909/5820] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:08.707672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.710627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c08cfe-1bc0-490c-8882-6762e8c7a924 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.717490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.718207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.718486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.718763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] PATH_INFO: `/v3/projects/3b5f9b61fb314906bccb0d6de96ab503/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.723184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e019d-d441-475b-8df2-6d4861a3a2f9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.730313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.732158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.732952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.733848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.734528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.736181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.742120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5f9b61fb314906bccb0d6de96ab503, user_id=5085205c047641888857281b4a6ea37a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.744849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.760077 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2910/5821] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:08.763263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.773396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2912/5822] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:08 2022] PUT /identity/v3/projects/3b5f9b61fb314906bccb0d6de96ab503/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:08.774474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.781301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fbef5-a907-4cde-91d2-0e3dbe5f213a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.867169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadf402-c4a9-4dea-8b3d-809e4992df26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.867856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadf402-c4a9-4dea-8b3d-809e4992df26 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.868126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadf402-c4a9-4dea-8b3d-809e4992df26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.868405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadf402-c4a9-4dea-8b3d-809e4992df26 admin admin] PATH_INFO: `/v3/projects/5644b74ce00c42719cbdfcecd732b06c/users/7e084d52162e4b1ba2f5ecfe1a6a4d0e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.890621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8d3c-827c-43b1-ad97-e31a561d2815 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.895948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadf402-c4a9-4dea-8b3d-809e4992df26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5644b74ce00c42719cbdfcecd732b06c, user_id=7e084d52162e4b1ba2f5ecfe1a6a4d0e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.901074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadf402-c4a9-4dea-8b3d-809e4992df26 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.901703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:08.902542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.902797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.903197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.903908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:08.905767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:08.922934 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2913/5823] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:08.926918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:08.929942 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2911/5824] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:08 2022] PUT /identity/v3/projects/5644b74ce00c42719cbdfcecd732b06c/users/7e084d52162e4b1ba2f5ecfe1a6a4d0e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:08.941419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085610e8-7942-470f-a83f-4db02be97a17 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:08.941785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085610e8-7942-470f-a83f-4db02be97a17 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:08.942066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085610e8-7942-470f-a83f-4db02be97a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:08.967274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e544e-d20b-4b1c-a656-f2554f3504b1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:08.991357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-085610e8-7942-470f-a83f-4db02be97a17 None None] MFA Rules not processed for user `7e084d52162e4b1ba2f5ecfe1a6a4d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:08.992654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-085610e8-7942-470f-a83f-4db02be97a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:09.004907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:09.045345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67494abe-cc1b-4ecc-91fc-7a75422b66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.046183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67494abe-cc1b-4ecc-91fc-7a75422b66a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.046595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67494abe-cc1b-4ecc-91fc-7a75422b66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.046872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67494abe-cc1b-4ecc-91fc-7a75422b66a2 admin admin] PATH_INFO: `/v3/projects/3b5f9b61fb314906bccb0d6de96ab503/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.049679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67494abe-cc1b-4ecc-91fc-7a75422b66a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5f9b61fb314906bccb0d6de96ab503, user_id=5085205c047641888857281b4a6ea37a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.051053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:09.053172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67494abe-cc1b-4ecc-91fc-7a75422b66a2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.068017 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2914/5825] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:08 2022] PUT /identity/v3/projects/3b5f9b61fb314906bccb0d6de96ab503/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:09.078984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb1152-9a07-4753-989d-8c064c39e06d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.079182 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2912/5826] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:09.079288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb1152-9a07-4753-989d-8c064c39e06d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.079485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb1152-9a07-4753-989d-8c064c39e06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.085082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085610e8-7942-470f-a83f-4db02be97a17 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.102732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085610e8-7942-470f-a83f-4db02be97a17 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.118016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-46bb1152-9a07-4753-989d-8c064c39e06d None None] MFA Rules not processed for user `5085205c047641888857281b4a6ea37a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:09.119660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-46bb1152-9a07-4753-989d-8c064c39e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:09.172783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.173542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.173938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.174276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.175078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.177099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.216079 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2915/5827] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:09.216302 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2913/5828] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/projects => generated 391 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:09.217623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.217784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0624d-676a-4582-89e1-67ddee78e331 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.218055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0624d-676a-4582-89e1-67ddee78e331 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.218446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0624d-676a-4582-89e1-67ddee78e331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.243023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c0e0624d-676a-4582-89e1-67ddee78e331 None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:09.244255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c0e0624d-676a-4582-89e1-67ddee78e331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:09.253801 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2916/5829] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:09.254756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0624d-676a-4582-89e1-67ddee78e331 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.316520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ef0e-59aa-4665-aea9-f57e7c36961a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.321437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea19e7-97a1-48f5-9bfd-257f7fb0ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.322078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea19e7-97a1-48f5-9bfd-257f7fb0ab59 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.322353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea19e7-97a1-48f5-9bfd-257f7fb0ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.322623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea19e7-97a1-48f5-9bfd-257f7fb0ab59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.323438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea19e7-97a1-48f5-9bfd-257f7fb0ab59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.325518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea19e7-97a1-48f5-9bfd-257f7fb0ab59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.352865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0624d-676a-4582-89e1-67ddee78e331 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.358446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.359104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.359394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.359697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.360309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.362348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.365370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2914/5830] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:09.367139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af4dd7-5882-479b-bbdf-698315516e5e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.367732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af4dd7-5882-479b-bbdf-698315516e5e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.368240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af4dd7-5882-479b-bbdf-698315516e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.401983 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2917/5831] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/users => generated 302 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:09.403056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.406827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-65af4dd7-5882-479b-bbdf-698315516e5e None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:09.408781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-65af4dd7-5882-479b-bbdf-698315516e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:09.416956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e1ca2-ee8c-4aaf-9206-813e73f67fd7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.466256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:09.474284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:09.492029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2915/5832] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:09.493274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af4dd7-5882-479b-bbdf-698315516e5e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.508836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.509482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.509789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.510061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.511150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.513096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.542428 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2918/5833] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/users => generated 285 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:09.543266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.557368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff535-894b-4175-90b4-3e0b49b91245 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.562680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cec925-7365-456c-8e9b-9b0b22f62ace tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12de706808bb460e892dfeff76d60313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3bc2a51f0f04ce2bdcefb403244a912', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.563348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cec925-7365-456c-8e9b-9b0b22f62ace tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.563631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cec925-7365-456c-8e9b-9b0b22f62ace tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.563946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cec925-7365-456c-8e9b-9b0b22f62ace tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313/password` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.612972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af4dd7-5882-479b-bbdf-698315516e5e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.626214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.627218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.627704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.628001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.628978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.631961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.639977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-05cec925-7365-456c-8e9b-9b0b22f62ace tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:09.648376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2916/5834] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:09.649367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.656831 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2919/5835] 10.222.0.26 () {60 vars in 1269 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/users/12de706808bb460e892dfeff76d60313/password => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:09.657778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cec925-7365-456c-8e9b-9b0b22f62ace tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.664543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c8616-40bf-4791-bc21-4e083af55148 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.670469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cec925-7365-456c-8e9b-9b0b22f62ace tempest-IdentityV3UsersTest-2122549308 tempest-IdentityV3UsersTest-2122549308-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.672382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.673868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.674329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.674903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.675920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.678159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.678972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.679133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.679372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] PATH_INFO: `/v3/projects/f96a6d836aa443dc822fcc6fc70011eb/users/f978f790d0ea4ab6832c9bf2030e7256/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.679682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.693329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2917/5836] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:09.697800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.703273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96a6d836aa443dc822fcc6fc70011eb, user_id=f978f790d0ea4ab6832c9bf2030e7256, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.706422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.710053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e4b59-b6d2-45b8-8a0b-0bd17ff085bf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.716574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.718168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.718476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.718718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.719362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.721118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.722949 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2920/5837] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:09 2022] PUT /identity/v3/projects/f96a6d836aa443dc822fcc6fc70011eb/users/f978f790d0ea4ab6832c9bf2030e7256/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:09.727126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.741856 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2918/5838] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:09.742955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.755625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a016eb-5b1e-4045-96f4-64b39e505ad2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.799725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc0426-f959-4ebf-bc67-e87f64133b1b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.804494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.805048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.805315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.805653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.806403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.808088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.826455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb749-2a10-496e-a0b8-5e9b8d9c8de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.826455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb749-2a10-496e-a0b8-5e9b8d9c8de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.826455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb749-2a10-496e-a0b8-5e9b8d9c8de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.826455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb749-2a10-496e-a0b8-5e9b8d9c8de1 admin admin] PATH_INFO: `/v3/projects/1cb3db8b80c34e8380aad24cc9ca742a/users/1abe9cd0f8714c5c9341801b154f080e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.827322 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2921/5839] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:09.827322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:09.853435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60996a30-e407-4251-9651-fb5ce15e9fe6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:09.861724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b462639-48b0-4e6e-b4f0-20ec27ad4117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:09.862015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbeb749-2a10-496e-a0b8-5e9b8d9c8de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb3db8b80c34e8380aad24cc9ca742a, user_id=1abe9cd0f8714c5c9341801b154f080e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.862130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462639-48b0-4e6e-b4f0-20ec27ad4117 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.862130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462639-48b0-4e6e-b4f0-20ec27ad4117 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.862130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b462639-48b0-4e6e-b4f0-20ec27ad4117 admin admin] PATH_INFO: `/v3/projects/f96a6d836aa443dc822fcc6fc70011eb/users/f978f790d0ea4ab6832c9bf2030e7256/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.864798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbeb749-2a10-496e-a0b8-5e9b8d9c8de1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.864967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b462639-48b0-4e6e-b4f0-20ec27ad4117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96a6d836aa443dc822fcc6fc70011eb, user_id=f978f790d0ea4ab6832c9bf2030e7256, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:09.868308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b462639-48b0-4e6e-b4f0-20ec27ad4117 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:09.884994 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2922/5840] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:09 2022] PUT /identity/v3/projects/f96a6d836aa443dc822fcc6fc70011eb/users/f978f790d0ea4ab6832c9bf2030e7256/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:09.889535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2919/5841] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:09 2022] PUT /identity/v3/projects/1cb3db8b80c34e8380aad24cc9ca742a/users/1abe9cd0f8714c5c9341801b154f080e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:09.895620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.897163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522791d1-7013-475a-8b8e-a2c009ed187a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:09.897551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522791d1-7013-475a-8b8e-a2c009ed187a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.898020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:09.898172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522791d1-7013-475a-8b8e-a2c009ed187a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.898618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:09.931263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-522791d1-7013-475a-8b8e-a2c009ed187a None None] MFA Rules not processed for user `1abe9cd0f8714c5c9341801b154f080e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:09.932676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-522791d1-7013-475a-8b8e-a2c009ed187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:09.938802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] MFA Rules not processed for user `f978f790d0ea4ab6832c9bf2030e7256`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:09.940545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:09.980578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:10.017067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=7,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:10.023986 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2920/5842] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:10.025174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522791d1-7013-475a-8b8e-a2c009ed187a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.108603 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2923/5843] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:10.121148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.122667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:10.124366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522791d1-7013-475a-8b8e-a2c009ed187a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.130276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.130437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.130783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.131064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.131773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.133821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.154677 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2921/5844] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:13:10 2022] POST /identity/v3/domains => generated 274 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:10.158675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.197363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df33b-587b-4351-9c60-4c30d0640e56 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.232386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a53046-937e-41ea-98e5-54e9803d88ff tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.237969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.239167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.239167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.239444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.240213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.241968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.262420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.263275 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2922/5845] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:13:10 2022] POST /identity/v3/roles => generated 239 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:10.263411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.263663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.264160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.264373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.277047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9059-944f-42da-ab74-07e9879d2525 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.284309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.285189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.285701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.285970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/26fa33b79fb446e08684a326ea80d92a/users/1abe9cd0f8714c5c9341801b154f080e/roles/3d615ab9ab8941769e8c87e9f59203bb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.304321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=26fa33b79fb446e08684a326ea80d92a, user_id=1abe9cd0f8714c5c9341801b154f080e, role_id=3d615ab9ab8941769e8c87e9f59203bb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.307608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.322249 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2923/5846] 10.222.0.26 () {60 vars in 1480 bytes} [Sat Sep 17 21:13:10 2022] PUT /identity/v3/domains/26fa33b79fb446e08684a326ea80d92a/users/1abe9cd0f8714c5c9341801b154f080e/roles/3d615ab9ab8941769e8c87e9f59203bb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:10.327520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.352959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.355509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.366852 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2924/5847] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:10.383158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:10.400755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985a4bd-41dd-422a-9bef-de4ea517a578 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.408705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.409344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.409645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.409918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.410590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.412223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.436548 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2924/5848] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:13:10 2022] POST /identity/v3/domains => generated 277 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:10.440561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.454278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6b6e3-1f21-448d-a46f-f25760523765 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.536309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:10.582115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef298bb-b295-4db7-b920-a818ab3b57bb tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.583043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef298bb-b295-4db7-b920-a818ab3b57bb tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.583328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef298bb-b295-4db7-b920-a818ab3b57bb tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.583800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef298bb-b295-4db7-b920-a818ab3b57bb tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.584759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef298bb-b295-4db7-b920-a818ab3b57bb tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.586612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef298bb-b295-4db7-b920-a818ab3b57bb tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.608746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2925/5849] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:13:10 2022] POST /identity/v3/roles => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:10.612867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.649766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42196cc5-e5cb-4ed0-953c-a23e165ee4ce tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.657012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.658733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.658935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.659165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.660005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.660208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be561f2d-de11-42d7-82ff-1af408c02b4d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.660490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be561f2d-de11-42d7-82ff-1af408c02b4d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.660826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be561f2d-de11-42d7-82ff-1af408c02b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.662677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.697980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-be561f2d-de11-42d7-82ff-1af408c02b4d None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:10.700627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-be561f2d-de11-42d7-82ff-1af408c02b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:10.718216 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2926/5850] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:10 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 21:13:10.727739 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2925/5851] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:13:10 2022] POST /identity/v3/groups => generated 327 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:10.732174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be561f2d-de11-42d7-82ff-1af408c02b4d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.750694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be561f2d-de11-42d7-82ff-1af408c02b4d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.756369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.757417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.757924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.758357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups/f9ec2139a287439fa0c13deb4c9d7e56/users/1abe9cd0f8714c5c9341801b154f080e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.773297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=f9ec2139a287439fa0c13deb4c9d7e56, user_id=1abe9cd0f8714c5c9341801b154f080e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.775355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.808151 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2927/5852] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:13:10 2022] PUT /identity/v3/groups/f9ec2139a287439fa0c13deb4c9d7e56/users/1abe9cd0f8714c5c9341801b154f080e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:10.809157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.822826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555542d8-2221-49db-bccb-3b45013a6ed5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.888543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.889224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.889531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.889811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/c663420cd06f48b1aee690528aacf801/groups/f9ec2139a287439fa0c13deb4c9d7e56/roles/6b2268a505644e0f8a387fea8eb41ae5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.892037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=c663420cd06f48b1aee690528aacf801, group_id=f9ec2139a287439fa0c13deb4c9d7e56, role_id=6b2268a505644e0f8a387fea8eb41ae5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.894781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:10.913530 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2926/5853] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:10 2022] PUT /identity/v3/domains/c663420cd06f48b1aee690528aacf801/groups/f9ec2139a287439fa0c13deb4c9d7e56/roles/6b2268a505644e0f8a387fea8eb41ae5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:10.915806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:10.928395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ed36c-67cb-4644-a99a-d73b5458426c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:10.991727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:10.992393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:10.992670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:10.992939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:10.993651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:10.995522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.015511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2928/5854] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:13:10 2022] POST /identity/v3/domains => generated 276 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:11.019581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.031747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65729dc3-e9e5-443e-a6ec-a1f898075523 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.102151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.102781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.103203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.103493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.104174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.105899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.126649 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2927/5855] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:13:11 2022] POST /identity/v3/roles => generated 240 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:11.127934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.142403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a117b-a418-469b-bd5f-5338b6b4ecc7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.147356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.148095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.148438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.148771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/f5f4fd321d0a41fe9724a4d935476826/users/1abe9cd0f8714c5c9341801b154f080e/roles/80d57ad6608243e393e278d6e56393bc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.163453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=f5f4fd321d0a41fe9724a4d935476826, user_id=1abe9cd0f8714c5c9341801b154f080e, role_id=80d57ad6608243e393e278d6e56393bc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.166414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.187950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2929/5856] 10.222.0.26 () {60 vars in 1480 bytes} [Sat Sep 17 21:13:11 2022] PUT /identity/v3/domains/f5f4fd321d0a41fe9724a4d935476826/users/1abe9cd0f8714c5c9341801b154f080e/roles/80d57ad6608243e393e278d6e56393bc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:11.192342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.202982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8d52-ba75-436d-b3c9-bf38e5b4823e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.251494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:11.266752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.268042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.268501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.268782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.269496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.271348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.290125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2928/5857] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:13:11 2022] POST /identity/v3/domains => generated 275 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:11.294504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.305538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda6b5e-3e19-4450-add3-70874f3ca3ee tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.366340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.367005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.367286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.367560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.368278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.370013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.389037 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2930/5858] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:13:11 2022] POST /identity/v3/roles => generated 240 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:11.393356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.405171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8a813-dee0-46bb-a61b-bb89055c42c4 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.410664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.411437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.411828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.412210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.412998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.415147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.449226 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2929/5859] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:13:11 2022] POST /identity/v3/groups => generated 325 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:11.454176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.462238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:11.466913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff44dd-f5cb-4fca-8f91-7f1184c2b824 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.474177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.474853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.475131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.475406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups/5ca318c7aa1147f8bbc2cef198915548/users/1abe9cd0f8714c5c9341801b154f080e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.482010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=5ca318c7aa1147f8bbc2cef198915548, user_id=1abe9cd0f8714c5c9341801b154f080e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.483986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.510935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2931/5860] 10.222.0.26 () {60 vars in 1360 bytes} [Sat Sep 17 21:13:11 2022] PUT /identity/v3/groups/5ca318c7aa1147f8bbc2cef198915548/users/1abe9cd0f8714c5c9341801b154f080e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:11.515514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.526834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8219c-a3b7-4610-bf51-f8cee35b8131 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.595800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3a3be-6b31-459f-8ab1-7f7975b8b566 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.596552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3a3be-6b31-459f-8ab1-7f7975b8b566 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.596907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3a3be-6b31-459f-8ab1-7f7975b8b566 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.597253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3a3be-6b31-459f-8ab1-7f7975b8b566 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/0bbf000a43e14e6281d748c877b49d3d/groups/5ca318c7aa1147f8bbc2cef198915548/roles/8682c9254e034a2b84a11980608dc92e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.599751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3a3be-6b31-459f-8ab1-7f7975b8b566 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=0bbf000a43e14e6281d748c877b49d3d, group_id=5ca318c7aa1147f8bbc2cef198915548, role_id=8682c9254e034a2b84a11980608dc92e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.603189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3a3be-6b31-459f-8ab1-7f7975b8b566 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.622869 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2930/5861] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:11 2022] PUT /identity/v3/domains/0bbf000a43e14e6281d748c877b49d3d/groups/5ca318c7aa1147f8bbc2cef198915548/roles/8682c9254e034a2b84a11980608dc92e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:11.638417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.721484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.722103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.722363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.744304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a923f-19d6-4e11-bc98-9b1adc42b292 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.751410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] MFA Rules not processed for user `12de706808bb460e892dfeff76d60313`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:11.752877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:11.857583 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2931/5862] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:11.858218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:11.859734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:11.910392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1abe9cd0f8714c5c9341801b154f080e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cb3db8b80c34e8380aad24cc9ca742a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.911162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.911757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.912364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] PATH_INFO: `/v3/auth/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.913524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.915512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.950324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2570a5b-b68c-43d6-b666-260cbf906cab None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:11.958996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:11.959759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:11.960865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:11.961551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] PATH_INFO: `/v3/users/12de706808bb460e892dfeff76d60313` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:11.963259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=12de706808bb460e892dfeff76d60313)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:11.966143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:11.968848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:12.015421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:12.055735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.061906 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2932/5863] 10.222.0.26 () {58 vars in 1143 bytes} [Sat Sep 17 21:13:11 2022] GET /identity/v3/auth/domains => generated 1170 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:12.064436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:12.075889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.089334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.090162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.091123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29f0e363-ecf8-44dc-86cd-289d54c556fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12de706808bb460e892dfeff76d60313'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.111050 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2932/5864] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:11 2022] DELETE /identity/v3/users/12de706808bb460e892dfeff76d60313 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:12.116086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:12.118276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:12.118669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:12.149705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637349c-1cda-45d7-bac7-23754ecd5b9f tempest-TokensV3TestJSON-1187056220 tempest-TokensV3TestJSON-1187056220-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:12.162889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:12.164387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:12.275072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:12.278655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:12.279455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:12.279738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/0bbf000a43e14e6281d748c877b49d3d/groups/5ca318c7aa1147f8bbc2cef198915548/roles/8682c9254e034a2b84a11980608dc92e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:12.283229 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2933/5865] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:12.291908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=0bbf000a43e14e6281d748c877b49d3d, group_id=5ca318c7aa1147f8bbc2cef198915548, role_id=8682c9254e034a2b84a11980608dc92e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:12.301687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:12.302886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 8682c9254e034a2b84a11980608dc92e was removed from group 5ca318c7aa1147f8bbc2cef198915548 on domain 0bbf000a43e14e6281d748c877b49d3d. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:12.303358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.310337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:12.312102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:12.314039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.342189 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2933/5866] 10.222.0.26 () {58 vars in 1467 bytes} [Sat Sep 17 21:13:12 2022] DELETE /identity/v3/domains/0bbf000a43e14e6281d748c877b49d3d/groups/5ca318c7aa1147f8bbc2cef198915548/roles/8682c9254e034a2b84a11980608dc92e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:12.348282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:12.351259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:12.438513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dc653-5105-4b18-855a-c3d3d5b423a7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:12.511256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc2cd4-262a-4996-88dc-575abbdeff8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:12.533163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:12.533882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:12.534185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:12.534509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups/5ca318c7aa1147f8bbc2cef198915548/users/1abe9cd0f8714c5c9341801b154f080e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:12.538160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=5ca318c7aa1147f8bbc2cef198915548, user_id=1abe9cd0f8714c5c9341801b154f080e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:12.548305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:12.556506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:12.566847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '1abe9cd0f8714c5c9341801b154f080e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.587600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:12.593648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:12.594273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:12.594551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:12.594819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:12.599153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:12.599341 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2934/5867] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:13:12 2022] DELETE /identity/v3/groups/5ca318c7aa1147f8bbc2cef198915548/users/1abe9cd0f8714c5c9341801b154f080e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:12.600644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:12.698555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:12.699788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:12.711487 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2934/5868] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:12.726344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc5751-6a5b-4e7b-b828-3ee75910e7b8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:12.731494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:12.732141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:12.732420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:12.732937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups/5ca318c7aa1147f8bbc2cef198915548` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:12.734139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_group(group_id=5ca318c7aa1147f8bbc2cef198915548)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:12.736097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:12.792509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2935/5869] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:13:12 2022] DELETE /identity/v3/groups/5ca318c7aa1147f8bbc2cef198915548 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:12.793918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:12.806385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680ce01-c693-4395-8e91-478d3c072226 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:12.873084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:12.874136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:12.874420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:12.874718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles/8682c9254e034a2b84a11980608dc92e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:12.876163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8682c9254e034a2b84a11980608dc92e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:12.878025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:12.918626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 8682c9254e034a2b84a11980608dc92e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:12.919336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.920033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:12.925294 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2935/5870] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:12 2022] DELETE /identity/v3/roles/8682c9254e034a2b84a11980608dc92e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:12.926492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:12.928547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:12.946410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b322d4-7c2d-4291-9fd8-64b5a4cb7b08 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:13.017743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:13.018408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:13.018688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:13.018962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/0bbf000a43e14e6281d748c877b49d3d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:13.021067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0bbf000a43e14e6281d748c877b49d3d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:13.023566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:13.051600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.052449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.080350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.081077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.085520 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2936/5871] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:12 2022] PATCH /identity/v3/domains/0bbf000a43e14e6281d748c877b49d3d => generated 276 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:13.088794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:13.103817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242e938-b2c2-4ed9-ac8e-fb4dbb32d8c1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:13.166063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:13.177010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:13.177831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:13.178133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:13.178389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/0bbf000a43e14e6281d748c877b49d3d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:13.190702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0bbf000a43e14e6281d748c877b49d3d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:13.192355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:13.216091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.226159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.299531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d', 'request_id': 'req-796b8e86-1b45-40dd-93bd-caf4071e184f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.308555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d', 'request_id': 'req-796b8e86-1b45-40dd-93bd-caf4071e184f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.321067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d', 'request_id': 'req-796b8e86-1b45-40dd-93bd-caf4071e184f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.383519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] The token cache is being invalidate because project 0bbf000a43e14e6281d748c877b49d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:13.386315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.387478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.408701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d', 'request_id': 'req-796b8e86-1b45-40dd-93bd-caf4071e184f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.417031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d', 'request_id': 'req-796b8e86-1b45-40dd-93bd-caf4071e184f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.418377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0bbf000a43e14e6281d748c877b49d3d', 'request_id': 'req-796b8e86-1b45-40dd-93bd-caf4071e184f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.427529 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2936/5872] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:13:13 2022] DELETE /identity/v3/domains/0bbf000a43e14e6281d748c877b49d3d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:13.433958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:13.436669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:13.455310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb83bdc-e203-495a-b64d-8d0de20047d8 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:13.487683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:13.569494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:13.570199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:13.570527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:13.570797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/f5f4fd321d0a41fe9724a4d935476826/users/1abe9cd0f8714c5c9341801b154f080e/roles/80d57ad6608243e393e278d6e56393bc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:13.574775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=f5f4fd321d0a41fe9724a4d935476826, user_id=1abe9cd0f8714c5c9341801b154f080e, role_id=80d57ad6608243e393e278d6e56393bc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:13.578498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:13.586360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 80d57ad6608243e393e278d6e56393bc was removed from user 1abe9cd0f8714c5c9341801b154f080e on domain f5f4fd321d0a41fe9724a4d935476826. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:13.586660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.587699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.622013 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2937/5873] 10.222.0.26 () {58 vars in 1464 bytes} [Sat Sep 17 21:13:13 2022] DELETE /identity/v3/domains/f5f4fd321d0a41fe9724a4d935476826/users/1abe9cd0f8714c5c9341801b154f080e/roles/80d57ad6608243e393e278d6e56393bc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:13.623114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:13.625672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:13.680145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b8e86-1b45-40dd-93bd-caf4071e184f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:13.700188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:13.704169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:13.797061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986fdd8-0701-40e0-94d8-f88c4cd53b11 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:13.804338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:13.805169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:13.805736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:13.806108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] PATH_INFO: `/v3/projects/f3bc2a51f0f04ce2bdcefb403244a912` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:13.808322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3bc2a51f0f04ce2bdcefb403244a912)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:13.831671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:13.864351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:13.865801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:13.867256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:13.872325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles/80d57ad6608243e393e278d6e56393bc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:13.878233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_role(role_id=80d57ad6608243e393e278d6e56393bc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:13.880321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:13.933720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3bc2a51f0f04ce2bdcefb403244a912', 'request_id': 'req-27461127-d0ed-46fd-ba39-a0950b34b674'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.949192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:13.952622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3bc2a51f0f04ce2bdcefb403244a912', 'request_id': 'req-27461127-d0ed-46fd-ba39-a0950b34b674'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.969356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 80d57ad6608243e393e278d6e56393bc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:13.969694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.970558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3bc2a51f0f04ce2bdcefb403244a912', 'request_id': 'req-27461127-d0ed-46fd-ba39-a0950b34b674'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.970690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.977785 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2937/5874] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:13 2022] DELETE /identity/v3/roles/80d57ad6608243e393e278d6e56393bc => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:13.981579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:13.983351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:13.991823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] The token cache is being invalidate because project f3bc2a51f0f04ce2bdcefb403244a912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:13.992211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:13.995949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-27461127-d0ed-46fd-ba39-a0950b34b674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.000977 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2938/5875] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:13 2022] DELETE /identity/v3/projects/f3bc2a51f0f04ce2bdcefb403244a912 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:14.007161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.007566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.007972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.062858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:14.065779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:14.102354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68d540-09aa-4ce2-ac27-c1db414929cc tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.110200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:14.111820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.112107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.112381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/f5f4fd321d0a41fe9724a4d935476826` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.113737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f5f4fd321d0a41fe9724a4d935476826)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:14.115397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:14.132073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:14.139684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.140500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.158093 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2939/5876] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:14.159122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:14.160435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:14.170021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.170907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-059932e8-3f7b-4219-ba56-cea2fe3de981 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.174504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2938/5877] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:13 2022] PATCH /identity/v3/domains/f5f4fd321d0a41fe9724a4d935476826 => generated 277 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:13:14.176105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.176540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.176813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.201808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:14.203234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:14.242772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5aff4-3cd7-4a99-9fa3-cc40c3fd5430 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.286767 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2939/5878] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:14.287610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:14.288934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:14.320607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:14.321765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.322412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.322685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.329030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:14.396513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26a004-8b17-4549-952c-d7a86633d4c2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.401554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:14.402135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.402439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.402746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/f5f4fd321d0a41fe9724a4d935476826` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.405613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:14.418579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f5f4fd321d0a41fe9724a4d935476826)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:14.420558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:14.425346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:14.427851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:14.441682 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2940/5879] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:14.442859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:14.494530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.503645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96580591-cca0-42b2-ab0b-1eababb956b3 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.511481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.516541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:14.518449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.518726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.519008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.519658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:14.519958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:14.521623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:14.548246 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2941/5880] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:14.552587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:14.564292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2a940-4ff6-4fa8-89c0-c9ca1d39c1ec admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.583932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826', 'request_id': 'req-730b286d-120a-4602-9c87-ce63bd6d4798'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.599594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826', 'request_id': 'req-730b286d-120a-4602-9c87-ce63bd6d4798'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.608800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826', 'request_id': 'req-730b286d-120a-4602-9c87-ce63bd6d4798'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.611085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:14.631366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] The token cache is being invalidate because project f5f4fd321d0a41fe9724a4d935476826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:14.632613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.634845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.649322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:14.649691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.649980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.650262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.651670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:14.656141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:14.660827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826', 'request_id': 'req-730b286d-120a-4602-9c87-ce63bd6d4798'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.675346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826', 'request_id': 'req-730b286d-120a-4602-9c87-ce63bd6d4798'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.676431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f5f4fd321d0a41fe9724a4d935476826', 'request_id': 'req-730b286d-120a-4602-9c87-ce63bd6d4798'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:14.689668 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2940/5881] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:13:14 2022] DELETE /identity/v3/domains/f5f4fd321d0a41fe9724a4d935476826 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:14.699233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:14.702424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:14.720748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2942/5882] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:14 2022] POST /identity/v3/projects => generated 399 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:14.729025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:14.732627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:14.864819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c20b60-302c-4ad5-80ae-ca2f425cc39f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.872151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b286d-120a-4602-9c87-ce63bd6d4798 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.882358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:14.883069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.883347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.883625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.884300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:14.900073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:14.955654 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2943/5883] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:14 2022] POST /identity/v3/users => generated 306 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:14.960730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:14.971194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:14.984985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c2447-2d42-4402-ab06-405ba6793000 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:14.992531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:14.993230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:14.993556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:14.993833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:14.994612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:14.996459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.011185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.012137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.012137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.012448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/c663420cd06f48b1aee690528aacf801/groups/f9ec2139a287439fa0c13deb4c9d7e56/roles/6b2268a505644e0f8a387fea8eb41ae5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.015947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=c663420cd06f48b1aee690528aacf801, group_id=f9ec2139a287439fa0c13deb4c9d7e56, role_id=6b2268a505644e0f8a387fea8eb41ae5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.019173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2944/5884] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:14 2022] GET /identity/v3/roles => generated 1881 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:15.020052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.025284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.027010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 6b2268a505644e0f8a387fea8eb41ae5 was removed from group f9ec2139a287439fa0c13deb4c9d7e56 on domain c663420cd06f48b1aee690528aacf801. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:15.027444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:15.028250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:15.042890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc5a91-e256-4963-88b0-e761d3172f27 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.064837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.064837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.064837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.064837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] PATH_INFO: `/v3/projects/b78a5787ed7f48da837d0cc382bb51ef/users/5026ac1128cb4e9eace6600e53c8e742/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.066141 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2941/5885] 10.222.0.26 () {58 vars in 1467 bytes} [Sat Sep 17 21:13:14 2022] DELETE /identity/v3/domains/c663420cd06f48b1aee690528aacf801/groups/f9ec2139a287439fa0c13deb4c9d7e56/roles/6b2268a505644e0f8a387fea8eb41ae5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:15.069690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.071265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:15.098822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78a5787ed7f48da837d0cc382bb51ef, user_id=5026ac1128cb4e9eace6600e53c8e742, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.103250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.134602 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2945/5886] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:15 2022] PUT /identity/v3/projects/b78a5787ed7f48da837d0cc382bb51ef/users/5026ac1128cb4e9eace6600e53c8e742/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:15.135584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.139381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:15.166665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a8ba-c713-42ee-bf8f-43b5cbc4a397 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.298867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.299651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513f02-bcb4-4d63-8bbe-3c7246e11ec8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.299788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.300119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.300483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups/f9ec2139a287439fa0c13deb4c9d7e56/users/1abe9cd0f8714c5c9341801b154f080e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.302376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=f9ec2139a287439fa0c13deb4c9d7e56, user_id=1abe9cd0f8714c5c9341801b154f080e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.304541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.314045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.314495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.314781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.315094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.315689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.317362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.322047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '1abe9cd0f8714c5c9341801b154f080e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:15.334065 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2946/5887] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:15 2022] GET /identity/v3/roles => generated 1881 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:15.340021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.351203 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2942/5888] 10.222.0.26 () {58 vars in 1344 bytes} [Sat Sep 17 21:13:15 2022] DELETE /identity/v3/groups/f9ec2139a287439fa0c13deb4c9d7e56/users/1abe9cd0f8714c5c9341801b154f080e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:15.352704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.368844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6a70a-1c94-40aa-a96f-3a807146d6d8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.386447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:15.463686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e92e0e-f0c9-41ec-83d0-855061b4068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.464576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e92e0e-f0c9-41ec-83d0-855061b4068c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.464949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e92e0e-f0c9-41ec-83d0-855061b4068c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.465317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e92e0e-f0c9-41ec-83d0-855061b4068c admin admin] PATH_INFO: `/v3/projects/b78a5787ed7f48da837d0cc382bb51ef/users/5026ac1128cb4e9eace6600e53c8e742/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.475658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e92e0e-f0c9-41ec-83d0-855061b4068c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78a5787ed7f48da837d0cc382bb51ef, user_id=5026ac1128cb4e9eace6600e53c8e742, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.482870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e92e0e-f0c9-41ec-83d0-855061b4068c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.504235 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2947/5889] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:15 2022] PUT /identity/v3/projects/b78a5787ed7f48da837d0cc382bb51ef/users/5026ac1128cb4e9eace6600e53c8e742/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:15.516069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f6c7f-dc1b-4d72-933a-bbd0a9865fa9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.516069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f6c7f-dc1b-4d72-933a-bbd0a9865fa9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.516529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f6c7f-dc1b-4d72-933a-bbd0a9865fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.532872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab6b49-8b44-42ed-aabc-8ebdcb1e5e95 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.563680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:15.580409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-733f6c7f-dc1b-4d72-933a-bbd0a9865fa9 None None] MFA Rules not processed for user `5026ac1128cb4e9eace6600e53c8e742`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:15.581893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-733f6c7f-dc1b-4d72-933a-bbd0a9865fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:15.711552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.712329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.712722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.713086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/groups/f9ec2139a287439fa0c13deb4c9d7e56` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.715124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_group(group_id=f9ec2139a287439fa0c13deb4c9d7e56)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.719057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.725082 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2948/5890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:15 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:15.729720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f6c7f-dc1b-4d72-933a-bbd0a9865fa9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.809810 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2943/5891] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:13:15 2022] DELETE /identity/v3/groups/f9ec2139a287439fa0c13deb4c9d7e56 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:15.811386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.856518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f6c7f-dc1b-4d72-933a-bbd0a9865fa9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.895174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd86d3d-4ed5-4f26-b7d7-c7471222c2f9 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.901706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.902548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.902832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.903156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles/6b2268a505644e0f8a387fea8eb41ae5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.904826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6b2268a505644e0f8a387fea8eb41ae5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.907557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.954611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:15.971777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 6b2268a505644e0f8a387fea8eb41ae5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:15.972300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:15.973724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:15.976596 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2944/5892] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:15 2022] DELETE /identity/v3/roles/6b2268a505644e0f8a387fea8eb41ae5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:15.983003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.987460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:16.029510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:16.030355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.030561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.030763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.031992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:16.034980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:16.065027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:16.089138 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2949/5893] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:15 2022] POST /identity/v3/projects => generated 399 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:16.090685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:16.092562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:16.106289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03be18-c427-49a0-934f-20d599de229a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:16.182474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2caaab-557d-47f4-bdc2-657fa957f7e3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:16.190779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:16.191819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.193653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.194159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.197340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:16.201962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:16.227372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:16.228526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.229143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.230407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/c663420cd06f48b1aee690528aacf801` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.232772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c663420cd06f48b1aee690528aacf801)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:16.234753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:16.260249 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2950/5894] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:16 2022] POST /identity/v3/users => generated 300 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:16.264086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:16.278239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c663420cd06f48b1aee690528aacf801'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.279266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c663420cd06f48b1aee690528aacf801'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.282332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe332fe-ec70-4b65-b30b-cdd5878d4973 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:16.322539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c663420cd06f48b1aee690528aacf801'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.323674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c663420cd06f48b1aee690528aacf801'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.331589 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2945/5895] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:15 2022] PATCH /identity/v3/domains/c663420cd06f48b1aee690528aacf801 => generated 278 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:16.341636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:16.344963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:16.411930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:16.412586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.412864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.413135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.420545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:16.422325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:16.461089 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2951/5896] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:16 2022] GET /identity/v3/roles => generated 1650 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:16.474806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:16.480000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:16.516448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995544f7-f156-4eaa-99a3-2159a11bf9de admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:16.523081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ab6fb-10ee-4eb5-8de7-e6ba32299bbf tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:16.528363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:16.528693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:16.529408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.529754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.530050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.530179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/c663420cd06f48b1aee690528aacf801` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.530520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.533709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.534494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:16.540299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:16.547257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c663420cd06f48b1aee690528aacf801)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:16.548811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:16.573609 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2952/5897] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:16 2022] GET /identity/v3/roles => generated 1650 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:16.578316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:16.606866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd6f5a-4dee-4b4e-8c1f-af850a1d7fd0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:16.619606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee520cda-a9e8-4f9b-ae4d-2ed4fbf4d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:16.621205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee520cda-a9e8-4f9b-ae4d-2ed4fbf4d4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.622079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee520cda-a9e8-4f9b-ae4d-2ed4fbf4d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.622849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee520cda-a9e8-4f9b-ae4d-2ed4fbf4d4ee admin admin] PATH_INFO: `/v3/projects/5e1484080d3942ae8808caba1dbb4392/users/d8e7dfea5df9430aa1af5d52cb9138d7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.633725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c663420cd06f48b1aee690528aacf801'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.665980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee520cda-a9e8-4f9b-ae4d-2ed4fbf4d4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1484080d3942ae8808caba1dbb4392, user_id=d8e7dfea5df9430aa1af5d52cb9138d7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:16.670164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee520cda-a9e8-4f9b-ae4d-2ed4fbf4d4ee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:16.707828 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2953/5898] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:16 2022] PUT /identity/v3/projects/5e1484080d3942ae8808caba1dbb4392/users/d8e7dfea5df9430aa1af5d52cb9138d7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:16.717957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c663420cd06f48b1aee690528aacf801'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.719139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:16.719422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:16.719694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:16.757657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:16.789979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:16.804128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] MFA Rules not processed for user `d8e7dfea5df9430aa1af5d52cb9138d7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:16.807284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c663420cd06f48b1aee690528aacf801', 'request_id': 'req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.808697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:16.833800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c663420cd06f48b1aee690528aacf801', 'request_id': 'req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.843314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c663420cd06f48b1aee690528aacf801', 'request_id': 'req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.860478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] The token cache is being invalidate because project c663420cd06f48b1aee690528aacf801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:16.860759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.861667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.871305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c663420cd06f48b1aee690528aacf801', 'request_id': 'req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.891228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c663420cd06f48b1aee690528aacf801', 'request_id': 'req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.893039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c663420cd06f48b1aee690528aacf801', 'request_id': 'req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:16.905774 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2946/5899] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:13:16 2022] DELETE /identity/v3/domains/c663420cd06f48b1aee690528aacf801 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:16.909863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:16.913425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:17.012435 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2954/5900] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:17.024356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:17.025969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:17.062491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b2130-6cd6-4bb6-929c-07bc08bb2fc1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:17.078854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:17.079627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:17.080191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:17.081982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/26fa33b79fb446e08684a326ea80d92a/users/1abe9cd0f8714c5c9341801b154f080e/roles/3d615ab9ab8941769e8c87e9f59203bb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:17.085681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=26fa33b79fb446e08684a326ea80d92a, user_id=1abe9cd0f8714c5c9341801b154f080e, role_id=3d615ab9ab8941769e8c87e9f59203bb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:17.094448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:17.122822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 3d615ab9ab8941769e8c87e9f59203bb was removed from user 1abe9cd0f8714c5c9341801b154f080e on domain 26fa33b79fb446e08684a326ea80d92a. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:17.123131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.124259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.157135 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2947/5901] 10.222.0.26 () {58 vars in 1464 bytes} [Sat Sep 17 21:13:16 2022] DELETE /identity/v3/domains/26fa33b79fb446e08684a326ea80d92a/users/1abe9cd0f8714c5c9341801b154f080e/roles/3d615ab9ab8941769e8c87e9f59203bb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:17.161621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:17.165890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:17.173148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554cd39-6a93-406f-8a19-5d67db516339 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:17.290092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:17.290837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:17.291186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:17.291676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:17.295994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:17.317489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f373d7-abfd-49e3-b97f-e08fedd1a94c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:17.323347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:17.324377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:17.324718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:17.325063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles/3d615ab9ab8941769e8c87e9f59203bb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:17.326644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_role(role_id=3d615ab9ab8941769e8c87e9f59203bb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:17.328768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:17.347615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:17.389985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 3d615ab9ab8941769e8c87e9f59203bb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:17.390215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.394305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.401823 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2948/5902] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:17 2022] DELETE /identity/v3/roles/3d615ab9ab8941769e8c87e9f59203bb => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:17.402974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:17.405644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:17.444567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:17.494724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:17.500254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:17.524862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2955/5903] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:17 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:17.607328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5eab-5106-4eaa-b3b5-87bbccf9fd19 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:17.622682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:17.629771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:17.630091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:17.630384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/26fa33b79fb446e08684a326ea80d92a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:17.634856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=26fa33b79fb446e08684a326ea80d92a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:17.637281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:17.695848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '26fa33b79fb446e08684a326ea80d92a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.697208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '26fa33b79fb446e08684a326ea80d92a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.757183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '26fa33b79fb446e08684a326ea80d92a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.757826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '26fa33b79fb446e08684a326ea80d92a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:17.760293 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2949/5904] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:17 2022] PATCH /identity/v3/domains/26fa33b79fb446e08684a326ea80d92a => generated 275 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:17.764830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:17.783610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd91b39-d5d6-41b6-90ec-6228aadc3c37 service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:17.952358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:17.953126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:17.953415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:17.953875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/26fa33b79fb446e08684a326ea80d92a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:17.991783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=26fa33b79fb446e08684a326ea80d92a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:17.993439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.023758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '26fa33b79fb446e08684a326ea80d92a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.029303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '26fa33b79fb446e08684a326ea80d92a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.169714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26fa33b79fb446e08684a326ea80d92a', 'request_id': 'req-3974ce8e-c933-486e-abf1-d7306d5147b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.182940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26fa33b79fb446e08684a326ea80d92a', 'request_id': 'req-3974ce8e-c933-486e-abf1-d7306d5147b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.195452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26fa33b79fb446e08684a326ea80d92a', 'request_id': 'req-3974ce8e-c933-486e-abf1-d7306d5147b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.221700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] The token cache is being invalidate because project 26fa33b79fb446e08684a326ea80d92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:18.222167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.223274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.232397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '26fa33b79fb446e08684a326ea80d92a', 'request_id': 'req-3974ce8e-c933-486e-abf1-d7306d5147b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.248027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '26fa33b79fb446e08684a326ea80d92a', 'request_id': 'req-3974ce8e-c933-486e-abf1-d7306d5147b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.250313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '26fa33b79fb446e08684a326ea80d92a', 'request_id': 'req-3974ce8e-c933-486e-abf1-d7306d5147b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.260740 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2956/5905] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:13:17 2022] DELETE /identity/v3/domains/26fa33b79fb446e08684a326ea80d92a => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:18.266613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.270650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:18.285529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44d283-4d7f-4a86-b4c8-a3b667f8e51e tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.300333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.301884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:18.346838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:18.398396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974ce8e-c933-486e-abf1-d7306d5147b6 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.408571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.409549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.410023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.410495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] PATH_INFO: `/v3/users/5026ac1128cb4e9eace6600e53c8e742` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.411849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5026ac1128cb4e9eace6600e53c8e742)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.414097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.417176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.417869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.418151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.418474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.419077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.420753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.442030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2950/5906] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:18.443593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.446082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:18.460636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bcc02-6e25-42c6-b920-adbbf14913be tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.465626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.466286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.466569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.466840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects/42783e4d01654ae7a43b8819a8a91b66/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.470722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_grant(project_id=42783e4d01654ae7a43b8819a8a91b66, user_id=5085205c047641888857281b4a6ea37a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.471928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.477255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5026ac1128cb4e9eace6600e53c8e742'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.478065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5026ac1128cb4e9eace6600e53c8e742'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.487528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5026ac1128cb4e9eace6600e53c8e742'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.488268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5026ac1128cb4e9eace6600e53c8e742'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.492467 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2951/5907] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:18 2022] PUT /identity/v3/projects/42783e4d01654ae7a43b8819a8a91b66/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:18.493478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.495458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:18.502585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a59a6c6a-6a1e-45d4-bb51-b208cd990a5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5026ac1128cb4e9eace6600e53c8e742'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.517863 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2957/5908] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:18 2022] DELETE /identity/v3/users/5026ac1128cb4e9eace6600e53c8e742 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:18.522809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.523102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.523377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.538004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:18.548919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:18.550113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:18.593870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebb639-49f6-4a11-9ca8-8fb969d0204f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.641705 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2958/5909] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:18.654564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.656430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:18.703126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.703757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.704086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.704364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.705259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.707545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.742198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:18.751591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66de36-fb53-467f-8e09-867f29fc84ef None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.756814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.757428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.757861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.758333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.769585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.771338 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2952/5910] 10.222.0.26 () {58 vars in 1146 bytes} [Sat Sep 17 21:13:18 2022] GET /identity/v3/auth/projects => generated 824 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:18.771700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.775190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.786291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22a84a-85f2-4bbb-b0aa-58c90bc82c9f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.792111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.792868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.793153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.793559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects/42783e4d01654ae7a43b8819a8a91b66/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.795531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2959/5911] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:18.795990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=42783e4d01654ae7a43b8819a8a91b66, user_id=5085205c047641888857281b4a6ea37a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.798890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.805751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role 6583e7a6ce674f6da4f19e31fe025509 was removed from user 5085205c047641888857281b4a6ea37a on project 42783e4d01654ae7a43b8819a8a91b66. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:18.806048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.806942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:18.827832 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2953/5912] 10.222.0.26 () {58 vars in 1467 bytes} [Sat Sep 17 21:13:18 2022] DELETE /identity/v3/projects/42783e4d01654ae7a43b8819a8a91b66/users/5085205c047641888857281b4a6ea37a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:18.834833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.842676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.844665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:18.852232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef6142-ba28-479e-8f6f-8b6ad5514328 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.934099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.934764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.935398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.935533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.936236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.938363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.938718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c8794-558e-4362-a7d4-b336d606f81c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:18.945566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:18.948122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:18.951479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:18.951792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] PATH_INFO: `/v3/users/7e084d52162e4b1ba2f5ecfe1a6a4d0e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:18.955180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e084d52162e4b1ba2f5ecfe1a6a4d0e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:18.958484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:18.966940 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2960/5913] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:18 2022] POST /identity/v3/users => generated 323 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:18.981731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:18.981731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:19.003355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cb5a6-7a22-4d7f-a1df-2d6b10e04ee7 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.016755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:19.017375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:19.018861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.019161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.019494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.020360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:19.022868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:19.066871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e084d52162e4b1ba2f5ecfe1a6a4d0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.087196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e084d52162e4b1ba2f5ecfe1a6a4d0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.108035 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2961/5914] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:18 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:19.110892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.116542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e084d52162e4b1ba2f5ecfe1a6a4d0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.117699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e084d52162e4b1ba2f5ecfe1a6a4d0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.118495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f1899a79-2311-4b54-b4a1-31943fafcbdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e084d52162e4b1ba2f5ecfe1a6a4d0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.147100 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2954/5915] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:18 2022] DELETE /identity/v3/users/7e084d52162e4b1ba2f5ecfe1a6a4d0e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:19.149766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcfd49-001a-417e-a285-4416803764af None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.150052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcfd49-001a-417e-a285-4416803764af None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.150754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcfd49-001a-417e-a285-4416803764af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.219459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5efcfd49-001a-417e-a285-4416803764af None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:19.220744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5efcfd49-001a-417e-a285-4416803764af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:19.241593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae3012-6495-4f29-ba5f-789f9d2b3c3b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.345225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:19.345805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.346142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.347110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.347788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:19.351120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:19.367363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:19.373793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2955/5916] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:19.375799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcfd49-001a-417e-a285-4416803764af None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.377297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5efcfd49-001a-417e-a285-4416803764af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:19.395754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcfd49-001a-417e-a285-4416803764af None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.405355 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2962/5917] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:19 2022] POST /identity/v3/projects => generated 363 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:19.406757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.411143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:19.411799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.412180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.412328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] PATH_INFO: `/v3/projects/b78a5787ed7f48da837d0cc382bb51ef` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.412755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:19.414949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78a5787ed7f48da837d0cc382bb51ef)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:19.417445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:19.502976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78a5787ed7f48da837d0cc382bb51ef', 'request_id': 'req-ce4b1fdd-905d-4147-9f10-26a99f1abefb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.507408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd8bd-4af0-4712-a545-08b674e716d1 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.514427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78a5787ed7f48da837d0cc382bb51ef', 'request_id': 'req-ce4b1fdd-905d-4147-9f10-26a99f1abefb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.527134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78a5787ed7f48da837d0cc382bb51ef', 'request_id': 'req-ce4b1fdd-905d-4147-9f10-26a99f1abefb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.546063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] The token cache is being invalidate because project b78a5787ed7f48da837d0cc382bb51ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:19.546063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.546063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.553752 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2956/5918] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:19 2022] DELETE /identity/v3/projects/b78a5787ed7f48da837d0cc382bb51ef => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:19.555347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.556762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:19.603168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:19.608182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:19.608824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.609103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.609375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.611469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:19.646799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b1fdd-905d-4147-9f10-26a99f1abefb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.651912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:19.652629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.652986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.654101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.654877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:19.656661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:19.680041 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2957/5919] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:13:19 2022] POST /identity/v3/roles => generated 239 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:19.681879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:19.682648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.684334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:19.696755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:19.698464 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2963/5920] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:19.699440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.710189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e276de-093a-4684-a459-0e63487b8c2c service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.715230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:19.716017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.716439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.716720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects/b282fd0edde44d57853df6e8363dbcda/users/3b81b1339a1b4c3ea62e33bc16d738dd/roles/bdc42bc14bc6441a978eddd78578f426` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.720013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe16f9-c1eb-4669-b53b-459567c6c910 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.726388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:19.727904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:19.728428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:19.728932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] PATH_INFO: `/v3/users/d8e7dfea5df9430aa1af5d52cb9138d7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:19.730554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e7dfea5df9430aa1af5d52cb9138d7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:19.733815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:19.734721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b282fd0edde44d57853df6e8363dbcda, user_id=3b81b1339a1b4c3ea62e33bc16d738dd, role_id=bdc42bc14bc6441a978eddd78578f426)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:19.738607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:19.765720 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2964/5921] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:19 2022] PUT /identity/v3/projects/b282fd0edde44d57853df6e8363dbcda/users/3b81b1339a1b4c3ea62e33bc16d738dd/roles/bdc42bc14bc6441a978eddd78578f426 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:19.770105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.814256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:19.827280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e7dfea5df9430aa1af5d52cb9138d7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.848681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e7dfea5df9430aa1af5d52cb9138d7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.863242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e7dfea5df9430aa1af5d52cb9138d7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.864052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e7dfea5df9430aa1af5d52cb9138d7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.864981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e7dfea5df9430aa1af5d52cb9138d7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:19.901060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2958/5922] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:19 2022] DELETE /identity/v3/users/d8e7dfea5df9430aa1af5d52cb9138d7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:19.910283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbb587-684a-4822-85ad-395add9ca5a5 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:19.936429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:19.938564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.030729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e956d64-9f31-4ac8-a1ee-efca6698ca2a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:20.031409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e956d64-9f31-4ac8-a1ee-efca6698ca2a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.031685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e956d64-9f31-4ac8-a1ee-efca6698ca2a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.031959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e956d64-9f31-4ac8-a1ee-efca6698ca2a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects/573198ce29ec4d39a243c927d2bed5ce/users/3b81b1339a1b4c3ea62e33bc16d738dd/roles/bdc42bc14bc6441a978eddd78578f426` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.034343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e956d64-9f31-4ac8-a1ee-efca6698ca2a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:create_grant(project_id=573198ce29ec4d39a243c927d2bed5ce, user_id=3b81b1339a1b4c3ea62e33bc16d738dd, role_id=bdc42bc14bc6441a978eddd78578f426)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:20.037203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e956d64-9f31-4ac8-a1ee-efca6698ca2a tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:20.054386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416400-ffd3-465f-aa82-a9774e6ff84d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:20.061940 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2965/5923] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:19 2022] PUT /identity/v3/projects/573198ce29ec4d39a243c927d2bed5ce/users/3b81b1339a1b4c3ea62e33bc16d738dd/roles/bdc42bc14bc6441a978eddd78578f426 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:20.065856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c110-d1a5-4ed1-8cb0-4f237187823a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.066155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c110-d1a5-4ed1-8cb0-4f237187823a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.066433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c110-d1a5-4ed1-8cb0-4f237187823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.101391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:20.125307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7d34c110-d1a5-4ed1-8cb0-4f237187823a None None] MFA Rules not processed for user `3b81b1339a1b4c3ea62e33bc16d738dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:20.126516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7d34c110-d1a5-4ed1-8cb0-4f237187823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.138471 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2966/5924] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:20 2022] POST /identity/v3/auth/tokens => generated 335 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 21:13:20.144807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.145773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.145773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.169099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:20.169530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.169813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.170570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] PATH_INFO: `/v3/projects/5644b74ce00c42719cbdfcecd732b06c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.172660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5644b74ce00c42719cbdfcecd732b06c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:20.173624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] MFA Rules not processed for user `3b81b1339a1b4c3ea62e33bc16d738dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:20.174969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.175507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:20.298081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5644b74ce00c42719cbdfcecd732b06c', 'request_id': 'req-e375c621-c226-44b3-9cf8-d8a95a34caf5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.314749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5644b74ce00c42719cbdfcecd732b06c', 'request_id': 'req-e375c621-c226-44b3-9cf8-d8a95a34caf5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.327663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5644b74ce00c42719cbdfcecd732b06c', 'request_id': 'req-e375c621-c226-44b3-9cf8-d8a95a34caf5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.345645 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2967/5925] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3225 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 17 21:13:20.349580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:20.355545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.366383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] The token cache is being invalidate because project 5644b74ce00c42719cbdfcecd732b06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:20.366688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.367838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.370563 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2959/5926] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:19 2022] DELETE /identity/v3/projects/5644b74ce00c42719cbdfcecd732b06c => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:20.371321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:20.374727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.524426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375c621-c226-44b3-9cf8-d8a95a34caf5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:20.525942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78233286-5610-4b15-8d29-d2c02a18a26b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:20.530283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:20.531177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.532081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:20.532224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.532421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] PATH_INFO: `/v3/projects/5e1484080d3942ae8808caba1dbb4392` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.534767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1484080d3942ae8808caba1dbb4392)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:20.537343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:20.593004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00ac81-df2a-49a9-8927-d28764889c57 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:20.593654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ac81-df2a-49a9-8927-d28764889c57 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.593952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ac81-df2a-49a9-8927-d28764889c57 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.594197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ac81-df2a-49a9-8927-d28764889c57 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.594806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1484080d3942ae8808caba1dbb4392', 'request_id': 'req-cb10a8c6-71f6-4506-a62a-64ec3b18d290'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.596017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fe00ac81-df2a-49a9-8927-d28764889c57 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.607499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1484080d3942ae8808caba1dbb4392', 'request_id': 'req-cb10a8c6-71f6-4506-a62a-64ec3b18d290'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.616481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1484080d3942ae8808caba1dbb4392', 'request_id': 'req-cb10a8c6-71f6-4506-a62a-64ec3b18d290'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.625625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:20.631481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] The token cache is being invalidate because project 5e1484080d3942ae8808caba1dbb4392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:20.632126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.633291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.635718 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2960/5927] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:20 2022] DELETE /identity/v3/projects/5e1484080d3942ae8808caba1dbb4392 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:20.636973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:20.639660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.684989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00ac81-df2a-49a9-8927-d28764889c57 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:20.691723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00ac81-df2a-49a9-8927-d28764889c57 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:20.710566 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2968/5928] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 17 21:13:20 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:20.712271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6b3d-1494-450a-8163-bd569e27923c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.712563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6b3d-1494-450a-8163-bd569e27923c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.712840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb6b3d-1494-450a-8163-bd569e27923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.722448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10a8c6-71f6-4506-a62a-64ec3b18d290 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:20.727328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:20.728053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.728337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.728625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] PATH_INFO: `/v3/users/f978f790d0ea4ab6832c9bf2030e7256` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.729933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f978f790d0ea4ab6832c9bf2030e7256)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:20.731949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:20.735966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4ccb6b3d-1494-450a-8163-bd569e27923c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:20.737888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ccb6b3d-1494-450a-8163-bd569e27923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.753609 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2969/5929] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:20.756345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.756639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.757018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.764217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.772415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] MFA Rules not processed for user `3b81b1339a1b4c3ea62e33bc16d738dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:20.773594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.788093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f978f790d0ea4ab6832c9bf2030e7256'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.810964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f978f790d0ea4ab6832c9bf2030e7256'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.822017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f978f790d0ea4ab6832c9bf2030e7256'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.822749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f978f790d0ea4ab6832c9bf2030e7256'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.823492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b4dee9bc-6911-4428-97fa-d526c42a941c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f978f790d0ea4ab6832c9bf2030e7256'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:20.845135 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2961/5930] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:20 2022] DELETE /identity/v3/users/f978f790d0ea4ab6832c9bf2030e7256 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:20.847355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324eda7-72a6-4f6e-a437-7fc12655a2e4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:20.847595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324eda7-72a6-4f6e-a437-7fc12655a2e4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:20.847850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324eda7-72a6-4f6e-a437-7fc12655a2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:20.878388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2970/5931] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 17 21:13:20.879770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:20.881782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.896372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3324eda7-72a6-4f6e-a437-7fc12655a2e4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:20.897542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3324eda7-72a6-4f6e-a437-7fc12655a2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:20.959235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:20.990094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3b240-43be-4b7c-bfa9-fe43aad5f319 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.001108 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2962/5932] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:21.006039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324eda7-72a6-4f6e-a437-7fc12655a2e4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.020871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324eda7-72a6-4f6e-a437-7fc12655a2e4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.026927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.027850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.028602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.029078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.030275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:21.031544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.032705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.050266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2963/5933] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:21.054643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.063068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:21.070822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117de87-b00e-4be5-8bf7-8e051118b337 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.079881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.081382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.084534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.085062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.086126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.088112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.098736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.099572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.099914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.100274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/roles/bdc42bc14bc6441a978eddd78578f426` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.102055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bdc42bc14bc6441a978eddd78578f426)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.104815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.142029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2964/5934] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:21 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:21.146342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.162535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invalidating the token cache because role bdc42bc14bc6441a978eddd78578f426 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:21.162919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.163780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.165896 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2971/5935] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:20 2022] DELETE /identity/v3/roles/bdc42bc14bc6441a978eddd78578f426 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:21.169878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.171337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.235942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce79ad8-55d8-4ad3-9bba-3b4b346944ab admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.280335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e79b1-ba01-47f6-812e-f2c7b360c688 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.285286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.286076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.286418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.286756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects/573198ce29ec4d39a243c927d2bed5ce` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.288092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_project(project_id=573198ce29ec4d39a243c927d2bed5ce)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.290551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.321899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.323190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.323540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.323885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.325345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.328067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.359735 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2965/5936] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:21 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:21.367594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573198ce29ec4d39a243c927d2bed5ce', 'request_id': 'req-79453048-6c8e-4c9b-81e0-5800e2612c20'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.370950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.372947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.377661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573198ce29ec4d39a243c927d2bed5ce', 'request_id': 'req-79453048-6c8e-4c9b-81e0-5800e2612c20'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.389929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573198ce29ec4d39a243c927d2bed5ce', 'request_id': 'req-79453048-6c8e-4c9b-81e0-5800e2612c20'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.393910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:21.420096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca2bf0-601d-41de-8acd-8238349cf58d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.430796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] The token cache is being invalidate because project 573198ce29ec4d39a243c927d2bed5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:21.433030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.433786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.444525 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2972/5937] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:21 2022] DELETE /identity/v3/projects/573198ce29ec4d39a243c927d2bed5ce => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:21.445542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.447480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.486823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.487751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.488532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.488532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.489658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.491325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.514895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2966/5938] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:21.514895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.521312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.531025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79453048-6c8e-4c9b-81e0-5800e2612c20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.536241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.537063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.537347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.537660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] PATH_INFO: `/v3/projects/f96a6d836aa443dc822fcc6fc70011eb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.538810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96a6d836aa443dc822fcc6fc70011eb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.541036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.550796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:21.609636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96a6d836aa443dc822fcc6fc70011eb', 'request_id': 'req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.617234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96a6d836aa443dc822fcc6fc70011eb', 'request_id': 'req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.617844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3a981-6a9f-449b-ba4a-31270e965b60 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.628066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96a6d836aa443dc822fcc6fc70011eb', 'request_id': 'req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.646495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] The token cache is being invalidate because project f96a6d836aa443dc822fcc6fc70011eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:21.647051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.647907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.649725 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2973/5939] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:21 2022] DELETE /identity/v3/projects/f96a6d836aa443dc822fcc6fc70011eb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:21.650948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.653663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.702000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4687d62-2569-433a-9bb7-e1bf346cbb20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.702663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687d62-2569-433a-9bb7-e1bf346cbb20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.703335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687d62-2569-433a-9bb7-e1bf346cbb20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.703623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687d62-2569-433a-9bb7-e1bf346cbb20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects/573198ce29ec4d39a243c927d2bed5ce` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.712115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4687d62-2569-433a-9bb7-e1bf346cbb20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_project(project_id=573198ce29ec4d39a243c927d2bed5ce)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.714151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4687d62-2569-433a-9bb7-e1bf346cbb20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c4687d62-2569-433a-9bb7-e1bf346cbb20 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Could not find project: 573198ce29ec4d39a243c927d2bed5ce.: keystone.exception.ProjectNotFound: Could not find project: 573198ce29ec4d39a243c927d2bed5ce. Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 17 21:13:21.730454 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 573198ce29ec4d39a243c927d2bed5ce. Sep 17 21:13:21.731722 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:21.732397 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2967/5940] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:21 2022] DELETE /identity/v3/projects/573198ce29ec4d39a243c927d2bed5ce => generated 113 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:13:21.736798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.737069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.737345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.757758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3e54-000b-4e6f-bcb0-c172cac628c0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.762602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:21.764203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.765077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e57b5-341f-4fe5-8c5a-163e77e77948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.766790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e57b5-341f-4fe5-8c5a-163e77e77948 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.767292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e57b5-341f-4fe5-8c5a-163e77e77948 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.767887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e57b5-341f-4fe5-8c5a-163e77e77948 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.768844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e57b5-341f-4fe5-8c5a-163e77e77948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.770687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e57b5-341f-4fe5-8c5a-163e77e77948 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.772755 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2968/5941] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:21.773656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.775573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.786085 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2974/5942] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:21.787969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c6a4-eb3e-4dac-bad2-41037f671b58 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.788563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c6a4-eb3e-4dac-bad2-41037f671b58 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.789108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c6a4-eb3e-4dac-bad2-41037f671b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.794151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291377d-54d8-46a8-a372-11dae3e73954 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.799084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.799833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.800187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.800533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/projects/b282fd0edde44d57853df6e8363dbcda` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.805352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b282fd0edde44d57853df6e8363dbcda)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.808272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.839304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-93e3c6a4-eb3e-4dac-bad2-41037f671b58 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:21.840591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-93e3c6a4-eb3e-4dac-bad2-41037f671b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.853387 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2975/5943] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:21.853387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3c6a4-eb3e-4dac-bad2-41037f671b58 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.864508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3c6a4-eb3e-4dac-bad2-41037f671b58 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:21.875623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:21.891281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b282fd0edde44d57853df6e8363dbcda', 'request_id': 'req-1b726498-771a-4240-8baa-b5e8a1293b2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.901104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b282fd0edde44d57853df6e8363dbcda', 'request_id': 'req-1b726498-771a-4240-8baa-b5e8a1293b2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.907590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b282fd0edde44d57853df6e8363dbcda', 'request_id': 'req-1b726498-771a-4240-8baa-b5e8a1293b2b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.919872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] The token cache is being invalidate because project b282fd0edde44d57853df6e8363dbcda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:21.920314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.921697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:21.924142 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2969/5944] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:21 2022] DELETE /identity/v3/projects/b282fd0edde44d57853df6e8363dbcda => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:21.925279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.926950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:21.939609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:21.940282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:21.940628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:21.940826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] PATH_INFO: `/v3/projects/8af940fb16b948fcb49befa103ec8d0e/users/0fc783e858c14bedba61cb07b3a9130d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:21.957741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af940fb16b948fcb49befa103ec8d0e, user_id=0fc783e858c14bedba61cb07b3a9130d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:21.961440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:21.967001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:21.980032 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2976/5945] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:21 2022] PUT /identity/v3/projects/8af940fb16b948fcb49befa103ec8d0e/users/0fc783e858c14bedba61cb07b3a9130d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:21.981164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:21.990833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:22.016382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b726498-771a-4240-8baa-b5e8a1293b2b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.074188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a066c0-99a1-4acc-9697-fb9fbb9703cd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.076541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd9c84-a469-4b1f-b8a8-4cdadd2c1009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.077200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd9c84-a469-4b1f-b8a8-4cdadd2c1009 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.077507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd9c84-a469-4b1f-b8a8-4cdadd2c1009 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.077893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd9c84-a469-4b1f-b8a8-4cdadd2c1009 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.078541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd9c84-a469-4b1f-b8a8-4cdadd2c1009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:22.078846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd9c84-a469-4b1f-b8a8-4cdadd2c1009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.080114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.080582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd9c84-a469-4b1f-b8a8-4cdadd2c1009 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.081026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.081380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.081774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/users/3b81b1339a1b4c3ea62e33bc16d738dd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.083009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3b81b1339a1b4c3ea62e33bc16d738dd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.085003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.100238 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2970/5946] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:22.100238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.100238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.100238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.128615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] MFA Rules not processed for user `0fc783e858c14bedba61cb07b3a9130d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:22.129891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:22.143946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b81b1339a1b4c3ea62e33bc16d738dd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:22.144623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b81b1339a1b4c3ea62e33bc16d738dd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:22.153838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b81b1339a1b4c3ea62e33bc16d738dd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:22.154505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b81b1339a1b4c3ea62e33bc16d738dd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:22.168905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b81b1339a1b4c3ea62e33bc16d738dd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:22.182559 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2977/5947] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:21 2022] DELETE /identity/v3/users/3b81b1339a1b4c3ea62e33bc16d738dd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:22.183909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.195074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:22.206203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:22.237373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2971/5948] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:22.237373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.237373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:22.246299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172163f-4c74-4171-84b3-56cf880107d2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.270381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cce85-ee78-49a1-9480-ea8ae743ef8c tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.275567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.276187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.276461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.276730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.277803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.280207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.313529 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2978/5949] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:22 2022] POST /identity/v3/projects => generated 397 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:22.314631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.315819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.316189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:22.316813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.317181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.317576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/0bbf000a43e14e6281d748c877b49d3d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.333433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Could not find domain: 0bbf000a43e14e6281d748c877b49d3d.: keystone.exception.DomainNotFound: Could not find domain: 0bbf000a43e14e6281d748c877b49d3d. Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0bbf000a43e14e6281d748c877b49d3d. Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:22.337389 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:22.339483 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 0bbf000a43e14e6281d748c877b49d3d. Sep 17 21:13:22.341022 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.345192 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2972/5950] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:22 2022] PATCH /identity/v3/domains/0bbf000a43e14e6281d748c877b49d3d => generated 112 bytes in 114 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:13:22.345192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.350254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:22.364287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e9289-e8f0-4a86-a4ca-e7b57264de3f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.435172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890d8b-6c20-47f9-82d6-851980b99f98 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.440236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.440747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.441699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.443550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.481926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.483319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.483786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.484238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.485037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.487616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.534835 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2973/5951] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:22 2022] POST /identity/v3/users => generated 299 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:22.535062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.535547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.535655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.561294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2979/5952] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:22.562586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.574485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96602b90-d7b6-4d02-9ba6-74e2e4101df5 service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.581596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.582503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.582778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.583048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.583688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.586761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.601929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2980/5953] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:22.605857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.621258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4552-0082-486a-8880-4ddb747fe009 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.627398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.628185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.628467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.628799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.629605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.632488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.657708 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2981/5954] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:22.657708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.665952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b3408-f8bc-4512-a677-c5f5ce7cff46 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.680157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5902de1-e237-4449-8b77-0166e59e8b7c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.685492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.689514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.689514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.689514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/f5f4fd321d0a41fe9724a4d935476826` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.699448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512f488-91e1-4966-88c0-f78b63cba312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.700435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512f488-91e1-4966-88c0-f78b63cba312 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.700726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512f488-91e1-4966-88c0-f78b63cba312 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.701282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512f488-91e1-4966-88c0-f78b63cba312 admin admin] PATH_INFO: `/v3/projects/7eea0d5995384120ab4f65114bdb9c65/users/101d81caee8d4569a15a7e0f42f5ddde/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.703859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Could not find domain: f5f4fd321d0a41fe9724a4d935476826.: keystone.exception.DomainNotFound: Could not find domain: f5f4fd321d0a41fe9724a4d935476826. Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f5f4fd321d0a41fe9724a4d935476826. Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:22.710078 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:22.711273 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f5f4fd321d0a41fe9724a4d935476826. Sep 17 21:13:22.712384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.713557 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2974/5955] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:22 2022] PATCH /identity/v3/domains/f5f4fd321d0a41fe9724a4d935476826 => generated 112 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:13:22.715490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.723631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512f488-91e1-4966-88c0-f78b63cba312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eea0d5995384120ab4f65114bdb9c65, user_id=101d81caee8d4569a15a7e0f42f5ddde, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:22.726674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512f488-91e1-4966-88c0-f78b63cba312 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:22.731792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f41793-e1cf-435b-9ebf-657a7ce1523b tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.737537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.738280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.738740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.739077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/c663420cd06f48b1aee690528aacf801` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.745136 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2982/5956] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:22 2022] PUT /identity/v3/projects/7eea0d5995384120ab4f65114bdb9c65/users/101d81caee8d4569a15a7e0f42f5ddde/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:22.756756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:13:22.758114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0639008-672a-48e7-876d-92af5ddcb662 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.758387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0639008-672a-48e7-876d-92af5ddcb662 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.758838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0639008-672a-48e7-876d-92af5ddcb662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Could not find domain: c663420cd06f48b1aee690528aacf801.: keystone.exception.DomainNotFound: Could not find domain: c663420cd06f48b1aee690528aacf801. Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c663420cd06f48b1aee690528aacf801. Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:22.762341 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:22.763717 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c663420cd06f48b1aee690528aacf801. Sep 17 21:13:22.764826 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.766097 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2975/5957] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:22 2022] PATCH /identity/v3/domains/c663420cd06f48b1aee690528aacf801 => generated 112 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:13:22.767514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.782797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:22.785420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd046a8-b976-4a85-af82-5572b0ea28e0 tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:22.807500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b0639008-672a-48e7-876d-92af5ddcb662 None None] MFA Rules not processed for user `101d81caee8d4569a15a7e0f42f5ddde`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:22.808962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0639008-672a-48e7-876d-92af5ddcb662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:22.866489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5085205c047641888857281b4a6ea37a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b5f9b61fb314906bccb0d6de96ab503', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:22.867550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:22.867940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:22.868561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] PATH_INFO: `/v3/domains/26fa33b79fb446e08684a326ea80d92a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:22.883687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Could not find domain: 26fa33b79fb446e08684a326ea80d92a.: keystone.exception.DomainNotFound: Could not find domain: 26fa33b79fb446e08684a326ea80d92a. Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 26fa33b79fb446e08684a326ea80d92a. Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:22.893223 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:22.894796 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 26fa33b79fb446e08684a326ea80d92a. Sep 17 21:13:22.896081 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:22.897486 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2976/5958] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:13:22 2022] PATCH /identity/v3/domains/26fa33b79fb446e08684a326ea80d92a => generated 112 bytes in 127 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:13:22.898940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.900906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:22.915551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2983/5959] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:22.920589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0639008-672a-48e7-876d-92af5ddcb662 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:22.932031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:23.015925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c73237-f9ee-456b-aa16-be189825090f tempest-TokensV3TestJSON-1762670774 tempest-TokensV3TestJSON-1762670774-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.020939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.021975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.022345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.022719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] PATH_INFO: `/v3/users/063583139dca4c2d94ac19baf096c122` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.024072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=063583139dca4c2d94ac19baf096c122)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.026485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.030374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0639008-672a-48e7-876d-92af5ddcb662 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.039717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.040714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.041688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.042216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.043118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.046341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.104274 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2984/5960] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:22 2022] POST /identity/v3/projects => generated 399 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:23.105755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.113765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063583139dca4c2d94ac19baf096c122'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.140567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063583139dca4c2d94ac19baf096c122'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.171344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063583139dca4c2d94ac19baf096c122'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.171880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063583139dca4c2d94ac19baf096c122'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.172596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1538bbf4-0ab2-48d0-9e90-c55c4ca51e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063583139dca4c2d94ac19baf096c122'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.189046 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2977/5961] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:22 2022] DELETE /identity/v3/users/063583139dca4c2d94ac19baf096c122 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:23.193920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.194317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.194867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.218436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e9316-be54-47a2-b21e-afc72b2dd3a4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.222651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:23.224348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:23.250735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:23.299813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.300463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.300925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.301276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.302028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.304041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.304669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2978/5962] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:23.314571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.316297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:23.336445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2985/5963] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:23 2022] POST /identity/v3/users => generated 306 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:23.340663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.345422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:23.365846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29293e4-b343-45da-8b1d-f39811e6116b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.371771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.372380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.372715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.372997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.374305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:23.374453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.375937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.394644 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2986/5964] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:23.396165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f33ad-0b29-418c-9d13-e2fa67d14cc9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.397276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.400525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.401165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.401614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.401892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.409489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22e09-6774-4670-9aa2-fadcd30b5f4f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.416039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.416817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.417160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.417439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] PATH_INFO: `/v3/projects/cb4545257180419cb934b6c06af8dd30/users/7144ae1f396c470d9ca04f6f2d650a8d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.417779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.419248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.432561 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2979/5965] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:23.439875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4545257180419cb934b6c06af8dd30, user_id=7144ae1f396c470d9ca04f6f2d650a8d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.443334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.461229 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2987/5966] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:23 2022] PUT /identity/v3/projects/cb4545257180419cb934b6c06af8dd30/users/7144ae1f396c470d9ca04f6f2d650a8d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:23.465626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.488396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c22dc3-febb-4732-868b-2381a8ba3e35 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.619332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec50db-5b4e-49f2-9093-69b141fe8676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.619971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec50db-5b4e-49f2-9093-69b141fe8676 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.620356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec50db-5b4e-49f2-9093-69b141fe8676 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.620635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec50db-5b4e-49f2-9093-69b141fe8676 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.621310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec50db-5b4e-49f2-9093-69b141fe8676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.623535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec50db-5b4e-49f2-9093-69b141fe8676 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.637733 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2980/5967] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:23.641540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.663853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d214d-2ce6-4929-b323-48a270486bd4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.677633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.678567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.678899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.679189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] PATH_INFO: `/v3/projects/cb4545257180419cb934b6c06af8dd30/users/7144ae1f396c470d9ca04f6f2d650a8d/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.681759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4545257180419cb934b6c06af8dd30, user_id=7144ae1f396c470d9ca04f6f2d650a8d, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.689177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.702091 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2988/5968] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:23 2022] PUT /identity/v3/projects/cb4545257180419cb934b6c06af8dd30/users/7144ae1f396c470d9ca04f6f2d650a8d/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:23.712906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.713182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.713398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.755922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] MFA Rules not processed for user `7144ae1f396c470d9ca04f6f2d650a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:23.757640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:23.770625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.772873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:23.859498 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2981/5969] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:23.872208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b256f3-c341-4772-90fb-27b9d95cd1a5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.872532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:23.874445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:23.878267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.879073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.879349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.879632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] PATH_INFO: `/v3/users/0fc783e858c14bedba61cb07b3a9130d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.880806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc783e858c14bedba61cb07b3a9130d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:23.883736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:23.967027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc783e858c14bedba61cb07b3a9130d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.967669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ba51-3c6f-4baf-8d49-20a7a821d5ea None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:23.967850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc783e858c14bedba61cb07b3a9130d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.971995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e67628-d719-4b21-a46d-fec1c8394f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:23.972773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e67628-d719-4b21-a46d-fec1c8394f80 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:23.973186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e67628-d719-4b21-a46d-fec1c8394f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:23.973614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e67628-d719-4b21-a46d-fec1c8394f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:23.976408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-23e67628-d719-4b21-a46d-fec1c8394f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:23.976857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc783e858c14bedba61cb07b3a9130d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.977677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc783e858c14bedba61cb07b3a9130d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:23.998696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc783e858c14bedba61cb07b3a9130d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.000522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:24.020596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:24.025019 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2989/5970] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:23 2022] DELETE /identity/v3/users/0fc783e858c14bedba61cb07b3a9130d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:24.026077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:24.029262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:24.074401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e67628-d719-4b21-a46d-fec1c8394f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:24.076166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e67628-d719-4b21-a46d-fec1c8394f80 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:24.086760 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2982/5971] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:23 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:24.087965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:24.088246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:24.088517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:24.110011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:24.111213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:24.120975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dfbc4-414b-4505-9dcf-2ae8228051c5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:24.123495 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2983/5972] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:24.135337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:24.135684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:24.137895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:24.137895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:24.138292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] PATH_INFO: `/v3/projects/42783e4d01654ae7a43b8819a8a91b66` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:24.139670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42783e4d01654ae7a43b8819a8a91b66)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:24.140608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:24.144045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:24.239097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42783e4d01654ae7a43b8819a8a91b66', 'request_id': 'req-1ff43de8-737b-486a-859f-ad33872cb674'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.258072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42783e4d01654ae7a43b8819a8a91b66', 'request_id': 'req-1ff43de8-737b-486a-859f-ad33872cb674'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.263274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a10f2-dd4f-4444-8b2e-968a486b56d1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:24.271682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42783e4d01654ae7a43b8819a8a91b66', 'request_id': 'req-1ff43de8-737b-486a-859f-ad33872cb674'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.292432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] The token cache is being invalidate because project 42783e4d01654ae7a43b8819a8a91b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:24.292721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.293613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.296207 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2990/5973] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:24 2022] DELETE /identity/v3/projects/42783e4d01654ae7a43b8819a8a91b66 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:24.301531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:24.303763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:24.374684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:24.375388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:24.375590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:24.376475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:24.381513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:24.427992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff43de8-737b-486a-859f-ad33872cb674 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:24.439865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:24.440631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:24.441000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:24.444124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] PATH_INFO: `/v3/users/5085205c047641888857281b4a6ea37a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:24.445365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5085205c047641888857281b4a6ea37a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:24.447531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:24.473441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:24.492506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:24.494479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:24.511995 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2984/5974] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:24.516407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5085205c047641888857281b4a6ea37a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.517604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5085205c047641888857281b4a6ea37a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.531717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5085205c047641888857281b4a6ea37a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.532602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5085205c047641888857281b4a6ea37a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.570091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5085205c047641888857281b4a6ea37a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:24.612191 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2991/5975] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:24 2022] DELETE /identity/v3/users/5085205c047641888857281b4a6ea37a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:24.899026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:24.899439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:24.903115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:24.905124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:24.954696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:25.027252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd306-96b3-4af8-938e-e9fc1d70343c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.033868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.034659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.035067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.035440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] PATH_INFO: `/v3/projects/8af940fb16b948fcb49befa103ec8d0e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.037382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af940fb16b948fcb49befa103ec8d0e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.039854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.112729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57967c-decf-473a-ab35-de252ed4257c service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.165337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af940fb16b948fcb49befa103ec8d0e', 'request_id': 'req-7e67e72a-522a-40be-9d7e-d7f22a1c60be'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.189819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af940fb16b948fcb49befa103ec8d0e', 'request_id': 'req-7e67e72a-522a-40be-9d7e-d7f22a1c60be'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.204818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af940fb16b948fcb49befa103ec8d0e', 'request_id': 'req-7e67e72a-522a-40be-9d7e-d7f22a1c60be'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.221847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] The token cache is being invalidate because project 8af940fb16b948fcb49befa103ec8d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:25.222135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.222799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e67e72a-522a-40be-9d7e-d7f22a1c60be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.225556 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2992/5976] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:24 2022] DELETE /identity/v3/projects/8af940fb16b948fcb49befa103ec8d0e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:25.231972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.232274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.233756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.270434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:25.271688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.280031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85064e-d3d2-481b-84d5-43145b6c9347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.280676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85064e-d3d2-481b-84d5-43145b6c9347 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.280951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85064e-d3d2-481b-84d5-43145b6c9347 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.281218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85064e-d3d2-481b-84d5-43145b6c9347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.283624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6f85064e-d3d2-481b-84d5-43145b6c9347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.305926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:25.355168 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2993/5977] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:25.356239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:25.358816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.360057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85064e-d3d2-481b-84d5-43145b6c9347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.361934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85064e-d3d2-481b-84d5-43145b6c9347 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.372484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760e4e7-44e2-4dac-853a-47d0aca12af4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.374212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2985/5978] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:24 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:25.381284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.382231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.383572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.383937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] PATH_INFO: `/v3/projects/3b5f9b61fb314906bccb0d6de96ab503` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.386830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b5f9b61fb314906bccb0d6de96ab503)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.387150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c84975-e836-4c88-9980-104b33d305fe None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.387433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c84975-e836-4c88-9980-104b33d305fe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.387702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c84975-e836-4c88-9980-104b33d305fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.390268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.422699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a3c84975-e836-4c88-9980-104b33d305fe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:25.423915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a3c84975-e836-4c88-9980-104b33d305fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.469895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2986/5979] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:25.474253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c84975-e836-4c88-9980-104b33d305fe None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:25.479316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b5f9b61fb314906bccb0d6de96ab503', 'request_id': 'req-fff28c12-3422-4f99-986e-417012dbaa91'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.488559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b5f9b61fb314906bccb0d6de96ab503', 'request_id': 'req-fff28c12-3422-4f99-986e-417012dbaa91'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.489951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c84975-e836-4c88-9980-104b33d305fe None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.501237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b5f9b61fb314906bccb0d6de96ab503', 'request_id': 'req-fff28c12-3422-4f99-986e-417012dbaa91'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.514726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] The token cache is being invalidate because project 3b5f9b61fb314906bccb0d6de96ab503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:25.515016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.515728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.520573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2994/5980] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:25 2022] DELETE /identity/v3/projects/3b5f9b61fb314906bccb0d6de96ab503 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:25.521729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:25.523334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.569419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:25.594854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.595896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.597065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.597520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.599049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:25.599546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.601968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.624652 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2987/5981] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:25.626099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:25.628474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.644137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9e8a7-6110-4570-a2b5-196b85fbbb70 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.644491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff28c12-3422-4f99-986e-417012dbaa91 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.649446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.650192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.650533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.650653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.650832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.651352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.651637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.651766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.652173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] PATH_INFO: `/v3/users/1abe9cd0f8714c5c9341801b154f080e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.653354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abe9cd0f8714c5c9341801b154f080e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.655327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.656165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.710293 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2988/5982] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:25 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:25.715452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:25.749872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abe9cd0f8714c5c9341801b154f080e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.750408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abe9cd0f8714c5c9341801b154f080e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.766886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abe9cd0f8714c5c9341801b154f080e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.767877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abe9cd0f8714c5c9341801b154f080e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.770157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:25.787843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abe9cd0f8714c5c9341801b154f080e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:25.813116 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2995/5983] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:25 2022] DELETE /identity/v3/users/1abe9cd0f8714c5c9341801b154f080e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:25.842597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03164a-9832-49e7-aae9-c32de9fcb773 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.948516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.949224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.949714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.949992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.951460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.953683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.986268 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2989/5984] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:25 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:25.990620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:25.992793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.008714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562155d-a24b-4615-b7fb-02b59cd5079f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:26.015124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:26.015861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.016520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.016520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.017158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:26.019075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:26.044525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:26.052086 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2996/5985] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:26.057440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db2daa-41ff-4546-b92a-241cb444e28c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:26.062436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe1b9-647b-4ee4-94ff-c6402c436f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:26.063050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe1b9-647b-4ee4-94ff-c6402c436f23 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.063630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe1b9-647b-4ee4-94ff-c6402c436f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.063905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe1b9-647b-4ee4-94ff-c6402c436f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.064696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffe1b9-647b-4ee4-94ff-c6402c436f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:26.066388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffe1b9-647b-4ee4-94ff-c6402c436f23 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:26.081418 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2990/5986] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:26.085942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:26.099003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78972aac-60c4-4c56-a36c-b1665b7788dc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:26.106851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:26.107832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.108324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.109678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] PATH_INFO: `/v3/projects/282a01b9b14640cb9918486126076716/users/1d37dcc7008f4edd98e5c040fc605e61/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.130576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282a01b9b14640cb9918486126076716, user_id=1d37dcc7008f4edd98e5c040fc605e61, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:26.134200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:26.154601 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2997/5987] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:26 2022] PUT /identity/v3/projects/282a01b9b14640cb9918486126076716/users/1d37dcc7008f4edd98e5c040fc605e61/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:26.164554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.164868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.165143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.174910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:26.191323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.221838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] MFA Rules not processed for user `1d37dcc7008f4edd98e5c040fc605e61`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:26.223689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.267918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:26.321448 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2991/5988] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:26.322902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:26.326093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.358802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08f498-796f-4f39-b14b-ca2ea93f50f0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:26.379089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:26.379782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.380060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.380343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] PATH_INFO: `/v3/users/a61d2ca32f8c48cdb67b10fdced8a721` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.388056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61d2ca32f8c48cdb67b10fdced8a721)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:26.390783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:26.424015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5cc3-640e-4a02-b1fb-cb68bdaea503 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:26.428879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:26.429890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.430269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.430638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] PATH_INFO: `/v3/projects/1cb3db8b80c34e8380aad24cc9ca742a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.431981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb3db8b80c34e8380aad24cc9ca742a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:26.434235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:26.504719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61d2ca32f8c48cdb67b10fdced8a721'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.505702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61d2ca32f8c48cdb67b10fdced8a721'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.519473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb3db8b80c34e8380aad24cc9ca742a', 'request_id': 'req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.527310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61d2ca32f8c48cdb67b10fdced8a721'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.528288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61d2ca32f8c48cdb67b10fdced8a721'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.549694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb3db8b80c34e8380aad24cc9ca742a', 'request_id': 'req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.557673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb3db8b80c34e8380aad24cc9ca742a', 'request_id': 'req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.560271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9acad54f-f1a5-4f84-9b22-4c9b6e09bb5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61d2ca32f8c48cdb67b10fdced8a721'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.588927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] The token cache is being invalidate because project 1cb3db8b80c34e8380aad24cc9ca742a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:26.589400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.591160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:26.600817 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2992/5989] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:26 2022] DELETE /identity/v3/projects/1cb3db8b80c34e8380aad24cc9ca742a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:26.602832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:26.603305 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2998/5990] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:26 2022] DELETE /identity/v3/users/a61d2ca32f8c48cdb67b10fdced8a721 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:26.605612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70668019-1ce1-4ca4-8e76-1960789a5f0d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.605918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70668019-1ce1-4ca4-8e76-1960789a5f0d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.606468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70668019-1ce1-4ca4-8e76-1960789a5f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.608799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.657608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-70668019-1ce1-4ca4-8e76-1960789a5f0d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:26.658978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70668019-1ce1-4ca4-8e76-1960789a5f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.671726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:26.731413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cce15-d08e-458f-a796-deb2b5ba7ddb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:26.735764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:26.774293 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 2999/5991] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:26.776174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302efdc-9e8e-41dc-874d-1aaf37ae793f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.778866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302efdc-9e8e-41dc-874d-1aaf37ae793f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.779156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302efdc-9e8e-41dc-874d-1aaf37ae793f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.816546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:26.817948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.818456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.819439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.820444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:26.822984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:26.824209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d302efdc-9e8e-41dc-874d-1aaf37ae793f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:26.825478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d302efdc-9e8e-41dc-874d-1aaf37ae793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.847211 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3000/5992] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:26.849837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a23c04-4feb-4589-b575-1fbe6749aec2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.850138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a23c04-4feb-4589-b575-1fbe6749aec2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.850965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a23c04-4feb-4589-b575-1fbe6749aec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.868383 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2993/5993] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:26 2022] POST /identity/v3/projects => generated 383 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:26.872298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:26.875698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.898078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e8a23c04-4feb-4589-b575-1fbe6749aec2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:26.899328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e8a23c04-4feb-4589-b575-1fbe6749aec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.966602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465b3a8-5faf-4e24-a60a-50fc63aa7532 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:26.973446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:26.974625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.975036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.975803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.049547 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3001/5994] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:27.049547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a23c04-4feb-4589-b575-1fbe6749aec2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.075934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a23c04-4feb-4589-b575-1fbe6749aec2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.082696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.083325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.083601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.084163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.086213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.089336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.089527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.091380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.107949 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2994/5995] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:27.109062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.126536 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3002/5996] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:27 2022] POST /identity/v3/users => generated 298 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:27.130862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.134728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fedae-a433-451c-9d70-745af5c05706 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.148442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97178dd-9ba3-4e4e-8688-1c78f72a84c4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.152516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.153586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.154118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.154273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.155737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.155887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:27.156253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.157853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.158361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.158795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.159751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.160773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.161952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.174981 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3003/5997] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:27.179010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.192010 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2995/5998] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:27.198670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.198874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1954b-e8d0-4c08-818a-5fad07cd0f52 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.224974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.226409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94288110-6c44-4283-a347-747ffda9904c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.226799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.227167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.227441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] PATH_INFO: `/v3/projects/903ce733beb7418aa13ad358335cec2d/users/a2183ed8377243509a9c5fde0a4b32fe/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.264278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.265003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.265356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.265701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.266370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.268891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.277453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903ce733beb7418aa13ad358335cec2d, user_id=a2183ed8377243509a9c5fde0a4b32fe, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.285207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.320210 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3004/5999] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:27 2022] PUT /identity/v3/projects/903ce733beb7418aa13ad358335cec2d/users/a2183ed8377243509a9c5fde0a4b32fe/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:27.326028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.350648 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2996/6000] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:27 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:27.354864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.369598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:27.404678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:27.413318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74889dcd-315d-4f68-8b75-e51f150480c7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.492527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34f5ff-b6ed-4f5c-b2cf-8bd7575f883b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.501796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.502771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.503337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.503943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.505119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.507594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.552495 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2997/6001] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:27 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:27.556476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.594109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9be15-92c5-4a66-90b8-dca572250457 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.596903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.597558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.597839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.598209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.599646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.601518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.602253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.603568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.603917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.606051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.607968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.610590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.615654 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3005/6002] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:27.620415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.637921 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2998/6003] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:27.641286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.646496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d2c23-37d5-46c3-a094-000e2ab98cca admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.655221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.655909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.659482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.659776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] PATH_INFO: `/v3/projects/903ce733beb7418aa13ad358335cec2d/users/a2183ed8377243509a9c5fde0a4b32fe/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.661564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3e749-b9de-4b03-ac4f-b6199c473cf6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.663484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903ce733beb7418aa13ad358335cec2d, user_id=a2183ed8377243509a9c5fde0a4b32fe, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.680735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.685577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80984a-90a7-47f6-ada1-075b73556003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.697359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80984a-90a7-47f6-ada1-075b73556003 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.697577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80984a-90a7-47f6-ada1-075b73556003 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.697858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80984a-90a7-47f6-ada1-075b73556003 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.699052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80984a-90a7-47f6-ada1-075b73556003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.702838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80984a-90a7-47f6-ada1-075b73556003 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.718296 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3006/6004] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:27 2022] PUT /identity/v3/projects/903ce733beb7418aa13ad358335cec2d/users/a2183ed8377243509a9c5fde0a4b32fe/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:27.719042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.723015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:27.734899 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 2999/6005] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:27.736594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768a08-dbbc-408c-a2c8-a80f70c895c2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.737043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768a08-dbbc-408c-a2c8-a80f70c895c2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.738069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768a08-dbbc-408c-a2c8-a80f70c895c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.790165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5a768a08-dbbc-408c-a2c8-a80f70c895c2 None None] MFA Rules not processed for user `a2183ed8377243509a9c5fde0a4b32fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:27.791389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5a768a08-dbbc-408c-a2c8-a80f70c895c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:27.814965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86f7e-37e8-49fe-8e5e-f1d2fb065656 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.822284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:27.823129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:27.823506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:27.823874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] PATH_INFO: `/v3/projects/9dd370c8e76749f69f6075678713bd39` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:27.825155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd370c8e76749f69f6075678713bd39)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:27.827678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:27.895420 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3000/6006] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:27.899548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a768a08-dbbc-408c-a2c8-a80f70c895c2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.904335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd370c8e76749f69f6075678713bd39', 'request_id': 'req-73f88fb8-84f0-4302-969b-4bcd3bcbde87'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:27.908215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a768a08-dbbc-408c-a2c8-a80f70c895c2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:27.914764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd370c8e76749f69f6075678713bd39', 'request_id': 'req-73f88fb8-84f0-4302-969b-4bcd3bcbde87'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:27.926581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd370c8e76749f69f6075678713bd39', 'request_id': 'req-73f88fb8-84f0-4302-969b-4bcd3bcbde87'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:27.943710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] The token cache is being invalidate because project 9dd370c8e76749f69f6075678713bd39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:27.944132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:27.944813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:27.947835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3007/6007] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:27 2022] DELETE /identity/v3/projects/9dd370c8e76749f69f6075678713bd39 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:27.948800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:27.952905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:27.988383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:28.000025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.001332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.001902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.003149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] PATH_INFO: `/v3/projects/327a4497ea0547a7b3e1cf94f68bd6f5/users/a41b7620772e4acd86b3b80ec2c4b3a9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.009044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:28.032047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327a4497ea0547a7b3e1cf94f68bd6f5, user_id=a41b7620772e4acd86b3b80ec2c4b3a9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.036331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.042446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f88fb8-84f0-4302-969b-4bcd3bcbde87 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.046760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fc029-c2e2-4318-84e4-7e3ba66e2161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.047444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc029-c2e2-4318-84e4-7e3ba66e2161 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.047680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc029-c2e2-4318-84e4-7e3ba66e2161 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.047925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc029-c2e2-4318-84e4-7e3ba66e2161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.050555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-736fc029-c2e2-4318-84e4-7e3ba66e2161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:28.059198 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3001/6008] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:27 2022] PUT /identity/v3/projects/327a4497ea0547a7b3e1cf94f68bd6f5/users/a41b7620772e4acd86b3b80ec2c4b3a9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:28.060969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:28.063920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:28.144102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fc029-c2e2-4318-84e4-7e3ba66e2161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.145976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fc029-c2e2-4318-84e4-7e3ba66e2161 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.157737 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3008/6009] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:28.159521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33581648-d6f9-4d0d-813d-51129d099cad None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.159917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33581648-d6f9-4d0d-813d-51129d099cad None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.160286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33581648-d6f9-4d0d-813d-51129d099cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.163932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a1cd0-f55f-45c9-bd97-be3652eee187 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.169225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.169928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.170278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.170551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] PATH_INFO: `/v3/users/536fbff9200d4e74a74cdd9b1a67fa48` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.171939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=536fbff9200d4e74a74cdd9b1a67fa48)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.174695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.197245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-33581648-d6f9-4d0d-813d-51129d099cad None None] MFA Rules not processed for user `a41b7620772e4acd86b3b80ec2c4b3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:28.199213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-33581648-d6f9-4d0d-813d-51129d099cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:28.246721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536fbff9200d4e74a74cdd9b1a67fa48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:28.273554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536fbff9200d4e74a74cdd9b1a67fa48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:28.295397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536fbff9200d4e74a74cdd9b1a67fa48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:28.296663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536fbff9200d4e74a74cdd9b1a67fa48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:28.297728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536fbff9200d4e74a74cdd9b1a67fa48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:28.326452 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3002/6010] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:28 2022] DELETE /identity/v3/users/536fbff9200d4e74a74cdd9b1a67fa48 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:28.339098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3009/6011] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:28.345736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:28.348540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:28.437302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7302c57-e6ce-4b33-a6d9-62188275f6c5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.443446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.444646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.444646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.444887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.446674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.447813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.454723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:28.485535 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3003/6012] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:28 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:28.488952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33581648-d6f9-4d0d-813d-51129d099cad None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:28.504565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33581648-d6f9-4d0d-813d-51129d099cad None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.578750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.579538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.579935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.580727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.581053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.583497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.594062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:28.617808 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3010/6013] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:28 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:28.622267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:28.636030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2f25c-bbdf-446b-ba5c-5024337d0bee admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.646373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.647173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.647540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.647907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.648654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.650408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.682796 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3004/6014] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:28.687167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:28.707222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4dcf2-de92-4d25-b42a-79da0994a499 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.716107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.716841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.717170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.717525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] PATH_INFO: `/v3/projects/5c9b436b63244be5a318d91932324b67/users/81e1255fe7c1439da39365e0d9f55fe1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.755449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9b436b63244be5a318d91932324b67, user_id=81e1255fe7c1439da39365e0d9f55fe1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.758574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.792324 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3011/6015] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:28 2022] PUT /identity/v3/projects/5c9b436b63244be5a318d91932324b67/users/81e1255fe7c1439da39365e0d9f55fe1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:28.796197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:28.809192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010c991-2cd0-411a-8146-5f6dc73955fa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.882221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf51c2-124c-4072-8df2-2fd21604c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.882501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf51c2-124c-4072-8df2-2fd21604c00c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.882854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf51c2-124c-4072-8df2-2fd21604c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.883363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf51c2-124c-4072-8df2-2fd21604c00c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.883952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf51c2-124c-4072-8df2-2fd21604c00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.886097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf51c2-124c-4072-8df2-2fd21604c00c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.901889 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3005/6016] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:28.903745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:28.924187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebc084-cc40-4360-a341-793bd5e06ece admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:28.936881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:28.937839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.938213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.939079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] PATH_INFO: `/v3/projects/5c9b436b63244be5a318d91932324b67/users/81e1255fe7c1439da39365e0d9f55fe1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:28.941371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9b436b63244be5a318d91932324b67, user_id=81e1255fe7c1439da39365e0d9f55fe1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:28.944719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:28.957653 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3012/6017] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:28 2022] PUT /identity/v3/projects/5c9b436b63244be5a318d91932324b67/users/81e1255fe7c1439da39365e0d9f55fe1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:28.968604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:28.969012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:28.969373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.012153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] MFA Rules not processed for user `81e1255fe7c1439da39365e0d9f55fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:29.014266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.135930 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3006/6018] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:29.141779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.166697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.172920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.188188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d248fd-dbae-487a-93b6-216b8e7a80c7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.200371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e1255fe7c1439da39365e0d9f55fe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9b436b63244be5a318d91932324b67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.203020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.203317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.203594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.208240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.211989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:29.293830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ddf3b-96b8-4953-b032-75350c4146e6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.294949 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3013/6019] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:29 2022] POST /identity/v3/projects => generated 359 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:29.296785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.319076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:29.393071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.395300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.395927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.396533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.404365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524921-d9b7-4b10-ade8-2a96cee8de31 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.410974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.419327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e1255fe7c1439da39365e0d9f55fe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9b436b63244be5a318d91932324b67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.419868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.420627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.420627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.421137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.423179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:29.478051 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3014/6020] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:29 2022] POST /identity/v3/users => generated 371 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:29.480671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.483166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.544298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.548386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:29.560725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece95b4f-4512-4aca-870a-84a8929eac04 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.569527 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3007/6021] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:29.569717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.570406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.570760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.571037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] PATH_INFO: `/v3/projects/f7c2bd2962c647748617610d30f56648` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.571927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.572279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c2bd2962c647748617610d30f56648)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.574853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:29.591019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa3602-0d0a-4dd5-aa12-e256d670f857 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.598853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e1255fe7c1439da39365e0d9f55fe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9b436b63244be5a318d91932324b67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.600745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.602646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.603529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.605601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:29.606354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.607912 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:13:29.607912 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:13:29.607912 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:13:29.610948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:29.612631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=8,keystone:SELECT=91,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:29.650557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c2bd2962c647748617610d30f56648', 'request_id': 'req-7b547ce8-faa2-4a58-9860-5611f1a2c420'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.662430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c2bd2962c647748617610d30f56648', 'request_id': 'req-7b547ce8-faa2-4a58-9860-5611f1a2c420'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.669133 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3008/6022] 10.222.0.26 () {58 vars in 1157 bytes} [Sat Sep 17 21:13:29 2022] GET /identity/v3/users?domain_id=default => generated 8915 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:29.675165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.694498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c2bd2962c647748617610d30f56648', 'request_id': 'req-7b547ce8-faa2-4a58-9860-5611f1a2c420'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.702233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd21f2-3b4c-4f21-80cd-5a247c4bc7bf tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.710034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] The token cache is being invalidate because project f7c2bd2962c647748617610d30f56648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:29.711465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.713016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b547ce8-faa2-4a58-9860-5611f1a2c420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.718762 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3015/6023] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:29 2022] DELETE /identity/v3/projects/f7c2bd2962c647748617610d30f56648 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:29.724011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6652-82a9-413f-a458-794f8788f153 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.725593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6652-82a9-413f-a458-794f8788f153 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.726139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6652-82a9-413f-a458-794f8788f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.763080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-985f6652-82a9-413f-a458-794f8788f153 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:29.764990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-985f6652-82a9-413f-a458-794f8788f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.831380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e1255fe7c1439da39365e0d9f55fe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9b436b63244be5a318d91932324b67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.832736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.833023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.833298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] PATH_INFO: `/v3/users/8b90dfffd38e46f9a20c1efa10d648a3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.851967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorizing `identity:update_user(user_id=8b90dfffd38e46f9a20c1efa10d648a3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.854110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:29.869693 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3016/6024] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:29.871337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34f1f-c737-40ea-b51e-f3fa4c2c07b7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.871629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34f1f-c737-40ea-b51e-f3fa4c2c07b7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.871903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34f1f-c737-40ea-b51e-f3fa4c2c07b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.900634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8aa34f1f-c737-40ea-b51e-f3fa4c2c07b7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:29.901948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8aa34f1f-c737-40ea-b51e-f3fa4c2c07b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.905442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '8b90dfffd38e46f9a20c1efa10d648a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.911933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3017/6025] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:29.916238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34f1f-c737-40ea-b51e-f3fa4c2c07b7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.916860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invalidating the token cache because user 8b90dfffd38e46f9a20c1efa10d648a3 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:29.917627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.919354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-69bd46c1-d883-4532-b5eb-2919244d55a0 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:29.924851 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3009/6026] 10.222.0.26 () {60 vars in 1244 bytes} [Sat Sep 17 21:13:29 2022] PATCH /identity/v3/users/8b90dfffd38e46f9a20c1efa10d648a3 => generated 485 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:29.926886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.927479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.927747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.928747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34f1f-c737-40ea-b51e-f3fa4c2c07b7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.935221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.936047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.936177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.936500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.937439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:29.937815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.941511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:29.952794 ncdv-master-1 devstack@keystone.service[99166]: WARNING keystone.auth.plugins.core [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] User is disabled: 8b90dfffd38e46f9a20c1efa10d648a3: AssertionError: User is disabled: 8b90dfffd38e46f9a20c1efa10d648a3 Sep 17 21:13:29.958518 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3018/6027] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:29.962703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.964181 ncdv-master-1 devstack@keystone.service[99166]: WARNING keystone.server.flask.application [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 17 21:13:29.964668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.966477 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3010/6028] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:29 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 41 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Sep 17 21:13:29.974334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:29.977667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:29.984610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd243f-a64e-4586-b057-8624a1a3309a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:29.989870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:29.990740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:29.991092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:29.991433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:29.992244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:29.994665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.002139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa141b1b-2d35-41b6-86a5-b44d0de24ba7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:30.009094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e1255fe7c1439da39365e0d9f55fe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9b436b63244be5a318d91932324b67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:30.010579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.011097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.011594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] PATH_INFO: `/v3/users/8b90dfffd38e46f9a20c1efa10d648a3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.043616 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3019/6029] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:29 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:30.046221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:30.070694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:30.082005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8b90dfffd38e46f9a20c1efa10d648a3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:30.091145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.153491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ee95b-21f2-4130-99b4-0f2ebc439879 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:30.161410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:30.162128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.162604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.162837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.163538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:30.167305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.201891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:30.213629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b90dfffd38e46f9a20c1efa10d648a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.215870 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3020/6030] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:30 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:30.221674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:30.244308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdac7e-99cc-49ce-b49d-33d877fda952 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:30.248912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b90dfffd38e46f9a20c1efa10d648a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.254353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:30.255985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.256422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.256890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.257940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:30.260600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.268149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b90dfffd38e46f9a20c1efa10d648a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.272449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b90dfffd38e46f9a20c1efa10d648a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.273165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b90dfffd38e46f9a20c1efa10d648a3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.291896 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3021/6031] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:30.296355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:30.297621 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3011/6032] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:29 2022] DELETE /identity/v3/users/8b90dfffd38e46f9a20c1efa10d648a3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:30.302260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:30.302980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:30.303630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:30.330217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5872c-3eeb-4bd5-819e-52ac1ab02e86 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:30.485653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9600df0-05e5-4bb5-aa06-e0c4b5034ad1 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:30.497228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e1255fe7c1439da39365e0d9f55fe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9b436b63244be5a318d91932324b67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:30.498564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.499036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.499474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] PATH_INFO: `/v3/projects/695efd7749454054884d93d4648c6552` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.502955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorizing `identity:delete_project(project_id=695efd7749454054884d93d4648c6552)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:30.506298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.516339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:30.517619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.517619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.517848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.518441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:30.520403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.539562 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3022/6033] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:30.541208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:30.607031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4373eff-a3ad-4241-8e6a-660a3a67506e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:30.616759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695efd7749454054884d93d4648c6552', 'request_id': 'req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.639253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695efd7749454054884d93d4648c6552', 'request_id': 'req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.647485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695efd7749454054884d93d4648c6552', 'request_id': 'req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.675982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] The token cache is being invalidate because project 695efd7749454054884d93d4648c6552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:30.677057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.677994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:30.680049 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3012/6034] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:30 2022] DELETE /identity/v3/projects/695efd7749454054884d93d4648c6552 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:30.685106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:30.688321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:30.701671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:30.806878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:30.809288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.810907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.811936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] PATH_INFO: `/v3/projects/203653c6a42345ee8fe68f0a0bc64a7d/users/5ae3425f3c704ece85cf9c6b54949fac/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.839288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80738d4e-e8f4-4c8f-baf0-b44f27a5fbc8 tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:30.846904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=203653c6a42345ee8fe68f0a0bc64a7d, user_id=5ae3425f3c704ece85cf9c6b54949fac, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:30.848712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1da494-1d69-41e6-b252-85a84768c50f tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e1255fe7c1439da39365e0d9f55fe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9b436b63244be5a318d91932324b67', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:30.850289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1da494-1d69-41e6-b252-85a84768c50f tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.850872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1da494-1d69-41e6-b252-85a84768c50f tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.851901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1da494-1d69-41e6-b252-85a84768c50f tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.854167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1da494-1d69-41e6-b252-85a84768c50f tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:30.854499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.857129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1da494-1d69-41e6-b252-85a84768c50f tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:30.885232 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3023/6035] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:30 2022] PUT /identity/v3/projects/203653c6a42345ee8fe68f0a0bc64a7d/users/5ae3425f3c704ece85cf9c6b54949fac/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-8f1da494-1d69-41e6-b252-85a84768c50f tempest-UsersNegativeTest-633664227 tempest-UsersNegativeTest-633664227-project-admin] Could not find domain: 19a1bfbd8c504a7facf2592d22ea01d0.: keystone.exception.DomainNotFound: Could not find domain: 19a1bfbd8c504a7facf2592d22ea01d0. Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 19a1bfbd8c504a7facf2592d22ea01d0. Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:13:30.887438 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Sep 17 21:13:30.888773 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:13:30.890107 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:13:30.891353 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:13:30.891353 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 19a1bfbd8c504a7facf2592d22ea01d0. Sep 17 21:13:30.891353 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:13:30.894963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:30.897514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:30.898594 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3013/6036] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:30 2022] POST /identity/v3/users => generated 112 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:13:30.900753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:30.901101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:30.902534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:30.969624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] MFA Rules not processed for user `5ae3425f3c704ece85cf9c6b54949fac`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:30.973007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.054736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cec77a-fe4e-47c3-b670-610fe143d20a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.070130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.071060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.071723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.072091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] PATH_INFO: `/v3/users/a41b7620772e4acd86b3b80ec2c4b3a9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.075929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41b7620772e4acd86b3b80ec2c4b3a9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.082386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.113843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:31.142828 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3014/6037] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:31.147097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.152153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.183036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca236518-7c6b-46b6-af9a-6232b5fe0525 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.191918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41b7620772e4acd86b3b80ec2c4b3a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:31.193035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41b7620772e4acd86b3b80ec2c4b3a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:31.198504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.199246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.200514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.200924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.201704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.204389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.213843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41b7620772e4acd86b3b80ec2c4b3a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:31.214768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41b7620772e4acd86b3b80ec2c4b3a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:31.239356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e64e8e54-7e6e-429c-9a2e-0321620ac7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41b7620772e4acd86b3b80ec2c4b3a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:31.268455 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3024/6038] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:30 2022] DELETE /identity/v3/users/a41b7620772e4acd86b3b80ec2c4b3a9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:31.272958 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3015/6039] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:31 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:31.274342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.275736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.276070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.276383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.276695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.303052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:31.304715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.350806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7372064-1c1d-41fa-8fc0-35ace1d4d0fe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.357237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.358200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.358575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.358936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.359726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.361891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.390158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:31.393669 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3025/6040] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:31.404128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.406438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.412849 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3016/6041] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:31 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:31.414716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.430871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5089e-d341-4527-bc34-14013f64b0c5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.436802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.437652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.439138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.439696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.440408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.443028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.466342 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3017/6042] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:31.468492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.485374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6d841-b2ec-475e-ab7c-c8097e2d24b8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.485668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3740b42-8352-4dae-8812-ef4e507c49ca None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.489754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.490594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.490960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.491367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.491552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.492318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.492689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.492967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] PATH_INFO: `/v3/projects/4937c938251c4205b713c49ccd707775/users/991fa7d515ac4460ad5d95dc7b5e04af/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.503905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.506018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.513081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4937c938251c4205b713c49ccd707775, user_id=991fa7d515ac4460ad5d95dc7b5e04af, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.516415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.524679 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3026/6043] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:31.538581 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3018/6044] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:31 2022] PUT /identity/v3/projects/4937c938251c4205b713c49ccd707775/users/991fa7d515ac4460ad5d95dc7b5e04af/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:31.542573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.553646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7fa1c-b7f7-4218-92db-eb8033f7614f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.610240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:31.655792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcd4ae-5aa6-4166-8e39-cf8c4a6123a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.657331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcd4ae-5aa6-4166-8e39-cf8c4a6123a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.657959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcd4ae-5aa6-4166-8e39-cf8c4a6123a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.658451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcd4ae-5aa6-4166-8e39-cf8c4a6123a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.659321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcd4ae-5aa6-4166-8e39-cf8c4a6123a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.661740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcd4ae-5aa6-4166-8e39-cf8c4a6123a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.679799 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3027/6045] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:31.680612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.721489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2003982-4c6a-4a3c-9df9-a80177215596 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.721489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.721489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.721489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.722146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] PATH_INFO: `/v3/projects/4937c938251c4205b713c49ccd707775/users/991fa7d515ac4460ad5d95dc7b5e04af/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.722146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4937c938251c4205b713c49ccd707775, user_id=991fa7d515ac4460ad5d95dc7b5e04af, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:31.722146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:31.738294 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3019/6046] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:31 2022] PUT /identity/v3/projects/4937c938251c4205b713c49ccd707775/users/991fa7d515ac4460ad5d95dc7b5e04af/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:31.751053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.751351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.751629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.788899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] MFA Rules not processed for user `991fa7d515ac4460ad5d95dc7b5e04af`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:31.790191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.873502 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3028/6047] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:31.883155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.884899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.971484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04226b40-58da-4a2f-8768-450bb8e976cc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:31.976744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:31.977692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:31.978134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:31.979706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:31.982656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:31.988404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:31.989865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:32.135827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:32.139261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:32.177739 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3020/6048] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:32.201988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0864e56-f6be-4869-ba5f-fe38e31f3450 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:32.211384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:32.213639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:32.214613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:32.215089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] PATH_INFO: `/v3/projects/327a4497ea0547a7b3e1cf94f68bd6f5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:32.217252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327a4497ea0547a7b3e1cf94f68bd6f5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:32.219648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:32.364944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327a4497ea0547a7b3e1cf94f68bd6f5', 'request_id': 'req-9808249b-4d8b-4a56-ae5c-18da169e5911'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.385587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327a4497ea0547a7b3e1cf94f68bd6f5', 'request_id': 'req-9808249b-4d8b-4a56-ae5c-18da169e5911'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.387720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:32.402977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327a4497ea0547a7b3e1cf94f68bd6f5', 'request_id': 'req-9808249b-4d8b-4a56-ae5c-18da169e5911'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.430865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] The token cache is being invalidate because project 327a4497ea0547a7b3e1cf94f68bd6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:32.431120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.434241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.437623 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3029/6049] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:31 2022] DELETE /identity/v3/projects/327a4497ea0547a7b3e1cf94f68bd6f5 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:32.444506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:32.466655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ada13-965a-4c5b-9252-5fcd4087eaeb service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:32.590675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:32.591685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:32.592248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:32.593108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] PATH_INFO: `/v3/users/81e1255fe7c1439da39365e0d9f55fe1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:32.594919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e1255fe7c1439da39365e0d9f55fe1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:32.600752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:32.617222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:32.691010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e1255fe7c1439da39365e0d9f55fe1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.727133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e1255fe7c1439da39365e0d9f55fe1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.742090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e1255fe7c1439da39365e0d9f55fe1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.742724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e1255fe7c1439da39365e0d9f55fe1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.743375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e1255fe7c1439da39365e0d9f55fe1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:32.772822 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3021/6050] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:32 2022] DELETE /identity/v3/users/81e1255fe7c1439da39365e0d9f55fe1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:34.060290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:34.062172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:34.282626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808249b-4d8b-4a56-ae5c-18da169e5911 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:34.403787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86c678-d47b-4fef-83fd-2f617871c6fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:34.406197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86c678-d47b-4fef-83fd-2f617871c6fd service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:34.407578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86c678-d47b-4fef-83fd-2f617871c6fd service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:34.408639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86c678-d47b-4fef-83fd-2f617871c6fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:34.413982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af86c678-d47b-4fef-83fd-2f617871c6fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:34.436486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:34.438601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:34.450909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86c678-d47b-4fef-83fd-2f617871c6fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:34.452799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86c678-d47b-4fef-83fd-2f617871c6fd service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:34.490874 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3030/6051] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:34.616136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76268f5b-d57c-48ed-b583-dad80819488a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:34.652870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:34.654255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:34.655783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:34.656191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] PATH_INFO: `/v3/projects/5c9b436b63244be5a318d91932324b67` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:34.658262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c9b436b63244be5a318d91932324b67)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:34.663968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:34.772523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c9b436b63244be5a318d91932324b67', 'request_id': 'req-7afe14a1-1308-470f-8fec-1fbbfd1f2401'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:34.805247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c9b436b63244be5a318d91932324b67', 'request_id': 'req-7afe14a1-1308-470f-8fec-1fbbfd1f2401'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:34.816043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c9b436b63244be5a318d91932324b67', 'request_id': 'req-7afe14a1-1308-470f-8fec-1fbbfd1f2401'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:34.851438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] The token cache is being invalidate because project 5c9b436b63244be5a318d91932324b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:34.851803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:34.852413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7afe14a1-1308-470f-8fec-1fbbfd1f2401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:34.861957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084afc6-a557-4d5c-95d7-a98c7cb61d84 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:34.862424 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3022/6052] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:34 2022] DELETE /identity/v3/projects/5c9b436b63244be5a318d91932324b67 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:34.862580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084afc6-a557-4d5c-95d7-a98c7cb61d84 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:34.862687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084afc6-a557-4d5c-95d7-a98c7cb61d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:34.908348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4084afc6-a557-4d5c-95d7-a98c7cb61d84 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:34.909696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4084afc6-a557-4d5c-95d7-a98c7cb61d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:35.004243 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3031/6053] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:35.012430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6a4d-ea70-439f-a300-6a561cc4ed5b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.012767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6a4d-ea70-439f-a300-6a561cc4ed5b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.013053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6a4d-ea70-439f-a300-6a561cc4ed5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.050065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-542b6a4d-ea70-439f-a300-6a561cc4ed5b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:35.051318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-542b6a4d-ea70-439f-a300-6a561cc4ed5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:35.071169 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3023/6054] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:35.077003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084afc6-a557-4d5c-95d7-a98c7cb61d84 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.117627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084afc6-a557-4d5c-95d7-a98c7cb61d84 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.132215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.133131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.133913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.137726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.138591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:35.140555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.145132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.150395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:35.174332 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3032/6055] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:35.181985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b6a4d-ea70-439f-a300-6a561cc4ed5b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.195498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b6a4d-ea70-439f-a300-6a561cc4ed5b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.201396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.202156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.202438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.202711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.203941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.206010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.213640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:35.273125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3024/6056] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:35 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:35.280686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.295060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626336-84ce-4972-b657-4a2df75a13d6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.418321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.419094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.419863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.420343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.421121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.423755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.468342 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3033/6057] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:35 2022] POST /identity/v3/users => generated 303 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:35.469277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.491373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0ca08-ea8b-4af5-b09d-c4d686ec4f21 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.497088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.499321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.501148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.501657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.502589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.505011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.529356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.542173 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3025/6058] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:35.544395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd1879-0f3e-4830-90d4-ea6e4e3be33f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.551355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.552516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.552966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.553544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] PATH_INFO: `/v3/projects/23e96a6683074a48a277c8844cadea4c/users/314be7ada32c478a8a719352ac346fa5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.601799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e96a6683074a48a277c8844cadea4c, user_id=314be7ada32c478a8a719352ac346fa5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.614224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.647225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3034/6059] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:35 2022] PUT /identity/v3/projects/23e96a6683074a48a277c8844cadea4c/users/314be7ada32c478a8a719352ac346fa5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:35.651603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.662869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de199aa-482c-4931-a768-e122b6f62175 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.819324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.819982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.820289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.820563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.821526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.823239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.847815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.855997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3026/6060] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:35.857309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.861934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252eeb1-07b4-415f-a087-391cf272ad04 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.864069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:35.876273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a211-f54a-4773-a58e-7f185ee42f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.877307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a211-f54a-4773-a58e-7f185ee42f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.877622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a211-f54a-4773-a58e-7f185ee42f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.878129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a211-f54a-4773-a58e-7f185ee42f6e admin admin] PATH_INFO: `/v3/projects/23e96a6683074a48a277c8844cadea4c/users/314be7ada32c478a8a719352ac346fa5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.882434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a211-f54a-4773-a58e-7f185ee42f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e96a6683074a48a277c8844cadea4c, user_id=314be7ada32c478a8a719352ac346fa5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.886240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a211-f54a-4773-a58e-7f185ee42f6e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.917478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0263fc-b5e7-4cb0-8549-ac896b4038ab admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.918397 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3035/6061] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:35 2022] PUT /identity/v3/projects/23e96a6683074a48a277c8844cadea4c/users/314be7ada32c478a8a719352ac346fa5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:35.930104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ad314-6204-434b-be46-bd4d0c52ec95 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.930354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ad314-6204-434b-be46-bd4d0c52ec95 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.930666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ad314-6204-434b-be46-bd4d0c52ec95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.014495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5f0ad314-6204-434b-be46-bd4d0c52ec95 None None] MFA Rules not processed for user `314be7ada32c478a8a719352ac346fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:36.015730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5f0ad314-6204-434b-be46-bd4d0c52ec95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:36.045703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:36.046366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.046647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.046922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] PATH_INFO: `/v3/users/784233ee78764e7dbb774d3cb50165e3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.050956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784233ee78764e7dbb774d3cb50165e3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:36.052946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:36.170493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784233ee78764e7dbb774d3cb50165e3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.170727 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3036/6062] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:36.172755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ad314-6204-434b-be46-bd4d0c52ec95 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:36.187659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ad314-6204-434b-be46-bd4d0c52ec95 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:36.189326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784233ee78764e7dbb774d3cb50165e3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.192338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:36.192992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.193264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.193566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.194204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:36.196141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:36.202143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784233ee78764e7dbb774d3cb50165e3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.203257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784233ee78764e7dbb774d3cb50165e3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.204219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784233ee78764e7dbb774d3cb50165e3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.208477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:36.237552 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3027/6063] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:35 2022] DELETE /identity/v3/users/784233ee78764e7dbb774d3cb50165e3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:36.259724 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3037/6064] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:36 2022] POST /identity/v3/projects => generated 393 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:36.260966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:36.262476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:36.390397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a655c-8d92-40f3-9289-77e9a1211e97 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:36.428700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:36.430982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:36.448064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:36.564072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:36.565442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.566243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.566685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.568037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:36.568965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8878e-d65d-4574-9c5b-b14f9a01b76c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:36.571213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:36.573379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:36.574030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.574309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.574576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.576462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:36.617753 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3028/6065] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:36 2022] POST /identity/v3/users => generated 297 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:36.618940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:36.623753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:36.670376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c0ba3-97f0-45b6-a0df-5995a52f7119 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:36.684231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:36.684928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.685213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.685535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] PATH_INFO: `/v3/projects/0b6f25c3540546bf9bffd5610f4f86c8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.693668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6f25c3540546bf9bffd5610f4f86c8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:36.695863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:36.697827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:36.701941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:36.719148 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3038/6066] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:36.720493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:36.736552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdd013-7c99-4d68-99b8-d45e769fedaf service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:36.747498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:36.749975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.750320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.750566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.751465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:36.754554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:36.770264 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3039/6067] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:36.774194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:36.800571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6860c-8553-4314-988c-97d62d55a1a7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:36.826787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6f25c3540546bf9bffd5610f4f86c8', 'request_id': 'req-87f76bb0-a981-4f61-93c1-5d6551da95f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.851456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6f25c3540546bf9bffd5610f4f86c8', 'request_id': 'req-87f76bb0-a981-4f61-93c1-5d6551da95f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.865565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6f25c3540546bf9bffd5610f4f86c8', 'request_id': 'req-87f76bb0-a981-4f61-93c1-5d6551da95f7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.923243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] The token cache is being invalidate because project 0b6f25c3540546bf9bffd5610f4f86c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:36.926650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.928147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-87f76bb0-a981-4f61-93c1-5d6551da95f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:36.933780 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3029/6068] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:36 2022] DELETE /identity/v3/projects/0b6f25c3540546bf9bffd5610f4f86c8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:36.941717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23753-078c-430f-92db-5d5c1cdbd304 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.942169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23753-078c-430f-92db-5d5c1cdbd304 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.943032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23753-078c-430f-92db-5d5c1cdbd304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.990772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:36.992719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:36.993002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:36.993274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:36.993973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:36.995787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.013588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-01b23753-078c-430f-92db-5d5c1cdbd304 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:37.015604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-01b23753-078c-430f-92db-5d5c1cdbd304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.020160 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3040/6069] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:37.024841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.028292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.035484 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3030/6070] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:37.043019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e20859-cc20-449b-9d30-0d6dac3e3e30 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.043691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e20859-cc20-449b-9d30-0d6dac3e3e30 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.044117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e20859-cc20-449b-9d30-0d6dac3e3e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.060476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea3a3d-e351-4fc3-8eca-d72a50522ee2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.075673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dc612-8d56-494e-8a2a-3f49b1b751e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.076389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc612-8d56-494e-8a2a-3f49b1b751e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.076674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc612-8d56-494e-8a2a-3f49b1b751e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.076947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dc612-8d56-494e-8a2a-3f49b1b751e5 admin admin] PATH_INFO: `/v3/projects/eb5cce257b344428b21b831ce9c4136a/users/ad76233b943e40c8859b6c6917316709/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.077919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:37.108943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-95e20859-cc20-449b-9d30-0d6dac3e3e30 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:37.109361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc612-8d56-494e-8a2a-3f49b1b751e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5cce257b344428b21b831ce9c4136a, user_id=ad76233b943e40c8859b6c6917316709, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.110619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-95e20859-cc20-449b-9d30-0d6dac3e3e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.112114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dc612-8d56-494e-8a2a-3f49b1b751e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.139367 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3031/6071] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:37.140788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e20859-cc20-449b-9d30-0d6dac3e3e30 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.144371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3041/6072] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:37 2022] PUT /identity/v3/projects/eb5cce257b344428b21b831ce9c4136a/users/ad76233b943e40c8859b6c6917316709/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:37.154863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.155638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.156121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.167231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e20859-cc20-449b-9d30-0d6dac3e3e30 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.220928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] MFA Rules not processed for user `ad76233b943e40c8859b6c6917316709`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:37.224549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.250320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:37.320222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.320995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.321353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.321733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.322747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:37.322747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.326219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.353362 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3032/6073] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:37.354872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.395066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d7037-b12c-441f-a50c-7b6720e2e5a2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.403320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.404232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.404709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.405139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.405986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.417831 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3042/6074] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:37.427729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.432031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.434224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.498624 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3033/6075] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:37 2022] POST /identity/v3/projects => generated 361 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:37.502595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.662126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f263a2-70b9-4513-afd0-9dcf6adbfb26 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.694868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c7c0-ff4f-4512-94e7-f5331f61a349 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.717216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.717929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.718215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.719036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.719712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.723925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.798353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.798636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.798753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.799364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.800268 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3034/6076] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:37 2022] POST /identity/v3/users => generated 281 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:37.804532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.804817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.827285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d6f0c-f1d3-4aef-a47f-e840202e11ba admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.835534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.836534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.836984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.837722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.839137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.841664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.889489 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3035/6077] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:37.893552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.898971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.900464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.919432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b6e3e-3bfa-4736-bc14-cf5f22541e6e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.920307 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3043/6078] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:37.951703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adff71b-e7e2-4b40-9cd5-d50c6f9d778d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.953667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adff71b-e7e2-4b40-9cd5-d50c6f9d778d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.954071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adff71b-e7e2-4b40-9cd5-d50c6f9d778d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.955833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adff71b-e7e2-4b40-9cd5-d50c6f9d778d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.958343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adff71b-e7e2-4b40-9cd5-d50c6f9d778d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.960906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adff71b-e7e2-4b40-9cd5-d50c6f9d778d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:38.008499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:38.017731 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3036/6079] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:38.023504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f47d45-c893-4f84-940b-67d071fca6b8 service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:38.038529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:38.039550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:38.040141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:38.040641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] PATH_INFO: `/v3/projects/3021b865c7b2490d86f3d811b6b6c329/users/d55d866c764243269b2ab84a167440ca/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:38.061066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:38.064703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3021b865c7b2490d86f3d811b6b6c329, user_id=d55d866c764243269b2ab84a167440ca, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:38.069668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:38.092260 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3044/6080] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:38 2022] PUT /identity/v3/projects/3021b865c7b2490d86f3d811b6b6c329/users/d55d866c764243269b2ab84a167440ca/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:38.104751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ef834-80a7-4de2-bb32-f0504534fc55 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:38.105095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ef834-80a7-4de2-bb32-f0504534fc55 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:38.105308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ef834-80a7-4de2-bb32-f0504534fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:38.204912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1e9ef834-80a7-4de2-bb32-f0504534fc55 None None] MFA Rules not processed for user `d55d866c764243269b2ab84a167440ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:38.207168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1e9ef834-80a7-4de2-bb32-f0504534fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:38.257766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:38.341808 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3037/6081] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:38.396991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:38.522985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57cf43-f7f2-4e94-a877-bd0587ec153b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:38.533000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:38.534238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:38.535813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:38.537338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] PATH_INFO: `/v3/users/d55d866c764243269b2ab84a167440ca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:38.539030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55d866c764243269b2ab84a167440ca)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:38.549038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:38.680078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55d866c764243269b2ab84a167440ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:38.681946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55d866c764243269b2ab84a167440ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:38.707211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55d866c764243269b2ab84a167440ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:38.708235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55d866c764243269b2ab84a167440ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:38.750335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55d866c764243269b2ab84a167440ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:38.789387 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3045/6082] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:38 2022] DELETE /identity/v3/users/d55d866c764243269b2ab84a167440ca => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:38.794032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:38.794850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:38.795315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:38.840182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:38.842095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:38.999800 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3038/6083] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:39.043437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:39.045965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:39.180179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882785a-8c0d-41e9-87fd-5bd8548c502a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:39.197660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:39.198378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:39.200221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:39.200499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:39.228832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:39.231394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:39.259661 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3046/6084] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:39.276518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:39.278057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:39.490979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b4159-ad42-48ff-9b6e-79f5f46714b4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:39.500966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd93143-1571-41ed-b5c2-8421e42fa1bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:39.501814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93143-1571-41ed-b5c2-8421e42fa1bb service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:39.502131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93143-1571-41ed-b5c2-8421e42fa1bb service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:39.502407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93143-1571-41ed-b5c2-8421e42fa1bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:39.504459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2dd93143-1571-41ed-b5c2-8421e42fa1bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:39.545232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd93143-1571-41ed-b5c2-8421e42fa1bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:39.553642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd93143-1571-41ed-b5c2-8421e42fa1bb service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:39.595868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:39.597621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:39.604685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3039/6085] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:39.632467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d91b-313d-4661-9091-967dc4aa7c81 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:39.647273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:39.648028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:39.648205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:39.648475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] PATH_INFO: `/v3/users/314be7ada32c478a8a719352ac346fa5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:39.650628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=314be7ada32c478a8a719352ac346fa5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:39.652812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:39.665311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:39.803436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '314be7ada32c478a8a719352ac346fa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:39.806389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '314be7ada32c478a8a719352ac346fa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:39.826239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '314be7ada32c478a8a719352ac346fa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:39.827101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '314be7ada32c478a8a719352ac346fa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:39.862536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '314be7ada32c478a8a719352ac346fa5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:39.909972 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3047/6086] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:39 2022] DELETE /identity/v3/users/314be7ada32c478a8a719352ac346fa5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:39.915755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:39.916220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:39.916497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:39.956778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:39.960745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:40.100209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:40.125527 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3040/6087] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:40.166982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:40.168771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:40.326662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94015-6387-4630-9eda-061cb2db2095 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:40.336737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1471f51-a1d5-4b5e-96c4-1c154594ab9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:40.337451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1471f51-a1d5-4b5e-96c4-1c154594ab9f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:40.337762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1471f51-a1d5-4b5e-96c4-1c154594ab9f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:40.338013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1471f51-a1d5-4b5e-96c4-1c154594ab9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:40.361020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1471f51-a1d5-4b5e-96c4-1c154594ab9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:40.362618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1471f51-a1d5-4b5e-96c4-1c154594ab9f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:40.412501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3048/6088] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:40.703846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:40.705623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:40.728479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e984eb-62f7-4640-ab15-57ee6d0d6774 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:40.738968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:40.739695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:40.740000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:40.740305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] PATH_INFO: `/v3/users/101d81caee8d4569a15a7e0f42f5ddde` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:40.744218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=101d81caee8d4569a15a7e0f42f5ddde)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:40.746637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:40.877522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101d81caee8d4569a15a7e0f42f5ddde'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:40.897195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101d81caee8d4569a15a7e0f42f5ddde'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:40.910264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101d81caee8d4569a15a7e0f42f5ddde'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:40.911203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101d81caee8d4569a15a7e0f42f5ddde'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:40.911909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101d81caee8d4569a15a7e0f42f5ddde'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:40.933227 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3041/6089] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:40 2022] DELETE /identity/v3/users/101d81caee8d4569a15a7e0f42f5ddde => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:40.937599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:40.938594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:40.939190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:40.990793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:40.995068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:41.190450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3049/6090] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:41.239585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:41.241756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:41.345826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac29d7-dadd-4f2f-9a45-4ba4c321b1fc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:41.353114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:41.353954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:41.354565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:41.354929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:41.371790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:41.371790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:41.396176 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3042/6091] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:41.834147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:41.839026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:41.855247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6440c9-5c08-4460-b74e-9625ca33db26 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:41.860891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:41.861686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:41.862003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:41.862341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] PATH_INFO: `/v3/projects/3021b865c7b2490d86f3d811b6b6c329` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:41.863579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3021b865c7b2490d86f3d811b6b6c329)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:41.865675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:41.877749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:41.879568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:41.900315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e7c1-feea-454b-8773-b611bb2a6a3a service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:41.907637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:41.908278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:41.908549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:41.908872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] PATH_INFO: `/v3/projects/23e96a6683074a48a277c8844cadea4c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:41.910475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e96a6683074a48a277c8844cadea4c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:41.912518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:41.921230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:42.017333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3021b865c7b2490d86f3d811b6b6c329', 'request_id': 'req-281bb152-b6f0-4003-8c94-b4d23da5d8f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.022042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e96a6683074a48a277c8844cadea4c', 'request_id': 'req-7467a128-9b8e-4c16-b234-aac5c3f8b413'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.028516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3021b865c7b2490d86f3d811b6b6c329', 'request_id': 'req-281bb152-b6f0-4003-8c94-b4d23da5d8f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.041617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e96a6683074a48a277c8844cadea4c', 'request_id': 'req-7467a128-9b8e-4c16-b234-aac5c3f8b413'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.044602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3021b865c7b2490d86f3d811b6b6c329', 'request_id': 'req-281bb152-b6f0-4003-8c94-b4d23da5d8f9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.077421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e96a6683074a48a277c8844cadea4c', 'request_id': 'req-7467a128-9b8e-4c16-b234-aac5c3f8b413'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.086849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:42.089037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] The token cache is being invalidate because project 3021b865c7b2490d86f3d811b6b6c329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:42.089333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.090219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.097568 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3050/6092] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:41 2022] DELETE /identity/v3/projects/3021b865c7b2490d86f3d811b6b6c329 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:42.098834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:42.101004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:42.112897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] The token cache is being invalidate because project 23e96a6683074a48a277c8844cadea4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:42.113210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.114328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7467a128-9b8e-4c16-b234-aac5c3f8b413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.116221 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3043/6093] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:41 2022] DELETE /identity/v3/projects/23e96a6683074a48a277c8844cadea4c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:42.117754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56f600-4138-4edd-9955-a76593750b9b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:42.118237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56f600-4138-4edd-9955-a76593750b9b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:42.118685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56f600-4138-4edd-9955-a76593750b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:42.160921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5e56f600-4138-4edd-9955-a76593750b9b None None] MFA Rules not processed for user `caa2c2f4848e497f80d4a439c785c2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:42.162447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e56f600-4138-4edd-9955-a76593750b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:42.213786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bb152-b6f0-4003-8c94-b4d23da5d8f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:42.313426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:42.314256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:42.314851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:42.315375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] PATH_INFO: `/v3/projects/7eea0d5995384120ab4f65114bdb9c65` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:42.315517 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3044/6094] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:42.316821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:42.317103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:42.317381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:42.320519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eea0d5995384120ab4f65114bdb9c65)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:42.323078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:42.364452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:42.365836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:42.409731 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3045/6095] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:42.409731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:42.411344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:42.414451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eea0d5995384120ab4f65114bdb9c65', 'request_id': 'req-475b65d1-929b-479a-9e47-0a26e48f2e01'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.422236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eea0d5995384120ab4f65114bdb9c65', 'request_id': 'req-475b65d1-929b-479a-9e47-0a26e48f2e01'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.431782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dffbe5-1e53-44ad-8b86-4713578948b5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:42.434694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eea0d5995384120ab4f65114bdb9c65', 'request_id': 'req-475b65d1-929b-479a-9e47-0a26e48f2e01'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.436375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:42.453405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] The token cache is being invalidate because project 7eea0d5995384120ab4f65114bdb9c65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:42.453838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.454604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-475b65d1-929b-479a-9e47-0a26e48f2e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.457976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3051/6096] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:42 2022] DELETE /identity/v3/projects/7eea0d5995384120ab4f65114bdb9c65 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:42.463260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:42.463709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:42.463988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:42.492482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:42.494783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:42.562148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:42.563168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:42.563650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:42.564136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] PATH_INFO: `/v3/users/ad76233b943e40c8859b6c6917316709` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:42.566584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad76233b943e40c8859b6c6917316709)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:42.568876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:42.685188 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3052/6097] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:42.689886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:42.694257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:42.701757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad76233b943e40c8859b6c6917316709'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.728113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad76233b943e40c8859b6c6917316709'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.730194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368cdd2-5383-4355-8981-0afbade64027 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:42.744202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:42.745121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:42.745423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:42.745727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] PATH_INFO: `/v3/users/7144ae1f396c470d9ca04f6f2d650a8d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:42.748739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad76233b943e40c8859b6c6917316709'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.749326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7144ae1f396c470d9ca04f6f2d650a8d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:42.749511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad76233b943e40c8859b6c6917316709'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.750289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad76233b943e40c8859b6c6917316709'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.754458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:42.780761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3046/6098] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:42 2022] DELETE /identity/v3/users/ad76233b943e40c8859b6c6917316709 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:42.781670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:42.783280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:42.793284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:42.875155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7144ae1f396c470d9ca04f6f2d650a8d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.876275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7144ae1f396c470d9ca04f6f2d650a8d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.904311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7144ae1f396c470d9ca04f6f2d650a8d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.905163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7144ae1f396c470d9ca04f6f2d650a8d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.920274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7144ae1f396c470d9ca04f6f2d650a8d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:42.942573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3053/6099] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:42 2022] DELETE /identity/v3/users/7144ae1f396c470d9ca04f6f2d650a8d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:42.952450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d32c95-7ffc-478c-ada6-9359ce428b52 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.044397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.045254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.045567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.045859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.046525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:43.046826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.049206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.069182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:43.070858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:43.081144 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3047/6100] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:43.088653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e528bb-780e-4d3e-bb1f-948da2f4e57f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.094208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.094927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.095253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.095526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.096170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.098977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.120559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:43.124187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:43.180777 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3054/6101] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:43 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:43.184920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:43.270166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcccff-02ea-42c0-8a6c-3783dd7880b1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.323849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:43.371750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.375466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc000f-c6f8-4593-80ac-5c4fdb25d085 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.376455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.376733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.376992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.382973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:43.387509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.389718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.390201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.390671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.392059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.396927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.409925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.411542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.428475 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3048/6102] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:43.430305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:43.432590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:43.443285 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3055/6103] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:43 2022] POST /identity/v3/users => generated 295 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:43.447196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:43.463256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:43.468275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c65760-4f58-4523-9e6d-b19de2c363a6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.472174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377d888-d3d1-4aa1-8fb4-3efe41f670c7 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.478385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.481324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.481763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.482899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.484400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.484906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.486718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.487705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.488088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.488460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] PATH_INFO: `/v3/projects/cb4545257180419cb934b6c06af8dd30` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.491059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4545257180419cb934b6c06af8dd30)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.507583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.526863 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3056/6104] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:43.532339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:43.553250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e742d-6c93-4b35-8d06-c48559b562b1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.570420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.571330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.571693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.571993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.572830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.576248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.600473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3057/6105] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:43.604027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:43.623237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4545257180419cb934b6c06af8dd30', 'request_id': 'req-528b5b53-9a0d-47ad-8607-3f53243e36f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:43.629656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae5f02-ff93-4844-9c9b-417686607369 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:43.640015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4545257180419cb934b6c06af8dd30', 'request_id': 'req-528b5b53-9a0d-47ad-8607-3f53243e36f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:43.652267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4545257180419cb934b6c06af8dd30', 'request_id': 'req-528b5b53-9a0d-47ad-8607-3f53243e36f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:43.699013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] The token cache is being invalidate because project cb4545257180419cb934b6c06af8dd30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:43.699455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:43.717369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-528b5b53-9a0d-47ad-8607-3f53243e36f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:43.723136 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3049/6106] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:43 2022] DELETE /identity/v3/projects/cb4545257180419cb934b6c06af8dd30 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:43.730343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ece1c-7d71-4945-a28a-48c3e888b8ea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.734159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ece1c-7d71-4945-a28a-48c3e888b8ea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.734551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ece1c-7d71-4945-a28a-48c3e888b8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.778356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a64ece1c-7d71-4945-a28a-48c3e888b8ea None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:43.785190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a64ece1c-7d71-4945-a28a-48c3e888b8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:43.840124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e16aa3-f541-4e26-a74c-271b24b307e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:43.841786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e16aa3-f541-4e26-a74c-271b24b307e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.842164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e16aa3-f541-4e26-a74c-271b24b307e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.842639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e16aa3-f541-4e26-a74c-271b24b307e5 admin admin] PATH_INFO: `/v3/projects/720881c6f7304aadbf069698c97dc5c5/users/8ddb1d0a7bca457493d7ac9a58a01e64/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.881937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e16aa3-f541-4e26-a74c-271b24b307e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=720881c6f7304aadbf069698c97dc5c5, user_id=8ddb1d0a7bca457493d7ac9a58a01e64, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:43.889612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e16aa3-f541-4e26-a74c-271b24b307e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:43.913809 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3050/6107] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:43.923673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae634-06a2-4fc7-aae5-bd2c2ee760e5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.923982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae634-06a2-4fc7-aae5-bd2c2ee760e5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.924231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae634-06a2-4fc7-aae5-bd2c2ee760e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.929500 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3058/6108] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:43 2022] PUT /identity/v3/projects/720881c6f7304aadbf069698c97dc5c5/users/8ddb1d0a7bca457493d7ac9a58a01e64/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:43.934128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762e066-fb27-4275-8d2b-1c636566d90b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:43.934128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762e066-fb27-4275-8d2b-1c636566d90b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:43.934128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762e066-fb27-4275-8d2b-1c636566d90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:43.972283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-747ae634-06a2-4fc7-aae5-bd2c2ee760e5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:43.974936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9762e066-fb27-4275-8d2b-1c636566d90b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:43.976263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9762e066-fb27-4275-8d2b-1c636566d90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:43.978446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-747ae634-06a2-4fc7-aae5-bd2c2ee760e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:44.085585 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3051/6109] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:44.087596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae634-06a2-4fc7-aae5-bd2c2ee760e5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.098406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae634-06a2-4fc7-aae5-bd2c2ee760e5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.105092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.106050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.106417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.106900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.107838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:44.108523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.110307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.130505 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3059/6110] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:44.134119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762e066-fb27-4275-8d2b-1c636566d90b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.138528 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3052/6111] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:44.144127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.146711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:44.159054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762e066-fb27-4275-8d2b-1c636566d90b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.176431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.177062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.177491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.177885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.178751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.181953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:44.183585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.255273 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3060/6112] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:44 2022] POST /identity/v3/projects => generated 391 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:44.258548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.260011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd7150-ea6d-46c6-a8fa-a54c2fcfc141 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.294452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:44.341608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.343151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.343600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.344100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.472010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489fa46-c926-413c-b2c4-f1d0a64f5801 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.486321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.486567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.486955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.487223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.487495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.488136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.488279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.493122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.512454 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3053/6113] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:44.513365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.516013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:44.568979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3061/6114] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:44 2022] POST /identity/v3/users => generated 296 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:44.571241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe38162-a17a-4608-a6a0-705b3e1a8719 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.579659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.580571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.580957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.581341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] PATH_INFO: `/v3/projects/eb5cce257b344428b21b831ce9c4136a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.581499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.596003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2541e53-1ff8-4a8b-9f50-b575071f3336 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.598532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5cce257b344428b21b831ce9c4136a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.601963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.602262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.602890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.603166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.603518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.604253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.606098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.620821 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3062/6115] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:44.626654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.640305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dbe0c-6f56-4325-8d97-9d968d8f5384 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.646107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.647185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.647541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.647904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.648754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.651337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.671351 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3063/6116] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:44.675665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:44.704178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70661b-9516-4e56-b206-73b8d74ec8f0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:44.713360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5cce257b344428b21b831ce9c4136a', 'request_id': 'req-d6911c8c-8c35-48e8-80d1-4476f53b37c0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:44.730738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5cce257b344428b21b831ce9c4136a', 'request_id': 'req-d6911c8c-8c35-48e8-80d1-4476f53b37c0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:44.752068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5cce257b344428b21b831ce9c4136a', 'request_id': 'req-d6911c8c-8c35-48e8-80d1-4476f53b37c0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:44.787719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] The token cache is being invalidate because project eb5cce257b344428b21b831ce9c4136a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:44.787998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:44.797778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6911c8c-8c35-48e8-80d1-4476f53b37c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:44.801947 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3054/6117] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:44 2022] DELETE /identity/v3/projects/eb5cce257b344428b21b831ce9c4136a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:44.806743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04de94-c798-4b55-a9ab-40c72c231d7b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.808872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04de94-c798-4b55-a9ab-40c72c231d7b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.809149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04de94-c798-4b55-a9ab-40c72c231d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.834009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:44.875114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fc04de94-c798-4b55-a9ab-40c72c231d7b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:44.883873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc04de94-c798-4b55-a9ab-40c72c231d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:44.890660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb7ce-d231-45cb-aaa0-39f7b58355cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:44.891369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb7ce-d231-45cb-aaa0-39f7b58355cb admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.891657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb7ce-d231-45cb-aaa0-39f7b58355cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.891958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb7ce-d231-45cb-aaa0-39f7b58355cb admin admin] PATH_INFO: `/v3/projects/f71b91269e9d4db39bc785cc7dde4ff0/users/ecc4023bff1c4de8ab0d41883d431b19/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.920072 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3055/6118] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:44.921794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8eea-df21-4581-aa33-8523b4631bb4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:44.922083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8eea-df21-4581-aa33-8523b4631bb4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:44.922355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8eea-df21-4581-aa33-8523b4631bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:44.950048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb7ce-d231-45cb-aaa0-39f7b58355cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71b91269e9d4db39bc785cc7dde4ff0, user_id=ecc4023bff1c4de8ab0d41883d431b19, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:44.952844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb7ce-d231-45cb-aaa0-39f7b58355cb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:44.991762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c98a8eea-df21-4581-aa33-8523b4631bb4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:44.993028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c98a8eea-df21-4581-aa33-8523b4631bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:44.999488 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3064/6119] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:44 2022] PUT /identity/v3/projects/f71b91269e9d4db39bc785cc7dde4ff0/users/ecc4023bff1c4de8ab0d41883d431b19/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:45.009812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3baa06-c872-4db5-9d96-112cf61d7037 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:45.010556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3baa06-c872-4db5-9d96-112cf61d7037 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:45.010556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3baa06-c872-4db5-9d96-112cf61d7037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:45.098514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5b3baa06-c872-4db5-9d96-112cf61d7037 None None] MFA Rules not processed for user `ecc4023bff1c4de8ab0d41883d431b19`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:45.100067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5b3baa06-c872-4db5-9d96-112cf61d7037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:45.181909 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3056/6120] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:45.185273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8eea-df21-4581-aa33-8523b4631bb4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:45.206432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8eea-df21-4581-aa33-8523b4631bb4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:45.218318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:45.221693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:45.222163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:45.235336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:45.237377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:45.238118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:45.239165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:45.242332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:45.288903 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3065/6121] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:45.292553 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3057/6122] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:45.298709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3baa06-c872-4db5-9d96-112cf61d7037 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:45.314289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3baa06-c872-4db5-9d96-112cf61d7037 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:45.329586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:45.330372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:45.331703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:45.332704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:45.335219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:45.336751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:45.338424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:45.349313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:45.420456 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3066/6123] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:45 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:45.425205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:45.482360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf8f69-8e65-4aa3-9562-82bc7c5d2fe0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:45.621649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad4ed7-7e67-4a8d-a059-364bfcce6351 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:45.665351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:45.666007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:45.666280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:45.666545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:45.667222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:45.669268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:45.716803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:45.717449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:45.717762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:45.722008 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3067/6124] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:45 2022] POST /identity/v3/users => generated 299 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:45.722977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:45.726315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:45.746965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608a390-8bbd-49e2-bb99-1a4fb1951b3a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:45.759574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:45.760134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:45.760484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:45.760786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:45.761560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:45.763169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:45.787490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3068/6125] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:45.788409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:45.820249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bded53-3cf6-41b6-8c74-ec2f30e54b3e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:45.826622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:45.827304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:45.827576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:45.827848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] PATH_INFO: `/v3/projects/c703f88d8dc7445988c318d0be25cd7b/users/e192cf2c5aad4a7aa9fb76105494cf4c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:45.861336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c703f88d8dc7445988c318d0be25cd7b, user_id=e192cf2c5aad4a7aa9fb76105494cf4c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:45.864209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:45.894847 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3069/6126] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:45 2022] PUT /identity/v3/projects/c703f88d8dc7445988c318d0be25cd7b/users/e192cf2c5aad4a7aa9fb76105494cf4c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:45.896870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:45.933910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:45.941238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:45.943177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:45.992212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3058/6127] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:46.091487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ec1a8-017e-4f10-b8a9-cdbafe93c3b3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:46.098365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a31fc-a27f-450e-a791-23043579fd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:46.099326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a31fc-a27f-450e-a791-23043579fd3d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.100317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a31fc-a27f-450e-a791-23043579fd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.100695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a31fc-a27f-450e-a791-23043579fd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.101578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a31fc-a27f-450e-a791-23043579fd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:46.103410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a31fc-a27f-450e-a791-23043579fd3d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:46.116319 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3070/6128] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:46.120566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:46.133058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b1f53-8ccf-4006-b33c-91b6985a43b0 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:46.138382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:46.139080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.139358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.139631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] PATH_INFO: `/v3/projects/c703f88d8dc7445988c318d0be25cd7b/users/e192cf2c5aad4a7aa9fb76105494cf4c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.141937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c703f88d8dc7445988c318d0be25cd7b, user_id=e192cf2c5aad4a7aa9fb76105494cf4c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:46.144695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:46.168632 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3059/6129] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:46 2022] PUT /identity/v3/projects/c703f88d8dc7445988c318d0be25cd7b/users/e192cf2c5aad4a7aa9fb76105494cf4c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:46.170812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78a023-ffe7-495e-b3ef-c38229cc4a98 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.171383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78a023-ffe7-495e-b3ef-c38229cc4a98 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.173710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78a023-ffe7-495e-b3ef-c38229cc4a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.211766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4a78a023-ffe7-495e-b3ef-c38229cc4a98 None None] MFA Rules not processed for user `e192cf2c5aad4a7aa9fb76105494cf4c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:46.213741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a78a023-ffe7-495e-b3ef-c38229cc4a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:46.221026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:46.329496 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3071/6130] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:46.333854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:46.437682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70075bcf-869d-41cd-b13f-7bfbbbbc6d88 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:46.443956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:46.444662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.444941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.445215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.445963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:46.448154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:46.495554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3060/6131] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:46 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:46.500295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78a023-ffe7-495e-b3ef-c38229cc4a98 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:46.512761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78a023-ffe7-495e-b3ef-c38229cc4a98 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:46.604705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:46.605835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.606473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.606955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.608112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:46.610371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:46.643055 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3072/6132] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:46 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:46.647727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:46.662373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70c428-1980-4a16-851a-d9aafd08a966 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:46.683695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:46.684636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.685485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.686152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.687038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:46.698840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:46.712929 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3061/6133] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:46.716773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:46.731933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8f4c-f59f-43a6-9560-1b228198c99d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:46.737780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11fd09-a6ff-4050-b0f7-52954b7a7c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:46.738726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11fd09-a6ff-4050-b0f7-52954b7a7c19 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.739435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11fd09-a6ff-4050-b0f7-52954b7a7c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.739723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11fd09-a6ff-4050-b0f7-52954b7a7c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.740468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11fd09-a6ff-4050-b0f7-52954b7a7c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:46.742392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11fd09-a6ff-4050-b0f7-52954b7a7c19 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:46.777296 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3073/6134] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:46.778566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:46.789398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd5b43-1b40-4ddd-a572-ca882fbebbd8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:46.805652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:46.806359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.806635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.807249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] PATH_INFO: `/v3/projects/bd12d96ba64644d595330f1712f5d7c4/users/81edb457a59a4b5d85f9bac0931d2ded/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.844848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd12d96ba64644d595330f1712f5d7c4, user_id=81edb457a59a4b5d85f9bac0931d2ded, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:46.847701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:46.873969 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3062/6135] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:46 2022] PUT /identity/v3/projects/bd12d96ba64644d595330f1712f5d7c4/users/81edb457a59a4b5d85f9bac0931d2ded/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:46.882062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:46.882608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:46.883039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:46.943069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] MFA Rules not processed for user `81edb457a59a4b5d85f9bac0931d2ded`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:46.944275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:47.022160 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3074/6136] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:47.037649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:47.039369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:47.121369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa753af0-407d-4c02-869c-ae6f72694613 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:47.130391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da119c12-e4c7-483e-a026-ec00c0bcbf87 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:47.131747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da119c12-e4c7-483e-a026-ec00c0bcbf87 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:47.131747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da119c12-e4c7-483e-a026-ec00c0bcbf87 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:47.131747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da119c12-e4c7-483e-a026-ec00c0bcbf87 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:47.233621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:47.242312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:47.246602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:47.272279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da119c12-e4c7-483e-a026-ec00c0bcbf87 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:47.273864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da119c12-e4c7-483e-a026-ec00c0bcbf87 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:47.285826 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3063/6137] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:47.285985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:47.372241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbda59-5a88-4cd3-ab1f-844cbb7dfe90 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:47.377714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:47.378492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:47.378964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:47.379348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] PATH_INFO: `/v3/users/1d37dcc7008f4edd98e5c040fc605e61` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:47.380728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d37dcc7008f4edd98e5c040fc605e61)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:47.382816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:47.490032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d37dcc7008f4edd98e5c040fc605e61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:47.491144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d37dcc7008f4edd98e5c040fc605e61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:47.509336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d37dcc7008f4edd98e5c040fc605e61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:47.512709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d37dcc7008f4edd98e5c040fc605e61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:47.558096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d37dcc7008f4edd98e5c040fc605e61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:47.575780 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3075/6138] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:47 2022] DELETE /identity/v3/users/1d37dcc7008f4edd98e5c040fc605e61 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:47.580660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:47.581247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:47.581247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:47.628142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:47.629367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:47.844098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3064/6139] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:47.923953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:47.925586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:48.110351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957980d-a20f-4ae7-b988-571f2bbcc869 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:48.119862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ca25e-1ed5-438c-8331-62e95f8ab52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:48.121702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca25e-1ed5-438c-8331-62e95f8ab52d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:48.122176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca25e-1ed5-438c-8331-62e95f8ab52d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:48.122755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca25e-1ed5-438c-8331-62e95f8ab52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:48.136880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca25e-1ed5-438c-8331-62e95f8ab52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:48.139023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca25e-1ed5-438c-8331-62e95f8ab52d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:48.153252 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3076/6140] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:48.481323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:48.484487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:48.511938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25ee06-d329-494f-aa46-1f355256cc01 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:48.520414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:48.521448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:48.521826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:48.522143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] PATH_INFO: `/v3/users/e192cf2c5aad4a7aa9fb76105494cf4c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:48.524025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e192cf2c5aad4a7aa9fb76105494cf4c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:48.526434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:48.601136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e192cf2c5aad4a7aa9fb76105494cf4c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:48.625492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e192cf2c5aad4a7aa9fb76105494cf4c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:48.640516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e192cf2c5aad4a7aa9fb76105494cf4c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:48.647410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e192cf2c5aad4a7aa9fb76105494cf4c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:48.648219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e192cf2c5aad4a7aa9fb76105494cf4c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:48.662544 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3065/6141] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:48 2022] DELETE /identity/v3/users/e192cf2c5aad4a7aa9fb76105494cf4c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:48.667217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:48.667745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:48.667745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:48.696084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:48.698815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:48.738423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:48.805528 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3077/6142] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:48.905737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:48.907704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:48.918298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:48.920844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:48.944873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c641d41-01c0-41cf-a64f-3bd738fcd43f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:48.955269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:48.957040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:48.958616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:48.959061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] PATH_INFO: `/v3/projects/282a01b9b14640cb9918486126076716` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:48.963242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282a01b9b14640cb9918486126076716)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:48.971475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:49.038170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6eca9-963a-4205-9958-200418a63bcb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:49.050258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:49.051015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:49.051297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:49.052086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:49.079329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:49.082281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:49.095167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282a01b9b14640cb9918486126076716', 'request_id': 'req-8a20e668-6e8a-420d-92e4-451faf92f496'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.095493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:49.113697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282a01b9b14640cb9918486126076716', 'request_id': 'req-8a20e668-6e8a-420d-92e4-451faf92f496'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.137854 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3066/6143] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:49.139059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282a01b9b14640cb9918486126076716', 'request_id': 'req-8a20e668-6e8a-420d-92e4-451faf92f496'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.158559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] The token cache is being invalidate because project 282a01b9b14640cb9918486126076716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:49.162310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.164606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.175645 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3078/6144] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:48 2022] DELETE /identity/v3/projects/282a01b9b14640cb9918486126076716 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:49.179581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:49.195264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bcac1-6270-4a36-9300-6479a37cb247 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:49.361627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:49.364073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:49.367591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:49.368439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] PATH_INFO: `/v3/users/a2183ed8377243509a9c5fde0a4b32fe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:49.371411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2183ed8377243509a9c5fde0a4b32fe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:49.379636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:49.505175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2183ed8377243509a9c5fde0a4b32fe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.570491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2183ed8377243509a9c5fde0a4b32fe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.596935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2183ed8377243509a9c5fde0a4b32fe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.597699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2183ed8377243509a9c5fde0a4b32fe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.598521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2183ed8377243509a9c5fde0a4b32fe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:49.687830 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3067/6145] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:49 2022] DELETE /identity/v3/users/a2183ed8377243509a9c5fde0a4b32fe => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:50.164806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:50.166795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:50.260155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:50.261907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:50.373731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20e668-6e8a-420d-92e4-451faf92f496 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:50.454368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff261-4a99-459d-bec0-c12962f85021 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:50.461391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:50.462471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:50.462756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:50.463027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] PATH_INFO: `/v3/projects/c703f88d8dc7445988c318d0be25cd7b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:50.464642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c703f88d8dc7445988c318d0be25cd7b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:50.466845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:50.502289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:50.522513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:50.536137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:50.536867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:50.537207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:50.537578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:50.540772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:50.589547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c703f88d8dc7445988c318d0be25cd7b', 'request_id': 'req-7dab482a-12bd-43dd-bad1-64457cbc4191'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:50.622666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c703f88d8dc7445988c318d0be25cd7b', 'request_id': 'req-7dab482a-12bd-43dd-bad1-64457cbc4191'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:50.638627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c703f88d8dc7445988c318d0be25cd7b', 'request_id': 'req-7dab482a-12bd-43dd-bad1-64457cbc4191'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:50.659847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] The token cache is being invalidate because project c703f88d8dc7445988c318d0be25cd7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:50.661638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:50.663076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:50.666811 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3068/6146] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:50 2022] DELETE /identity/v3/projects/c703f88d8dc7445988c318d0be25cd7b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:50.671580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:50.674260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:50.696818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:50.698857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:50.789317 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3079/6147] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:50.849298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab482a-12bd-43dd-bad1-64457cbc4191 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:50.866665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:50.867333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:50.867685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:50.867954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] PATH_INFO: `/v3/users/81edb457a59a4b5d85f9bac0931d2ded` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:50.874107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81edb457a59a4b5d85f9bac0931d2ded)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:50.878782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:50.992592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81edb457a59a4b5d85f9bac0931d2ded'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.012555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81edb457a59a4b5d85f9bac0931d2ded'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.018349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:51.020261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:51.029261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81edb457a59a4b5d85f9bac0931d2ded'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.030035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81edb457a59a4b5d85f9bac0931d2ded'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.030615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19a4baf-5bb7-498e-966f-4bbc1696e526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81edb457a59a4b5d85f9bac0931d2ded'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.041049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72465fb-8e0d-4366-9d7e-5401c3fa44f3 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:51.048027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:51.049290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:51.049795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:51.050427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] PATH_INFO: `/v3/projects/903ce733beb7418aa13ad358335cec2d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:51.051829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903ce733beb7418aa13ad358335cec2d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:51.054079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:51.069312 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3069/6148] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:50 2022] DELETE /identity/v3/users/81edb457a59a4b5d85f9bac0931d2ded => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:51.151959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903ce733beb7418aa13ad358335cec2d', 'request_id': 'req-59732c22-fd64-4821-ad2c-25b724609ec8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.159800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903ce733beb7418aa13ad358335cec2d', 'request_id': 'req-59732c22-fd64-4821-ad2c-25b724609ec8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.182545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903ce733beb7418aa13ad358335cec2d', 'request_id': 'req-59732c22-fd64-4821-ad2c-25b724609ec8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.201691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] The token cache is being invalidate because project 903ce733beb7418aa13ad358335cec2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:51.202128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.202799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-59732c22-fd64-4821-ad2c-25b724609ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:51.209176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3080/6149] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:51 2022] DELETE /identity/v3/projects/903ce733beb7418aa13ad358335cec2d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:51.210928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a16496-8454-42af-92f2-59b6d3018b17 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:51.211314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a16496-8454-42af-92f2-59b6d3018b17 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:51.211582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a16496-8454-42af-92f2-59b6d3018b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:51.242239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a5a16496-8454-42af-92f2-59b6d3018b17 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:51.243668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a5a16496-8454-42af-92f2-59b6d3018b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:51.288445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:51.372737 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3070/6150] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:51.373437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352edde-30b3-4e1d-b474-ce5872d5ca32 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:51.373775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352edde-30b3-4e1d-b474-ce5872d5ca32 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:51.374050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352edde-30b3-4e1d-b474-ce5872d5ca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:51.405369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5352edde-30b3-4e1d-b474-ce5872d5ca32 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:51.407218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5352edde-30b3-4e1d-b474-ce5872d5ca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:51.421904 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3081/6151] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:51.427522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a16496-8454-42af-92f2-59b6d3018b17 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:51.447852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a16496-8454-42af-92f2-59b6d3018b17 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:51.455421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:51.456656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:51.457391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:51.457870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:51.459422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:51.459880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:51.472113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:51.496932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3071/6152] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:51.502119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352edde-30b3-4e1d-b474-ce5872d5ca32 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:51.535846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352edde-30b3-4e1d-b474-ce5872d5ca32 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:51.547795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:51.548600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:51.548917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:51.549187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:51.550411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:51.556526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:51.671576 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3082/6153] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:51 2022] POST /identity/v3/projects => generated 355 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:51.675621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:51.719476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b925a37-2a75-46ba-87b0-a2eb6595b287 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:51.848048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80084320-f219-419a-851b-65660080fe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:51.849140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084320-f219-419a-851b-65660080fe7e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:51.849140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084320-f219-419a-851b-65660080fe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:51.849558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084320-f219-419a-851b-65660080fe7e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:51.861984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084320-f219-419a-851b-65660080fe7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:51.864669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084320-f219-419a-851b-65660080fe7e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:51.943746 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3072/6154] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:51 2022] POST /identity/v3/users => generated 278 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:51.947992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:51.979637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda4d2-f647-4ff8-af08-e9e426ed20ab admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.008800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.009736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.012838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.013232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.015782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.021129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.046870 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3083/6155] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:52.048611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80084320-f219-419a-851b-65660080fe7e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.063634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80084320-f219-419a-851b-65660080fe7e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.083403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.084307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.084679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.085091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.085896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.088789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.098664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.100544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:52.126920 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3073/6156] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:13:52.129702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1803fb-23f7-4bb7-87c8-0b5c61624c57 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.131541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.150534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937ce51-d969-478d-9611-5c61b2910ab5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.151859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.153073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.154201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.155358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] PATH_INFO: `/v3/projects/bd12d96ba64644d595330f1712f5d7c4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.161596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd12d96ba64644d595330f1712f5d7c4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.164547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.165322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ebcd7-f3e3-43c4-bbee-e06cf7967d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.166227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ebcd7-f3e3-43c4-bbee-e06cf7967d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.169113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ebcd7-f3e3-43c4-bbee-e06cf7967d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.169604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ebcd7-f3e3-43c4-bbee-e06cf7967d60 admin admin] PATH_INFO: `/v3/projects/593ad6965f254b1e9f27922fedcba4bb/users/d73b40017f3f4ff9a19f5e6c838dc2a6/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.226443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ebcd7-f3e3-43c4-bbee-e06cf7967d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=593ad6965f254b1e9f27922fedcba4bb, user_id=d73b40017f3f4ff9a19f5e6c838dc2a6, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.234347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ebcd7-f3e3-43c4-bbee-e06cf7967d60 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.258003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:52.288579 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3074/6157] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:52 2022] PUT /identity/v3/projects/593ad6965f254b1e9f27922fedcba4bb/users/d73b40017f3f4ff9a19f5e6c838dc2a6/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:52.307856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.310119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.311189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.315550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd12d96ba64644d595330f1712f5d7c4', 'request_id': 'req-b092665f-3d52-4f6f-b226-bfacf8588ea5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:52.327913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd12d96ba64644d595330f1712f5d7c4', 'request_id': 'req-b092665f-3d52-4f6f-b226-bfacf8588ea5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:52.340673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd12d96ba64644d595330f1712f5d7c4', 'request_id': 'req-b092665f-3d52-4f6f-b226-bfacf8588ea5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:52.356494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] The token cache is being invalidate because project bd12d96ba64644d595330f1712f5d7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:52.357430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:52.359409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b092665f-3d52-4f6f-b226-bfacf8588ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:52.365489 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3084/6158] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:52 2022] DELETE /identity/v3/projects/bd12d96ba64644d595330f1712f5d7c4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:52.366600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73087c-74e0-4902-bc3e-6ed9e233c203 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.366887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73087c-74e0-4902-bc3e-6ed9e233c203 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.367158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73087c-74e0-4902-bc3e-6ed9e233c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.392240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] MFA Rules not processed for user `d73b40017f3f4ff9a19f5e6c838dc2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:52.393777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:52.395463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ea73087c-74e0-4902-bc3e-6ed9e233c203 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:52.397887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ea73087c-74e0-4902-bc3e-6ed9e233c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:52.486270 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3085/6159] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:52.488343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc481b3-f887-4068-aa22-157a5613d1eb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.488737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc481b3-f887-4068-aa22-157a5613d1eb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.489087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc481b3-f887-4068-aa22-157a5613d1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.541324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1dc481b3-f887-4068-aa22-157a5613d1eb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:52.543989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1dc481b3-f887-4068-aa22-157a5613d1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:52.553999 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3086/6160] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:52.557926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc481b3-f887-4068-aa22-157a5613d1eb None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.560648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:52.571922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc481b3-f887-4068-aa22-157a5613d1eb None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.578769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.579142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.579542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.579976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.580746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:52.581142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.583154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.601040 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3075/6161] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:52.612668 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3087/6162] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:13:52.613985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.615561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:52.616951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.639353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9754e-2bea-40af-83c3-3358e30dc77e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.648234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cce978-3f5b-4f9d-84d6-f9cea9e165d3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.648862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.651250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.651974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.651974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.652748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.657233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.677984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.678619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.678890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.679160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.679802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.681788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.722614 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3088/6163] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:52 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:52.725173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.757919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3076/6164] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:52 2022] POST /identity/v3/projects => generated 357 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:13:52.762598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:52.931775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9011-d247-49e1-85b9-952d885eaa70 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.943961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.944555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.944769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.945042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.947379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1cc01-ffa3-4e5a-ab3e-d3c37b1be97d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:52.949856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.951864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:52.967306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:52.968066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:52.968396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:52.968860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:52.969666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:52.973745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.019651 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3077/6165] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:52 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:53.023967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.025205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:53.042637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3089/6166] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:13:52 2022] POST /identity/v3/users => generated 286 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:53.043939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.057350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d110ae-65e5-482a-b721-25273236c00f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.063045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.063824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.064269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.064824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.068435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.070874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.084823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca96ff-5083-4603-8ecd-c61a727aa9de admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.086161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3078/6167] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:53.090490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.100865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.101549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.102333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.102333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.102758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.104453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.123604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736c84e-3c06-439b-9503-f54407225adb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.132137 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3090/6168] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:53.138996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.140590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.142047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.142415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.142750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] PATH_INFO: `/v3/projects/361627f60e3d4489ab33eca5c020c4e4/users/8898ebc5fa2f44d6b343d29989605c0c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.182022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160037e-6b46-4fad-8bb3-40fdc91b1189 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.202925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.203531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.203811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.204074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.204838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.208720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.214633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361627f60e3d4489ab33eca5c020c4e4, user_id=8898ebc5fa2f44d6b343d29989605c0c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.226790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.256699 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3091/6169] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:53.259554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.269793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3079/6170] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:53 2022] PUT /identity/v3/projects/361627f60e3d4489ab33eca5c020c4e4/users/8898ebc5fa2f44d6b343d29989605c0c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:13:53.274609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.289008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b363aa0-23df-4194-a77d-17a3fdfa2c60 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.369399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f00115-4591-4d2a-9c41-a360c7ccfc10 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.377163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.377992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.378279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.378645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.379297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.381156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.401501 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3080/6171] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:13:53.403803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.418376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100199bd-fa21-4c1c-aeca-f7349afd881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.419066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100199bd-fa21-4c1c-aeca-f7349afd881c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.419340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100199bd-fa21-4c1c-aeca-f7349afd881c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.419612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100199bd-fa21-4c1c-aeca-f7349afd881c admin admin] PATH_INFO: `/v3/projects/ad306136dc654559a74a42717663c547/users/62b8f1ec47d04fd4a0ae80c363d41e9d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.446643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100199bd-fa21-4c1c-aeca-f7349afd881c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad306136dc654559a74a42717663c547, user_id=62b8f1ec47d04fd4a0ae80c363d41e9d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.447496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611a1a7-86be-4007-ad3b-7ef8b53a1d0b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.450659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100199bd-fa21-4c1c-aeca-f7349afd881c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.452554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.453340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.453642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.453919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] PATH_INFO: `/v3/projects/361627f60e3d4489ab33eca5c020c4e4/users/8898ebc5fa2f44d6b343d29989605c0c/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.456741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361627f60e3d4489ab33eca5c020c4e4, user_id=8898ebc5fa2f44d6b343d29989605c0c, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.459521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.475460 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3092/6173] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:53 2022] PUT /identity/v3/projects/ad306136dc654559a74a42717663c547/users/62b8f1ec47d04fd4a0ae80c363d41e9d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:53.486043 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3081/6173] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:13:53 2022] PUT /identity/v3/projects/361627f60e3d4489ab33eca5c020c4e4/users/8898ebc5fa2f44d6b343d29989605c0c/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:53.501986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e90cd-b255-4fc5-928d-5593164aa3ff None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.502744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e90cd-b255-4fc5-928d-5593164aa3ff None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.503330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e90cd-b255-4fc5-928d-5593164aa3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.538731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.542101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:53.587629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4f4e90cd-b255-4fc5-928d-5593164aa3ff None None] MFA Rules not processed for user `62b8f1ec47d04fd4a0ae80c363d41e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:53.589288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:53.598590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4f4e90cd-b255-4fc5-928d-5593164aa3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:53.708744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d552e76-2684-47d1-aff1-93b17dcbe289 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.719880 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3093/6174] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:13:53.719880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260be9c6-4d63-439e-b288-9cdd69b91c4d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.719880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260be9c6-4d63-439e-b288-9cdd69b91c4d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.719880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260be9c6-4d63-439e-b288-9cdd69b91c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.737020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.738172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.738653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.739118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.741872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:53.763656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-260be9c6-4d63-439e-b288-9cdd69b91c4d None None] MFA Rules not processed for user `8898ebc5fa2f44d6b343d29989605c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:53.764889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-260be9c6-4d63-439e-b288-9cdd69b91c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:53.785409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:53.788140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:53.814715 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3082/6175] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:13:53.816403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:53.818237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:53.856018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:53.976888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1281971-40b7-476a-95c4-4e0d79a2268c service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:53.987239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:53.987888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:53.988165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:53.988435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:53.989414 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3094/6176] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:53.995556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260be9c6-4d63-439e-b288-9cdd69b91c4d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:54.015477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:54.019236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:54.028194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260be9c6-4d63-439e-b288-9cdd69b91c4d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:54.041618 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3083/6177] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:54.045516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8898ebc5fa2f44d6b343d29989605c0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '361627f60e3d4489ab33eca5c020c4e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:54.046065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:54.046343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:54.046692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:54.047392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:54.047694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:54.055628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:54.096332 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3095/6178] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:54.097542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:54.124896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802254c-f0ac-4c09-b4f5-8ee297d16fb3 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:54.142924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8898ebc5fa2f44d6b343d29989605c0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '361627f60e3d4489ab33eca5c020c4e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:54.144215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:54.144518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:54.144805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:54.145499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:54.147516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:54.211341 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3084/6179] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:13:54 2022] POST /identity/v3/projects => generated 348 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:13:54.215418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:54.232280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78bb9f-8548-4506-8274-a8245b3de1b6 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:54.355368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8898ebc5fa2f44d6b343d29989605c0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '361627f60e3d4489ab33eca5c020c4e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:54.355997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:54.356286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:54.356562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:54.357227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:13:54.357588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:54.361900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:54.395978 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3096/6180] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:13:54.440312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:54.593662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019688-4940-4ff8-b56e-bbcd62afdd98 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:54.600888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:54.602829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:54.605940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:54.613517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:54.631074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:54.632760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:54.646319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3085/6181] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:55.882606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:55.894720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956aee6c-8b82-4710-ba13-ac14c0e24bba tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:55.900161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8898ebc5fa2f44d6b343d29989605c0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '361627f60e3d4489ab33eca5c020c4e4', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:55.901006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:55.901399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:55.901817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] PATH_INFO: `/v3/projects/82294ed909e34100af99fe18c5dbbe72` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:55.903037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorizing `identity:delete_project(project_id=82294ed909e34100af99fe18c5dbbe72)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:55.905127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:55.987087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82294ed909e34100af99fe18c5dbbe72', 'request_id': 'req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:55.997648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82294ed909e34100af99fe18c5dbbe72', 'request_id': 'req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:56.014425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82294ed909e34100af99fe18c5dbbe72', 'request_id': 'req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:56.035066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] The token cache is being invalidate because project 82294ed909e34100af99fe18c5dbbe72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:13:56.035366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:56.036441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f83b171e-bea3-4eaf-bacb-6b95da6e79e9 tempest-QuotasTest-2080743630 tempest-QuotasTest-2080743630-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:56.038307 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3097/6182] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:13:55 2022] DELETE /identity/v3/projects/82294ed909e34100af99fe18c5dbbe72 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:57.442115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:57.506890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:57.603868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07297132-bd7e-43b6-88e2-f0bc38e8c92c service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:57.711834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:57.712975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:57.714036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:57.714608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] PATH_INFO: `/v3/users/d73b40017f3f4ff9a19f5e6c838dc2a6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:57.717293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73b40017f3f4ff9a19f5e6c838dc2a6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:57.721054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:57.813529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73b40017f3f4ff9a19f5e6c838dc2a6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:57.856063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73b40017f3f4ff9a19f5e6c838dc2a6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:57.877079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73b40017f3f4ff9a19f5e6c838dc2a6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:57.880305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73b40017f3f4ff9a19f5e6c838dc2a6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:57.881146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73b40017f3f4ff9a19f5e6c838dc2a6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:57.922297 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3086/6183] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:57 2022] DELETE /identity/v3/users/d73b40017f3f4ff9a19f5e6c838dc2a6 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:57.926983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:57.927490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:57.927602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:57.974795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:57.976079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:58.095734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:58.141856 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3098/6184] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:58.152001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:58.153591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:58.278473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:58.302127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f4cf6-afb0-4055-8b6d-ac33e6ec38e1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:58.344980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f272bac-01ee-43a7-8a1a-ea84e891c766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:58.345659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272bac-01ee-43a7-8a1a-ea84e891c766 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:58.345938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272bac-01ee-43a7-8a1a-ea84e891c766 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:58.346211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272bac-01ee-43a7-8a1a-ea84e891c766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:58.371550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272bac-01ee-43a7-8a1a-ea84e891c766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:58.373803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272bac-01ee-43a7-8a1a-ea84e891c766 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:58.408479 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3087/6185] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:58.646474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:58.647964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:58.664561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de559ea-c5a3-4aab-9936-c1b4d9be895e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:58.680973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:58.682160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:58.682597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:58.683023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] PATH_INFO: `/v3/users/5ae3425f3c704ece85cf9c6b54949fac` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:58.684973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae3425f3c704ece85cf9c6b54949fac)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:58.687344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:58.765661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae3425f3c704ece85cf9c6b54949fac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:58.766699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae3425f3c704ece85cf9c6b54949fac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:58.783614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae3425f3c704ece85cf9c6b54949fac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:58.787450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae3425f3c704ece85cf9c6b54949fac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:58.803112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae3425f3c704ece85cf9c6b54949fac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:13:58.828297 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3099/6186] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:13:58 2022] DELETE /identity/v3/users/5ae3425f3c704ece85cf9c6b54949fac => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:58.833252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:58.833800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:58.833800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:58.877194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:58.878464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:59.065980 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3088/6187] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:59.171362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:59.173225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:59.286515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7b4ad-fe96-40e8-8614-11fc0d6cac52 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:59.293552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:59.294279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:59.294715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:59.294989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:59.353234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:59.359693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:59.388309 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3100/6188] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:00.105668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:00.107749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:00.135847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dc59d-a112-4d1d-a1b8-1ee92175b087 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:00.150358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:00.151621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:00.152263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:00.152739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] PATH_INFO: `/v3/projects/593ad6965f254b1e9f27922fedcba4bb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:00.156909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] RBAC: Authorizing `identity:delete_project(project_id=593ad6965f254b1e9f27922fedcba4bb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:00.159902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:00.240328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '593ad6965f254b1e9f27922fedcba4bb', 'request_id': 'req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.254162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '593ad6965f254b1e9f27922fedcba4bb', 'request_id': 'req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.262510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '593ad6965f254b1e9f27922fedcba4bb', 'request_id': 'req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.288279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] The token cache is being invalidate because project 593ad6965f254b1e9f27922fedcba4bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:00.288998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.290097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.293475 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3089/6189] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:00 2022] DELETE /identity/v3/projects/593ad6965f254b1e9f27922fedcba4bb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:00.298190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:00.311293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1973d24-1767-46c0-9e8d-1546aec5daae service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:00.428344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:00.440316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:00.440990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:00.441536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:00.442138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] PATH_INFO: `/v3/users/8898ebc5fa2f44d6b343d29989605c0c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:00.444431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8898ebc5fa2f44d6b343d29989605c0c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:00.446657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:00.549150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8898ebc5fa2f44d6b343d29989605c0c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.550589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8898ebc5fa2f44d6b343d29989605c0c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.572829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8898ebc5fa2f44d6b343d29989605c0c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.573776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8898ebc5fa2f44d6b343d29989605c0c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.602583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8898ebc5fa2f44d6b343d29989605c0c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:00.630539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3101/6190] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:00 2022] DELETE /identity/v3/users/8898ebc5fa2f44d6b343d29989605c0c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:02.358537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:02.361242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:02.389200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ca8cb-0c5f-4e68-a5d5-a0de5af7d38e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:02.390294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:02.391708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:02.522971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:02.523628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:02.523910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:02.524182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] PATH_INFO: `/v3/projects/361627f60e3d4489ab33eca5c020c4e4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:02.531233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=361627f60e3d4489ab33eca5c020c4e4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:02.533263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:02.578349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7b8ce-fe58-4bed-a48e-94a99613dd13 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:02.584517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:02.591240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:02.591835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:02.592294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:02.592813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] PATH_INFO: `/v3/projects/203653c6a42345ee8fe68f0a0bc64a7d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:02.597229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] RBAC: Authorizing `identity:delete_project(project_id=203653c6a42345ee8fe68f0a0bc64a7d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:02.599622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:02.665052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '361627f60e3d4489ab33eca5c020c4e4', 'request_id': 'req-e94504e2-3b68-4f4a-8648-a03b55ccd245'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.698218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '361627f60e3d4489ab33eca5c020c4e4', 'request_id': 'req-e94504e2-3b68-4f4a-8648-a03b55ccd245'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.706697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '361627f60e3d4489ab33eca5c020c4e4', 'request_id': 'req-e94504e2-3b68-4f4a-8648-a03b55ccd245'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.726525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203653c6a42345ee8fe68f0a0bc64a7d', 'request_id': 'req-c1a496ad-242a-4176-a385-5c73ddbf153e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.733248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] The token cache is being invalidate because project 361627f60e3d4489ab33eca5c020c4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:02.733571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.734297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e94504e2-3b68-4f4a-8648-a03b55ccd245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.736405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203653c6a42345ee8fe68f0a0bc64a7d', 'request_id': 'req-c1a496ad-242a-4176-a385-5c73ddbf153e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.736539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3090/6191] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:02 2022] DELETE /identity/v3/projects/361627f60e3d4489ab33eca5c020c4e4 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:02.742885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07b1cc-0db2-464a-9a3d-0877480e7bf0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:02.743367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07b1cc-0db2-464a-9a3d-0877480e7bf0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:02.743796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07b1cc-0db2-464a-9a3d-0877480e7bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:02.761967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203653c6a42345ee8fe68f0a0bc64a7d', 'request_id': 'req-c1a496ad-242a-4176-a385-5c73ddbf153e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.777937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] The token cache is being invalidate because project 203653c6a42345ee8fe68f0a0bc64a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:02.778616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.780513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:02.782981 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3102/6192] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:02 2022] DELETE /identity/v3/projects/203653c6a42345ee8fe68f0a0bc64a7d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:02.783645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-df07b1cc-0db2-464a-9a3d-0877480e7bf0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:02.784881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-df07b1cc-0db2-464a-9a3d-0877480e7bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:02.786751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:02.793883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:02.800833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:02.927535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a496ad-242a-4176-a385-5c73ddbf153e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:02.943123 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3091/6193] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:02.946314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:02.946989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:02.947258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:02.947384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4cfbf-baa2-4dc7-a615-6846ab56699b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:02.947573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] PATH_INFO: `/v3/users/991fa7d515ac4460ad5d95dc7b5e04af` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:02.947731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4cfbf-baa2-4dc7-a615-6846ab56699b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:02.947975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4cfbf-baa2-4dc7-a615-6846ab56699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:02.949289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991fa7d515ac4460ad5d95dc7b5e04af)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:02.951415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:02.988326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-02e4cfbf-baa2-4dc7-a615-6846ab56699b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:02.989764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02e4cfbf-baa2-4dc7-a615-6846ab56699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:03.013842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3092/6194] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:03.023027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4cfbf-baa2-4dc7-a615-6846ab56699b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.045018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4cfbf-baa2-4dc7-a615-6846ab56699b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.054249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991fa7d515ac4460ad5d95dc7b5e04af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.054716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.054925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991fa7d515ac4460ad5d95dc7b5e04af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.055533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.056170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.058848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.061959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:03.062429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.064520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.074455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991fa7d515ac4460ad5d95dc7b5e04af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.076145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991fa7d515ac4460ad5d95dc7b5e04af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.101490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991fa7d515ac4460ad5d95dc7b5e04af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.102121 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3093/6195] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:03.104204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.108157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:03.124470 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3103/6196] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:02 2022] DELETE /identity/v3/users/991fa7d515ac4460ad5d95dc7b5e04af => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:03.136646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9142-f60e-4eae-ba3a-e902bb5878fc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.194913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.196372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:03.246904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.248910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.249746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.250307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.251396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.254139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.304068 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3094/6197] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:03 2022] POST /identity/v3/projects => generated 391 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:03.308981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.331873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:03.357239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff70c7f-5f14-4693-b8cf-a2f450d3c2f1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.413070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:03.500821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510ad6b-5d84-457d-abf3-063e20b609de admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.505274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.509653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.510721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.512598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.518749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.520127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.520127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.520127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.520860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.523582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.525959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:03.596387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.602712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.627485 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3095/6198] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:03 2022] POST /identity/v3/users => generated 296 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:03.627485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.630719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:03.654819 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3104/6199] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:03.656120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.691900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e3ff0-e5e7-45e6-a3ec-56fb613d3adc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.711321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.715077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.715386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118b103-268a-45b8-964b-ff11d318e526 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.715527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.715527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] PATH_INFO: `/v3/users/ecc4023bff1c4de8ab0d41883d431b19` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.719548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc4023bff1c4de8ab0d41883d431b19)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.728598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.729583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.730489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.731336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.731801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.733200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.736278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.757200 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3105/6200] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:03.761610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.784354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53532b-3da0-4cf3-9258-01b95ee07892 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.820805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.821779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.822415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.823620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.825065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.827890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.852654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc4023bff1c4de8ab0d41883d431b19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.856396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3106/6201] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:03.862272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:03.881243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc4023bff1c4de8ab0d41883d431b19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.887695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48702e2-4ff8-48f1-8f05-928fbe925d03 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:03.900223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bd9ee-1ef6-4afe-9b21-0c4bfe1ced6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:03.900968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bd9ee-1ef6-4afe-9b21-0c4bfe1ced6a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.901255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bd9ee-1ef6-4afe-9b21-0c4bfe1ced6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.901562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bd9ee-1ef6-4afe-9b21-0c4bfe1ced6a admin admin] PATH_INFO: `/v3/projects/6cba6418a4dd49638b346b6e5c367f38/users/bfafdfdca1384c34ab470c22a1181a18/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.904569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc4023bff1c4de8ab0d41883d431b19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.905241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc4023bff1c4de8ab0d41883d431b19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.905933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-da2df24a-f1c6-4728-bd1a-8b97041277ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc4023bff1c4de8ab0d41883d431b19'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:03.926192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bd9ee-1ef6-4afe-9b21-0c4bfe1ced6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cba6418a4dd49638b346b6e5c367f38, user_id=bfafdfdca1384c34ab470c22a1181a18, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:03.931754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bd9ee-1ef6-4afe-9b21-0c4bfe1ced6a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:03.938876 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3096/6202] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:03 2022] DELETE /identity/v3/users/ecc4023bff1c4de8ab0d41883d431b19 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:03.940525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.940819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.941093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:03.973265 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3107/6203] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:03 2022] PUT /identity/v3/projects/6cba6418a4dd49638b346b6e5c367f38/users/bfafdfdca1384c34ab470c22a1181a18/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:03.984785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:03.985078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:03.985344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:04.020233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:04.021514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:04.075322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] MFA Rules not processed for user `bfafdfdca1384c34ab470c22a1181a18`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:04.078760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:04.208657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3097/6204] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:04.208880 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3108/6205] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:04.213015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:04.215202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:04.233304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:04.234873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba60bc-da03-4ea9-bd7a-6b4ec3c7f9e4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:04.235032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:04.240015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:04.240855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:04.241142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:04.241416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:04.242961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:04.245099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:04.260909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:04.295563 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3098/6206] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:04 2022] POST /identity/v3/projects => generated 391 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:04.296645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:04.423300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8ed7c-ae5f-4244-929e-ef10a7876c8f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:04.460484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:04.536233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953b907-b9aa-46a2-a4d6-92b236ada882 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:04.552015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:04.554111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:04.555914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:04.556325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:04.557782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:04.561902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:04.605440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:04.609401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:04.611090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:04.611466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:04.620176 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3099/6207] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:04 2022] POST /identity/v3/users => generated 302 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:04.622306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:04.630533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:04.632647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:04.653238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddba3fe-fc0b-4bd8-a336-8c90ea12657d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:04.662226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:04.662915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:04.663201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:04.663476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:04.664139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:04.669485 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3109/6208] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:14:04.677151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:04.710420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:04.715720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3100/6209] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:04.729482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7b52c-133e-4bce-b3c2-78d5fe352a0f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:04.744186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:04.744965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:04.745249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:04.745549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] PATH_INFO: `/v3/projects/1a9a6279e2db4fcc981becdf3bc940c1/users/ccaee99abc364180ad38f44298b736e5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:04.771104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9a6279e2db4fcc981becdf3bc940c1, user_id=ccaee99abc364180ad38f44298b736e5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:04.778055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:04.815321 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3110/6210] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:04 2022] PUT /identity/v3/projects/1a9a6279e2db4fcc981becdf3bc940c1/users/ccaee99abc364180ad38f44298b736e5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:04.817169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:04.832638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cf346-71d7-4b28-83ed-25e09c0889a6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:04.983825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4118c38-36f6-4d5a-a902-6e3b735eab07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:04.984967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4118c38-36f6-4d5a-a902-6e3b735eab07 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:04.985413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4118c38-36f6-4d5a-a902-6e3b735eab07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:04.986269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4118c38-36f6-4d5a-a902-6e3b735eab07 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:04.987285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4118c38-36f6-4d5a-a902-6e3b735eab07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:05.000092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4118c38-36f6-4d5a-a902-6e3b735eab07 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:05.024598 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3101/6211] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:05.027109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:05.040693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e5b8-1c0b-4dbd-b141-2c561559f3e5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:05.055924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:05.056654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:05.057002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:05.057793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] PATH_INFO: `/v3/projects/1a9a6279e2db4fcc981becdf3bc940c1/users/ccaee99abc364180ad38f44298b736e5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:05.073894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9a6279e2db4fcc981becdf3bc940c1, user_id=ccaee99abc364180ad38f44298b736e5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:05.082164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:05.111429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3111/6212] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:05 2022] PUT /identity/v3/projects/1a9a6279e2db4fcc981becdf3bc940c1/users/ccaee99abc364180ad38f44298b736e5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:05.112135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:05.112379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:05.112637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:05.177571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] MFA Rules not processed for user `ccaee99abc364180ad38f44298b736e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:05.178915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:05.290886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:05.337880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3102/6213] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:05.357385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:05.359404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:05.454712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3164a-cd0a-4c01-a298-0f8bdc3e3c6a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:05.476253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde6ca3-cbc6-49e6-a0df-090e72fab4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:05.477194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6ca3-cbc6-49e6-a0df-090e72fab4db service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:05.477873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6ca3-cbc6-49e6-a0df-090e72fab4db service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:05.478419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6ca3-cbc6-49e6-a0df-090e72fab4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:05.506407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde6ca3-cbc6-49e6-a0df-090e72fab4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:05.513612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde6ca3-cbc6-49e6-a0df-090e72fab4db service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:05.520934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:05.522686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:05.548979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3112/6214] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:05.691709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d24ce-d568-4858-92c6-aaa1feff2147 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:05.705383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:05.706089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:05.706425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:05.706697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] PATH_INFO: `/v3/projects/4937c938251c4205b713c49ccd707775` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:05.707920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4937c938251c4205b713c49ccd707775)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:05.712526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:05.787764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4937c938251c4205b713c49ccd707775', 'request_id': 'req-65bd84dc-0096-49d5-a85d-c050f73c5a75'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:05.811632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4937c938251c4205b713c49ccd707775', 'request_id': 'req-65bd84dc-0096-49d5-a85d-c050f73c5a75'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:05.830788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4937c938251c4205b713c49ccd707775', 'request_id': 'req-65bd84dc-0096-49d5-a85d-c050f73c5a75'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:05.861995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] The token cache is being invalidate because project 4937c938251c4205b713c49ccd707775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:05.866290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:05.868089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-65bd84dc-0096-49d5-a85d-c050f73c5a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:05.887167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3103/6215] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:05 2022] DELETE /identity/v3/projects/4937c938251c4205b713c49ccd707775 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:05.891836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c784a0-8a0a-4090-8c80-829603760352 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:05.891836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c784a0-8a0a-4090-8c80-829603760352 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:05.892516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c784a0-8a0a-4090-8c80-829603760352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:05.950490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-02c784a0-8a0a-4090-8c80-829603760352 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:05.953957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-02c784a0-8a0a-4090-8c80-829603760352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:06.092985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3113/6216] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:06.102007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e9143-3063-47ac-b82b-f16d5c76aaae None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:06.102297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e9143-3063-47ac-b82b-f16d5c76aaae None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:06.102883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e9143-3063-47ac-b82b-f16d5c76aaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:06.167214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:06.167495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:06.167770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:06.174219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-655e9143-3063-47ac-b82b-f16d5c76aaae None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:06.176982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-655e9143-3063-47ac-b82b-f16d5c76aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:06.193599 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3104/6217] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:06.197178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e9143-3063-47ac-b82b-f16d5c76aaae None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:06.228038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e9143-3063-47ac-b82b-f16d5c76aaae None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:06.232876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] MFA Rules not processed for user `8ddb1d0a7bca457493d7ac9a58a01e64`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:06.234276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:06.252634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:06.253259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:06.253557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:06.253830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:06.254574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:06.254787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:06.256424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:06.274383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:06.292125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3105/6218] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:06.303355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:06.332137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbcb0f-d83e-4f58-9406-5ce0f19fb7eb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:06.361375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:06.362090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:06.363412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:06.363687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:06.364410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:06.366627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:06.463925 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3114/6220] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:06.464480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:06.473885 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3106/6220] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:06 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:06.477797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:06.479219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:06.527027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d797130-7568-4f84-a9e6-939808161a4e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:06.627681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:06.681659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:06.683875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c073f6-8f98-4266-96b9-2cca697bd2b1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:06.684134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:06.684586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:06.684882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] PATH_INFO: `/v3/projects/f71b91269e9d4db39bc785cc7dde4ff0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:06.686629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71b91269e9d4db39bc785cc7dde4ff0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:06.693833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:06.708995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:06.709631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:06.709777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:06.709928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:06.710560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:06.712512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:06.785394 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3107/6221] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:06 2022] POST /identity/v3/users => generated 291 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:06.786008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:06.789081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:06.860770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71b91269e9d4db39bc785cc7dde4ff0', 'request_id': 'req-f896d61d-4614-462d-bca6-f137aa39aa53'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:06.880082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71b91269e9d4db39bc785cc7dde4ff0', 'request_id': 'req-f896d61d-4614-462d-bca6-f137aa39aa53'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:06.895738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71b91269e9d4db39bc785cc7dde4ff0', 'request_id': 'req-f896d61d-4614-462d-bca6-f137aa39aa53'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:06.935738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] The token cache is being invalidate because project f71b91269e9d4db39bc785cc7dde4ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:06.936421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:06.937895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:06.942966 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3115/6222] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:06 2022] DELETE /identity/v3/projects/f71b91269e9d4db39bc785cc7dde4ff0 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:06.946673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:06.946673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:06.991009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcdcc9-8b0b-45d3-bb42-c2aacece42ce admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.028039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:07.058293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896d61d-4614-462d-bca6-f137aa39aa53 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.079990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b9333-85ee-44b4-a794-fda824f42260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.080938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b9333-85ee-44b4-a794-fda824f42260 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.081391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b9333-85ee-44b4-a794-fda824f42260 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.082599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b9333-85ee-44b4-a794-fda824f42260 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.096959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b9333-85ee-44b4-a794-fda824f42260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.098692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b9333-85ee-44b4-a794-fda824f42260 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.116173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3116/6223] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:07.119641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e461-1f09-4351-bcec-e63b5525ddeb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.119926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e461-1f09-4351-bcec-e63b5525ddeb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.120498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e461-1f09-4351-bcec-e63b5525ddeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.146944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.147596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.147877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.148151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.154639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:07.159626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2a34e461-1f09-4351-bcec-e63b5525ddeb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:07.164075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a34e461-1f09-4351-bcec-e63b5525ddeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:07.186293 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3117/6224] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:07.188653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34e461-1f09-4351-bcec-e63b5525ddeb None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:07.213029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34e461-1f09-4351-bcec-e63b5525ddeb None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.234813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545db411-3460-487e-97c2-1154049a421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.236516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db411-3460-487e-97c2-1154049a421e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.236801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db411-3460-487e-97c2-1154049a421e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.237071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db411-3460-487e-97c2-1154049a421e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.237769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545db411-3460-487e-97c2-1154049a421e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.242231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545db411-3460-487e-97c2-1154049a421e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.271274 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3118/6225] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:07.272614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2afa-a47e-447a-b109-7709cc038984 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.277124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2afa-a47e-447a-b109-7709cc038984 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.277500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd2afa-a47e-447a-b109-7709cc038984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.309062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.310826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.321383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9bdd2afa-a47e-447a-b109-7709cc038984 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:07.322775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9bdd2afa-a47e-447a-b109-7709cc038984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:07.337625 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3108/6226] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:07.341110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:07.349184 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3119/6227] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:07.352597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2afa-a47e-447a-b109-7709cc038984 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:07.358545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:07.373219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3e73-912b-4dab-bcf1-2896044c6441 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.380775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd2afa-a47e-447a-b109-7709cc038984 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.390205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bea65c-7b98-4577-be84-bcb6a2d62f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.391992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bea65c-7b98-4577-be84-bcb6a2d62f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.392286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bea65c-7b98-4577-be84-bcb6a2d62f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.392566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bea65c-7b98-4577-be84-bcb6a2d62f35 admin admin] PATH_INFO: `/v3/projects/00b20946e00b4a559b37039b37d959e8/users/637e8d1af8384b5fa06414d502739b09/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.396455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.400044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.400330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.400643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.404299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:07.404573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.410765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.438583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bea65c-7b98-4577-be84-bcb6a2d62f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b20946e00b4a559b37039b37d959e8, user_id=637e8d1af8384b5fa06414d502739b09, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.445876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bea65c-7b98-4577-be84-bcb6a2d62f35 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.448847 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3120/6228] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:07.454781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:07.493499 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3109/6229] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:07 2022] PUT /identity/v3/projects/00b20946e00b4a559b37039b37d959e8/users/637e8d1af8384b5fa06414d502739b09/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:07.493783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2ea44-8589-4a14-9b8e-f0b0b1686e27 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.601004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b2973-35c4-4fe7-a4c3-af02598f4522 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.602108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b2973-35c4-4fe7-a4c3-af02598f4522 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.602449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b2973-35c4-4fe7-a4c3-af02598f4522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.708997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.709743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.711462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.711758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.713266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.715672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d73b2973-35c4-4fe7-a4c3-af02598f4522 None None] MFA Rules not processed for user `637e8d1af8384b5fa06414d502739b09`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:07.718437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.720405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d73b2973-35c4-4fe7-a4c3-af02598f4522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:07.858625 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3121/6230] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:07 2022] POST /identity/v3/projects => generated 359 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:07.865264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.018721 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3110/6231] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:08.023344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b2973-35c4-4fe7-a4c3-af02598f4522 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.053299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b2973-35c4-4fe7-a4c3-af02598f4522 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.062235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.063009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.063387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.063758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.064494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.066582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.068258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dfab0-c28a-4895-9b9d-f0b9a4ada00e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.076868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.077808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.078970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.079573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.080411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.091037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.148545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3111/6232] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:08 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:14:08.150104 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3122/6233] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:07 2022] POST /identity/v3/users => generated 280 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:08.154622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.157608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.186369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b832c-f6f6-450b-b196-f3395fe14974 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.223373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:08.311652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec6def-03b8-4780-a3d6-757fc9bf4daf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.322319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.323074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.323910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.324548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.325655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.325935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.326619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.327410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.327799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.328485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.333425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.335443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.357189 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3112/6234] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:08.364631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.390291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3123/6235] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:08 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:08.393177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.395142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228a53d-3886-4b0f-b053-b46ab4c1fbd4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.407699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.408694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.408980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.409253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.413352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.419400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0843f93-eb34-4088-ab7a-f7a3c133fa59 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.420104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.433421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.434109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.434388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.434660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.436367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.438080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.446869 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3113/6236] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:08.451607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.483993 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3124/6237] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:08.484352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f9724-8470-4b18-a7db-302a318b4451 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.491648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.510416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4cb46-2f9d-43fb-bc52-e9727231d1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.511557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4cb46-2f9d-43fb-bc52-e9727231d1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.512242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4cb46-2f9d-43fb-bc52-e9727231d1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.512404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4cb46-2f9d-43fb-bc52-e9727231d1ae admin admin] PATH_INFO: `/v3/projects/0420c3486c474bc9a1ad1b7dcacf4e85/users/6cee1c1229d847d284c4a5c9fbb754bd/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.519530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d98c5-acef-487f-9e04-5257fd33553c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.541631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.542342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.542709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.542863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] PATH_INFO: `/v3/projects/b31df8706e534b8799883c8a8b3328da/users/3059b130645f46f7aeb38a3d77b988de/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.558224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4cb46-2f9d-43fb-bc52-e9727231d1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0420c3486c474bc9a1ad1b7dcacf4e85, user_id=6cee1c1229d847d284c4a5c9fbb754bd, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.561278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4cb46-2f9d-43fb-bc52-e9727231d1ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.578975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31df8706e534b8799883c8a8b3328da, user_id=3059b130645f46f7aeb38a3d77b988de, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.589899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.600487 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3114/6238] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:08 2022] PUT /identity/v3/projects/0420c3486c474bc9a1ad1b7dcacf4e85/users/6cee1c1229d847d284c4a5c9fbb754bd/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:08.603986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad0974-80fd-48b9-9f50-b88d8acb1051 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.604480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad0974-80fd-48b9-9f50-b88d8acb1051 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.604809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad0974-80fd-48b9-9f50-b88d8acb1051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.636242 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3125/6239] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:08 2022] PUT /identity/v3/projects/b31df8706e534b8799883c8a8b3328da/users/3059b130645f46f7aeb38a3d77b988de/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:08.642704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.656409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:08.672938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b6ad0974-80fd-48b9-9f50-b88d8acb1051 None None] MFA Rules not processed for user `6cee1c1229d847d284c4a5c9fbb754bd`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:08.679044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b6ad0974-80fd-48b9-9f50-b88d8acb1051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:08.792045 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3115/6240] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:08.801918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a612f28-2069-49ef-9995-87ea8ace0b98 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.816062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975e2d3-d693-4a8f-944b-4262034ee320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.816720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975e2d3-d693-4a8f-944b-4262034ee320 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.816999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975e2d3-d693-4a8f-944b-4262034ee320 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.817268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975e2d3-d693-4a8f-944b-4262034ee320 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.817955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975e2d3-d693-4a8f-944b-4262034ee320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.819619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975e2d3-d693-4a8f-944b-4262034ee320 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.837507 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3126/6241] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:08.843145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad0974-80fd-48b9-9f50-b88d8acb1051 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.858231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad0974-80fd-48b9-9f50-b88d8acb1051 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.877124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.877837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.878176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.878538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] PATH_INFO: `/v3/projects/b31df8706e534b8799883c8a8b3328da/users/3059b130645f46f7aeb38a3d77b988de/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.881327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31df8706e534b8799883c8a8b3328da, user_id=3059b130645f46f7aeb38a3d77b988de, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.884049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.920024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3116/6242] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:08 2022] PUT /identity/v3/projects/b31df8706e534b8799883c8a8b3328da/users/3059b130645f46f7aeb38a3d77b988de/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:08.923496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.923782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.924053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:09.002816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] MFA Rules not processed for user `3059b130645f46f7aeb38a3d77b988de`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:09.004740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:09.073331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:09.075164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:09.232835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:09.329250 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3127/6243] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:09.342118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:09.345933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:09.363476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4c690-0c87-4e13-8b32-aecc6525816a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:09.368943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:09.369802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:09.370130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:09.370461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:09.476716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c58c6d-d78f-4172-a612-deab92900313 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:09.486442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:09.487861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:09.488471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:09.488911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:09.513295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:09.514958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:09.524096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:09.526519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:09.528691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3128/6244] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:09.567249 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3117/6245] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:14:10.299549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:10.310537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7809c7-3cf9-4f86-9dfe-d634eb2ed715 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:10.316130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3059b130645f46f7aeb38a3d77b988de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b31df8706e534b8799883c8a8b3328da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:10.317752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:10.318203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:10.318971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:10.319910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:10.320664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:10.322863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:10.351573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3129/6246] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:10.353901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:10.369522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef99e3-940e-4be6-a14f-f2d0618842ab service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:10.390039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3059b130645f46f7aeb38a3d77b988de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b31df8706e534b8799883c8a8b3328da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:10.390827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:10.391234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:10.391600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:10.392336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:10.394823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:10.462811 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3118/6247] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:10 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:10.467003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:10.482714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf2fad-6f11-442b-9175-7268063ac807 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:10.620608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3059b130645f46f7aeb38a3d77b988de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b31df8706e534b8799883c8a8b3328da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:10.624617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:10.624617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:10.624617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:10.624617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:10.624617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:10.630639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:10.658290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3130/6248] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:10.734821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:10.766397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5489d-2164-4e52-a303-b7c2bd3bc53d tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:10.774922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3059b130645f46f7aeb38a3d77b988de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b31df8706e534b8799883c8a8b3328da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:10.775905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:10.776400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:10.776884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] PATH_INFO: `/v3/projects/56cd9e0727614fd58fc28a4ba2cb2f03` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:10.779639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorizing `identity:delete_project(project_id=56cd9e0727614fd58fc28a4ba2cb2f03)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:10.782912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:10.814579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:10.899710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56cd9e0727614fd58fc28a4ba2cb2f03', 'request_id': 'req-ccaefc17-c294-4319-9f2d-03c72c0a6871'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:10.910700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56cd9e0727614fd58fc28a4ba2cb2f03', 'request_id': 'req-ccaefc17-c294-4319-9f2d-03c72c0a6871'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:10.923797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56cd9e0727614fd58fc28a4ba2cb2f03', 'request_id': 'req-ccaefc17-c294-4319-9f2d-03c72c0a6871'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:10.940909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] The token cache is being invalidate because project 56cd9e0727614fd58fc28a4ba2cb2f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:10.942633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:10.943598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ccaefc17-c294-4319-9f2d-03c72c0a6871 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:10.951034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3119/6249] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:10 2022] DELETE /identity/v3/projects/56cd9e0727614fd58fc28a4ba2cb2f03 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:11.419808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:11.421688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:11.546961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0369a-c488-4184-bd35-44b93ede32a8 tempest-VolumeQuotaClassesTest-566846268 tempest-VolumeQuotaClassesTest-566846268-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:11.571279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:11.637527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:11.638963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:11.639439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:11.639910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] PATH_INFO: `/v3/users/637e8d1af8384b5fa06414d502739b09` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:11.641388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] RBAC: Authorizing `identity:delete_user(user_id=637e8d1af8384b5fa06414d502739b09)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:11.643996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:11.749361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '637e8d1af8384b5fa06414d502739b09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:11.750061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '637e8d1af8384b5fa06414d502739b09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:11.771064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '637e8d1af8384b5fa06414d502739b09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:11.771638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '637e8d1af8384b5fa06414d502739b09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:11.794269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '637e8d1af8384b5fa06414d502739b09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:11.828435 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3131/6250] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:11 2022] DELETE /identity/v3/users/637e8d1af8384b5fa06414d502739b09 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:11.833262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:11.833857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:11.833857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:11.866762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:11.868174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:11.943910 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3120/6251] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:12.001712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:12.003716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:12.165338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb202c5-cd49-4f83-be7d-19b3bf55313e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:12.176265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:12.177030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:12.177588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:12.178069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:12.200915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:12.204638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:12.225241 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3132/6252] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:12.955889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:12.958134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:12.972911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5881113-9b08-4d71-971c-d3e425a2422d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:12.979736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:12.980699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:12.981141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:12.981621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] PATH_INFO: `/v3/projects/00b20946e00b4a559b37039b37d959e8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:12.983158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b20946e00b4a559b37039b37d959e8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:12.985190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:13.054522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b20946e00b4a559b37039b37d959e8', 'request_id': 'req-f018aaba-d3d9-4adb-891e-35b314e6221e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.066760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b20946e00b4a559b37039b37d959e8', 'request_id': 'req-f018aaba-d3d9-4adb-891e-35b314e6221e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.073854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b20946e00b4a559b37039b37d959e8', 'request_id': 'req-f018aaba-d3d9-4adb-891e-35b314e6221e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.087316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] The token cache is being invalidate because project 00b20946e00b4a559b37039b37d959e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:13.087757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.088609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.090337 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3121/6253] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:12 2022] DELETE /identity/v3/projects/00b20946e00b4a559b37039b37d959e8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:13.094536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:13.108364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7165309-b49d-4aa7-97f5-47e490e8644c service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:13.172344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:13.193868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:13.194849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:13.195126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:13.195395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] PATH_INFO: `/v3/users/3059b130645f46f7aeb38a3d77b988de` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:13.196610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3059b130645f46f7aeb38a3d77b988de)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:13.198683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:13.257321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3059b130645f46f7aeb38a3d77b988de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.258083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3059b130645f46f7aeb38a3d77b988de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.269506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3059b130645f46f7aeb38a3d77b988de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.270285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3059b130645f46f7aeb38a3d77b988de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.293323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2f162-78c9-49df-9c26-9c1e289456e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3059b130645f46f7aeb38a3d77b988de'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:13.314132 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3133/6254] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:13 2022] DELETE /identity/v3/users/3059b130645f46f7aeb38a3d77b988de => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:13.906314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:13.907957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:13.935384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018aaba-d3d9-4adb-891e-35b314e6221e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:14.005404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:14.076140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:14.076806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:14.077560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:14.077798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] PATH_INFO: `/v3/users/bfafdfdca1384c34ab470c22a1181a18` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:14.079090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfafdfdca1384c34ab470c22a1181a18)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:14.081330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:14.165515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfafdfdca1384c34ab470c22a1181a18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.190718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfafdfdca1384c34ab470c22a1181a18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.204813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfafdfdca1384c34ab470c22a1181a18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.205682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfafdfdca1384c34ab470c22a1181a18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.206466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfafdfdca1384c34ab470c22a1181a18'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.237138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:14.237720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:14.237720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:14.242808 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3122/6255] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:13 2022] DELETE /identity/v3/users/bfafdfdca1384c34ab470c22a1181a18 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:14.266575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:14.268409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:14.395782 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3134/6256] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:14.430594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:14.436298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:14.524950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:14.526765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:14.549422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4c2c2-4c57-4d0f-931f-0ee5489374b7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:14.561224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:14.562161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:14.562675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:14.563171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] PATH_INFO: `/v3/projects/b31df8706e534b8799883c8a8b3328da` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:14.568357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31df8706e534b8799883c8a8b3328da)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:14.572188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:14.578959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5aa48-6a46-4f8c-bd9d-1485ff37f0e7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:14.594544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ef6e4-518e-4415-a1d8-7d025e3dfbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:14.595937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef6e4-518e-4415-a1d8-7d025e3dfbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:14.596218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef6e4-518e-4415-a1d8-7d025e3dfbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:14.597072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef6e4-518e-4415-a1d8-7d025e3dfbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:14.624225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ef6e4-518e-4415-a1d8-7d025e3dfbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:14.626375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ef6e4-518e-4415-a1d8-7d025e3dfbb9 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:14.648912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31df8706e534b8799883c8a8b3328da', 'request_id': 'req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.650522 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3123/6257] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:14.657930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31df8706e534b8799883c8a8b3328da', 'request_id': 'req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.663089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:14.679354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31df8706e534b8799883c8a8b3328da', 'request_id': 'req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.709755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] The token cache is being invalidate because project b31df8706e534b8799883c8a8b3328da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:14.710032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.711733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc58dcd8-ee23-4ba3-9c9f-20f736b3918e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:14.715938 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3135/6258] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:14 2022] DELETE /identity/v3/projects/b31df8706e534b8799883c8a8b3328da => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:14.721000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580cc45-acba-4c3d-bb9f-037549d1dc25 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:14.721630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580cc45-acba-4c3d-bb9f-037549d1dc25 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:14.722120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580cc45-acba-4c3d-bb9f-037549d1dc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:14.744522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a580cc45-acba-4c3d-bb9f-037549d1dc25 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:14.746051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a580cc45-acba-4c3d-bb9f-037549d1dc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:14.813293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:14.864893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0307239-6be1-4e35-b536-a88b6451e3d3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:14.864893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0307239-6be1-4e35-b536-a88b6451e3d3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:14.865221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0307239-6be1-4e35-b536-a88b6451e3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:14.865369 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3124/6259] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:14.932557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f0307239-6be1-4e35-b536-a88b6451e3d3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:14.933987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f0307239-6be1-4e35-b536-a88b6451e3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:14.955125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580cc45-acba-4c3d-bb9f-037549d1dc25 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:14.964595 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3136/6260] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:14.969725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580cc45-acba-4c3d-bb9f-037549d1dc25 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:14.976959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:14.978456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:14.979212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:14.979704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:14.980602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:14.981192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:14.983642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:15.040316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0307239-6be1-4e35-b536-a88b6451e3d3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.064358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0307239-6be1-4e35-b536-a88b6451e3d3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:15.064565 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3125/6261] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:15.070138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:15.071067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:15.071511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:15.073431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:15.074354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:15.076720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:15.125933 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3137/6262] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:15 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:15.129488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.144815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510e640-fe13-416c-a3bf-e5335ba70697 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:15.230526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:15.231268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:15.231613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:15.232025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:15.232959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:15.235178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:15.276102 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3126/6263] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:15 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:15.277817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.294219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35585d98-9cdd-4f27-af4f-a0deee7ea618 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:15.307032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:15.310595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:15.311205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:15.311819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:15.312772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:15.315223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:15.333516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3138/6264] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:15.337290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.350965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6f0a3-8480-4b20-899e-6e967cf0f95b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:15.357749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d60aa3-0c36-4564-9d73-cbbc6451ca2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:15.359853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60aa3-0c36-4564-9d73-cbbc6451ca2c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:15.360470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60aa3-0c36-4564-9d73-cbbc6451ca2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:15.361180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60aa3-0c36-4564-9d73-cbbc6451ca2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:15.363157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d60aa3-0c36-4564-9d73-cbbc6451ca2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:15.366341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d60aa3-0c36-4564-9d73-cbbc6451ca2c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:15.425110 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3127/6265] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:15.431616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.443952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4a777-bb1e-4ef7-9978-9c495a07647c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:15.450787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:15.451905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:15.452254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:15.452590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] PATH_INFO: `/v3/projects/34de7789c0244148aea11a6e750b84bd/users/62ca0b3f2292407f887dbff6243abbda/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:15.500949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=34de7789c0244148aea11a6e750b84bd, user_id=62ca0b3f2292407f887dbff6243abbda, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:15.504720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:15.541671 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3139/6266] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:15 2022] PUT /identity/v3/projects/34de7789c0244148aea11a6e750b84bd/users/62ca0b3f2292407f887dbff6243abbda/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:15.555388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234df48c-bbc0-4717-b5bb-5dde246c71f5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:15.555759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234df48c-bbc0-4717-b5bb-5dde246c71f5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:15.556118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234df48c-bbc0-4717-b5bb-5dde246c71f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:15.600937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-234df48c-bbc0-4717-b5bb-5dde246c71f5 None None] MFA Rules not processed for user `62ca0b3f2292407f887dbff6243abbda`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:15.602552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-234df48c-bbc0-4717-b5bb-5dde246c71f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:15.748217 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3128/6267] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:15.749475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.857737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d89bec-84dc-4522-b68d-74cea35c24ee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:15.870861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:15.872807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:15.873319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:15.873851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:15.874700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:15.889440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:15.963075 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3140/6268] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:15 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:15.967386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234df48c-bbc0-4717-b5bb-5dde246c71f5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.982141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234df48c-bbc0-4717-b5bb-5dde246c71f5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:16.098130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:16.150290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:16.161319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.161707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.161952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.162646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:16.164673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:16.222995 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3129/6269] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:15 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:16.224714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:16.250395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69e9d-633d-46c6-acdd-51d6b8572c27 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:16.255760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:16.256396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.257723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.258031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.258748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:16.260491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:16.285340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3141/6270] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:16.289532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:16.309114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d14e0-b3e4-462f-9353-3b9bd9e37aa5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:16.348170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:16.348903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.349189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.349559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] PATH_INFO: `/v3/projects/3deb798b0f554dfcab3874dd3167cdf9/users/e3da7b3079b247e295ceb1e2d68925ff/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.412326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3deb798b0f554dfcab3874dd3167cdf9, user_id=e3da7b3079b247e295ceb1e2d68925ff, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:16.417349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:16.452241 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3130/6271] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:16 2022] PUT /identity/v3/projects/3deb798b0f554dfcab3874dd3167cdf9/users/e3da7b3079b247e295ceb1e2d68925ff/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:16.456762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:16.483029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d059c5b-f337-4019-867c-616652d5ff2d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:16.647339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:16.667060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:16.668394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.668683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.669178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.670040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:16.675229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:16.723516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:16.729399 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3142/6272] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:16.736039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3fceb-65cb-4ff6-97bf-c555f4249a56 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:16.741863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:16.743370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0357a5-b26a-40f7-a995-2e652aac8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:16.743688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:16.744477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0357a5-b26a-40f7-a995-2e652aac8d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.744826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0357a5-b26a-40f7-a995-2e652aac8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.745198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0357a5-b26a-40f7-a995-2e652aac8d98 admin admin] PATH_INFO: `/v3/projects/3deb798b0f554dfcab3874dd3167cdf9/users/e3da7b3079b247e295ceb1e2d68925ff/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.747420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0357a5-b26a-40f7-a995-2e652aac8d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3deb798b0f554dfcab3874dd3167cdf9, user_id=e3da7b3079b247e295ceb1e2d68925ff, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:16.750995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0357a5-b26a-40f7-a995-2e652aac8d98 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:16.766511 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3131/6273] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:16 2022] PUT /identity/v3/projects/3deb798b0f554dfcab3874dd3167cdf9/users/e3da7b3079b247e295ceb1e2d68925ff/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:16.777920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.778356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.779822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.795379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd1a9d-48f1-449f-8f88-60301842f2a7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:16.862758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] MFA Rules not processed for user `e3da7b3079b247e295ceb1e2d68925ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:16.864118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:16.976691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:16.977356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.979532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.979822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] PATH_INFO: `/v3/projects/6cba6418a4dd49638b346b6e5c367f38` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.984091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cba6418a4dd49638b346b6e5c367f38)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:16.986936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:17.067373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3132/6274] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:17.084476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:17.086661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:17.107314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cba6418a4dd49638b346b6e5c367f38', 'request_id': 'req-2dfea549-397c-428d-904a-de8676fd01a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.117447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cba6418a4dd49638b346b6e5c367f38', 'request_id': 'req-2dfea549-397c-428d-904a-de8676fd01a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.142887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cba6418a4dd49638b346b6e5c367f38', 'request_id': 'req-2dfea549-397c-428d-904a-de8676fd01a9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.194367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] The token cache is being invalidate because project 6cba6418a4dd49638b346b6e5c367f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:17.194821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.195629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.200637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3143/6275] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:16 2022] DELETE /identity/v3/projects/6cba6418a4dd49638b346b6e5c367f38 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:17.216554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:17.218761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:17.225938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70886f2-0021-4b7f-a0bb-c22ba9a2739b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:17.270250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:17.329944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc87110-12e8-45f5-9742-5bbd559190a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:17.330609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc87110-12e8-45f5-9742-5bbd559190a9 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:17.330886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc87110-12e8-45f5-9742-5bbd559190a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:17.331152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc87110-12e8-45f5-9742-5bbd559190a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:17.340845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4fc87110-12e8-45f5-9742-5bbd559190a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:17.423874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfea549-397c-428d-904a-de8676fd01a9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:17.437349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:17.440679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:17.441028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:17.441374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] PATH_INFO: `/v3/users/ccaee99abc364180ad38f44298b736e5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:17.445168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccaee99abc364180ad38f44298b736e5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:17.448465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:17.481867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc87110-12e8-45f5-9742-5bbd559190a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:17.483955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc87110-12e8-45f5-9742-5bbd559190a9 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:17.509987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:17.516455 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3133/6276] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:17.521151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:17.521896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:17.522428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:17.569390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccaee99abc364180ad38f44298b736e5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.570370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccaee99abc364180ad38f44298b736e5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.572741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] MFA Rules not processed for user `8ddb1d0a7bca457493d7ac9a58a01e64`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:17.574021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:17.593935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccaee99abc364180ad38f44298b736e5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.595352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccaee99abc364180ad38f44298b736e5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.620415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccaee99abc364180ad38f44298b736e5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:17.644092 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3144/6277] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:17 2022] DELETE /identity/v3/users/ccaee99abc364180ad38f44298b736e5 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:17.737685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3134/6278] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:17.787288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:17.788792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:17.966248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c526d38-374b-472a-9fc6-e2a6d5658524 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:17.999220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c744a9-a9a5-49bd-aa3b-0c2854edef58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:18.000240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c744a9-a9a5-49bd-aa3b-0c2854edef58 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:18.000851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c744a9-a9a5-49bd-aa3b-0c2854edef58 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:18.001637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c744a9-a9a5-49bd-aa3b-0c2854edef58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:18.009502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4c744a9-a9a5-49bd-aa3b-0c2854edef58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:18.221982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c744a9-a9a5-49bd-aa3b-0c2854edef58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:18.225179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c744a9-a9a5-49bd-aa3b-0c2854edef58 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:18.263497 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3145/6279] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:18.745352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:18.748088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:18.778180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b1c36-3d91-4f6b-bd87-f68609e1e085 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:18.919738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:18.920415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:18.920683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:18.920960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] PATH_INFO: `/v3/users/62ca0b3f2292407f887dbff6243abbda` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:18.922181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ca0b3f2292407f887dbff6243abbda)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:18.924380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:19.003791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:19.041580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ca0b3f2292407f887dbff6243abbda'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.060101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ca0b3f2292407f887dbff6243abbda'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.085323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ca0b3f2292407f887dbff6243abbda'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.086626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ca0b3f2292407f887dbff6243abbda'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.087458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ca0b3f2292407f887dbff6243abbda'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.124498 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3135/6280] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:18 2022] DELETE /identity/v3/users/62ca0b3f2292407f887dbff6243abbda => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:19.129198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cbd82-ceff-49b4-829d-045655091fef None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:19.131085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cbd82-ceff-49b4-829d-045655091fef None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:19.131593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cbd82-ceff-49b4-829d-045655091fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:19.166729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-812cbd82-ceff-49b4-829d-045655091fef None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:19.168735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-812cbd82-ceff-49b4-829d-045655091fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:19.252870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:19.275511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3146/6281] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:19.290017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:19.291642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:19.420888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221239b-46e0-417f-ac47-2c48188be869 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:19.425609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:19.426242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:19.426517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:19.426930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:19.440236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:19.441676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:19.453262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3136/6282] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:19.505518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cbd82-ceff-49b4-829d-045655091fef None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:19.507166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-812cbd82-ceff-49b4-829d-045655091fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:19.527748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:19.529706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:19.559133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f4809-c72e-44f3-8633-a3118347907d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:19.586547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:19.592381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:19.594033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:19.594483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] PATH_INFO: `/v3/projects/1a9a6279e2db4fcc981becdf3bc940c1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:19.596957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9a6279e2db4fcc981becdf3bc940c1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:19.599382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:19.776003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cbd82-ceff-49b4-829d-045655091fef None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:19.837779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9a6279e2db4fcc981becdf3bc940c1', 'request_id': 'req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.867206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9a6279e2db4fcc981becdf3bc940c1', 'request_id': 'req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.877673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9a6279e2db4fcc981becdf3bc940c1', 'request_id': 'req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.895774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] The token cache is being invalidate because project 1a9a6279e2db4fcc981becdf3bc940c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:19.896223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.897222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a12a7fe7-1985-4679-8ef2-8cc6aebe650d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.913844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3137/6283] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:19 2022] DELETE /identity/v3/projects/1a9a6279e2db4fcc981becdf3bc940c1 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:19.920219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eb099-5dca-4fba-b4fa-df04879103ba None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:19.920970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eb099-5dca-4fba-b4fa-df04879103ba None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:19.921635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eb099-5dca-4fba-b4fa-df04879103ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:19.961956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57b6e-94dd-4562-9908-74e11d576959 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:19.962873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57b6e-94dd-4562-9908-74e11d576959 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:19.963288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57b6e-94dd-4562-9908-74e11d576959 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:19.963665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57b6e-94dd-4562-9908-74e11d576959 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:19.971041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-23c57b6e-94dd-4562-9908-74e11d576959 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:19.988759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-801eb099-5dca-4fba-b4fa-df04879103ba None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:19.990889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-801eb099-5dca-4fba-b4fa-df04879103ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:20.011733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57b6e-94dd-4562-9908-74e11d576959 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.017988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57b6e-94dd-4562-9908-74e11d576959 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.030338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:20.055563 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3147/6284] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:20.076770 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3138/6285] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:20.086826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45fb0-dc5b-456a-85c8-9f8ddea1776b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.087307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45fb0-dc5b-456a-85c8-9f8ddea1776b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.087779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45fb0-dc5b-456a-85c8-9f8ddea1776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.131049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-13f45fb0-dc5b-456a-85c8-9f8ddea1776b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:20.133256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13f45fb0-dc5b-456a-85c8-9f8ddea1776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:20.147916 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3148/6286] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:20.152156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801eb099-5dca-4fba-b4fa-df04879103ba None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.167500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801eb099-5dca-4fba-b4fa-df04879103ba None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.173034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.173912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.174285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.174706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.175515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:20.175956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.177926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.198759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f45fb0-dc5b-456a-85c8-9f8ddea1776b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.203400 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3139/6287] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:20.210309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f45fb0-dc5b-456a-85c8-9f8ddea1776b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.224650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.225542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.226034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.226431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.227272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.232214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.297906 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3149/6288] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/projects => generated 397 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:20.302003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.317154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844b210-8d62-4223-90da-5f764aaca8e1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.488665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.489393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.489698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.490962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.493242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.496558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.576079 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3140/6289] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/users => generated 299 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:20.577648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.610497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54696f7a-7e07-462f-b8e0-40a76e6d9dac admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.611645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.612285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.621634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.622055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.623394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.625209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.654135 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3150/6290] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:20.656128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.675211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6aab-b6cc-450e-a671-1fc96e746ea8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.685952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478342f6-ffe2-42c6-8f6d-67ffa0e6cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.686590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478342f6-ffe2-42c6-8f6d-67ffa0e6cd73 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.686874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478342f6-ffe2-42c6-8f6d-67ffa0e6cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.687144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478342f6-ffe2-42c6-8f6d-67ffa0e6cd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.688845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478342f6-ffe2-42c6-8f6d-67ffa0e6cd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.690534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478342f6-ffe2-42c6-8f6d-67ffa0e6cd73 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.726391 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3141/6291] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:20.729548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.741596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf9e81-3526-4f01-a856-96c54b2ae8ff admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.754553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.755518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.755937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.756216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] PATH_INFO: `/v3/projects/79eff6f3f2aa46eeb2c7f8c7bb55fad7/users/1e44336ce1da4cd6b10da246e4ab8469/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.799901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79eff6f3f2aa46eeb2c7f8c7bb55fad7, user_id=1e44336ce1da4cd6b10da246e4ab8469, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.802674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.831942 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3151/6292] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:20 2022] PUT /identity/v3/projects/79eff6f3f2aa46eeb2c7f8c7bb55fad7/users/1e44336ce1da4cd6b10da246e4ab8469/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:20.843027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.843613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.843613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.880388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] MFA Rules not processed for user `1e44336ce1da4cd6b10da246e4ab8469`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:20.882567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.002025 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3142/6293] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:21.060384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:21.062551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.148402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:21.189675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f77bf-0734-4f94-8772-f0607236e706 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.195098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.195894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.196274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.196855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.209972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.211884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:21.224272 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3152/6294] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:21.398817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:21.402092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.452552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:21.521934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55101b-dab0-45b9-a234-b6866ff60319 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.531486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.533747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.534289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.534840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] PATH_INFO: `/v3/projects/34de7789c0244148aea11a6e750b84bd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.537374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34de7789c0244148aea11a6e750b84bd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.548696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:21.691319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34de7789c0244148aea11a6e750b84bd', 'request_id': 'req-175e6946-06ef-4236-874d-7ea107afb964'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.703828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34de7789c0244148aea11a6e750b84bd', 'request_id': 'req-175e6946-06ef-4236-874d-7ea107afb964'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.715281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34de7789c0244148aea11a6e750b84bd', 'request_id': 'req-175e6946-06ef-4236-874d-7ea107afb964'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.750816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] The token cache is being invalidate because project 34de7789c0244148aea11a6e750b84bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:21.751447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.752295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.754417 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3143/6295] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:21 2022] DELETE /identity/v3/projects/34de7789c0244148aea11a6e750b84bd => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:21.758463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:21.784226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441a3d8-4eb3-44ea-8613-acbe1a24efe4 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.895039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.895840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.896170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.896450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] PATH_INFO: `/v3/users/e3da7b3079b247e295ceb1e2d68925ff` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.898028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3da7b3079b247e295ceb1e2d68925ff)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.900182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.023912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3da7b3079b247e295ceb1e2d68925ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:22.024949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3da7b3079b247e295ceb1e2d68925ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:22.036745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3da7b3079b247e295ceb1e2d68925ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:22.037805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3da7b3079b247e295ceb1e2d68925ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:22.064074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9affff3d-f5a4-4a29-8aab-5d7e1782cdf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3da7b3079b247e295ceb1e2d68925ff'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:22.117187 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3153/6296] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:21 2022] DELETE /identity/v3/users/e3da7b3079b247e295ceb1e2d68925ff => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:24.976796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:24.980939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:25.005868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e6946-06ef-4236-874d-7ea107afb964 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:25.133597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:25.136790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:25.137156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:25.137442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] PATH_INFO: `/v3/projects/3deb798b0f554dfcab3874dd3167cdf9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:25.138802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3deb798b0f554dfcab3874dd3167cdf9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:25.140787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:25.270531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3deb798b0f554dfcab3874dd3167cdf9', 'request_id': 'req-ff4e70fb-6796-4470-8e09-4635351f00f4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:25.302392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3deb798b0f554dfcab3874dd3167cdf9', 'request_id': 'req-ff4e70fb-6796-4470-8e09-4635351f00f4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:25.308030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:25.309762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3deb798b0f554dfcab3874dd3167cdf9', 'request_id': 'req-ff4e70fb-6796-4470-8e09-4635351f00f4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:25.336029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] The token cache is being invalidate because project 3deb798b0f554dfcab3874dd3167cdf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:25.336423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:25.338002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff4e70fb-6796-4470-8e09-4635351f00f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:25.345153 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3144/6297] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:24 2022] DELETE /identity/v3/projects/3deb798b0f554dfcab3874dd3167cdf9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:25.350009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78bcf5-2775-452f-bb7b-39e88f80d429 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:25.350546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78bcf5-2775-452f-bb7b-39e88f80d429 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:25.350546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78bcf5-2775-452f-bb7b-39e88f80d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:25.388873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2c78bcf5-2775-452f-bb7b-39e88f80d429 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:25.390233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c78bcf5-2775-452f-bb7b-39e88f80d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:25.539847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:25.576113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7fbe-6ce7-4014-adfb-d35606ec89b9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:25.576496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7fbe-6ce7-4014-adfb-d35606ec89b9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:25.576654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed7fbe-6ce7-4014-adfb-d35606ec89b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:25.579551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3154/6298] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:25.601539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b3ed7fbe-6ce7-4014-adfb-d35606ec89b9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:25.604558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b3ed7fbe-6ce7-4014-adfb-d35606ec89b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:25.623061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78bcf5-2775-452f-bb7b-39e88f80d429 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:25.631468 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3145/6299] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:25.635998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78bcf5-2775-452f-bb7b-39e88f80d429 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:25.650905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:25.652214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:25.652660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:25.653064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:25.654555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:25.655269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:25.659604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:25.699562 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3155/6300] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:25.703991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed7fbe-6ce7-4014-adfb-d35606ec89b9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:25.717353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed7fbe-6ce7-4014-adfb-d35606ec89b9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:25.724535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:25.725659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:25.726160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:25.727775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:25.728539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:25.730940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:25.791738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:25.806160 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3146/6301] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:25 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:25.815490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c323408-0cee-4362-8901-a3768fce449a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:25.963990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:25.982184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:25.982702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:25.983157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:25.987101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:26.022258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:26.076328 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3156/6302] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:25 2022] POST /identity/v3/users => generated 291 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:26.080917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:26.102981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c97c4c-382d-487f-b136-72be050a6631 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:26.114364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:26.115480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:26.116127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:26.117270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:26.118253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:26.126477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:26.145946 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3147/6303] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:26.146936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:26.162802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d0ab-fb76-434a-add2-2be1a6796582 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:26.175705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2defa-ac76-4c0e-8335-2f38f2f89e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:26.179598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2defa-ac76-4c0e-8335-2f38f2f89e6f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:26.180348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2defa-ac76-4c0e-8335-2f38f2f89e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:26.180348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2defa-ac76-4c0e-8335-2f38f2f89e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:26.181033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2defa-ac76-4c0e-8335-2f38f2f89e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:26.185338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2defa-ac76-4c0e-8335-2f38f2f89e6f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:26.227488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:26.230339 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3157/6304] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:26.268900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fd54f-d7b1-47ad-a24a-04a911387f62 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:26.286552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:26.290856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:26.290856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:26.291880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] PATH_INFO: `/v3/projects/7e0169496f8740ca83bc6f92594ca95d/users/0042863b26ca408a9ab608edba9951ac/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:26.339454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0169496f8740ca83bc6f92594ca95d, user_id=0042863b26ca408a9ab608edba9951ac, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:26.346879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:26.379959 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3148/6305] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:26 2022] PUT /identity/v3/projects/7e0169496f8740ca83bc6f92594ca95d/users/0042863b26ca408a9ab608edba9951ac/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:26.393134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7ad0-457d-4693-b0f0-db07b1d36127 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:26.393134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7ad0-457d-4693-b0f0-db07b1d36127 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:26.393134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7ad0-457d-4693-b0f0-db07b1d36127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:26.427041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1f2a7ad0-457d-4693-b0f0-db07b1d36127 None None] MFA Rules not processed for user `0042863b26ca408a9ab608edba9951ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:26.428287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f2a7ad0-457d-4693-b0f0-db07b1d36127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:26.530528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:26.534608 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3158/6306] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:26.664593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b378025-6438-4f53-a8f9-7da4ac85718e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:26.681324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:26.684008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:26.684619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:26.685197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:26.689811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:26.691832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:26.786116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a7ad0-457d-4693-b0f0-db07b1d36127 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:26.798421 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3149/6307] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:26 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:26.806121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a7ad0-457d-4693-b0f0-db07b1d36127 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:26.887432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:26.961082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:26.961763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:26.962060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:26.962331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:26.963058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:26.965320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:27.019079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:27.026574 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3159/6308] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:26 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:27.035141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ca1a-3883-4eb0-a8d9-a4626b336301 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:27.062724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:27.064177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:27.066148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:27.067254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:27.069980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:27.076086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:27.110473 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3150/6309] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:27.115879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:27.135522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda576a-0b50-483f-ba02-37dee785a557 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:27.146356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:27.148315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:27.148611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:27.149447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] PATH_INFO: `/v3/projects/af958f07b8bd442cb3a41ddac03c1f2b/users/f35b4d4d03784526982ee60009ceb918/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:27.179388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af958f07b8bd442cb3a41ddac03c1f2b, user_id=f35b4d4d03784526982ee60009ceb918, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:27.183041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:27.205300 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3160/6310] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:27 2022] PUT /identity/v3/projects/af958f07b8bd442cb3a41ddac03c1f2b/users/f35b4d4d03784526982ee60009ceb918/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:27.209004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:27.225077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb2079-3999-4fa5-aff8-80fe7b834212 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:27.342216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746794c0-1704-4766-a323-31882eabb915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:27.343112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746794c0-1704-4766-a323-31882eabb915 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:27.343556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746794c0-1704-4766-a323-31882eabb915 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:27.343988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746794c0-1704-4766-a323-31882eabb915 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:27.344939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746794c0-1704-4766-a323-31882eabb915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:27.346769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746794c0-1704-4766-a323-31882eabb915 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:27.349600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:27.363818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3151/6311] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:27.367445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:27.383856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09babc31-cb2b-4dfc-99c9-85e2208ac86c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:27.389511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:27.390600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:27.390998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:27.391273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] PATH_INFO: `/v3/projects/af958f07b8bd442cb3a41ddac03c1f2b/users/f35b4d4d03784526982ee60009ceb918/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:27.393698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af958f07b8bd442cb3a41ddac03c1f2b, user_id=f35b4d4d03784526982ee60009ceb918, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:27.396416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:27.417520 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3161/6312] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:27 2022] PUT /identity/v3/projects/af958f07b8bd442cb3a41ddac03c1f2b/users/f35b4d4d03784526982ee60009ceb918/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:27.429323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:27.430120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:27.430530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:27.463901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] MFA Rules not processed for user `f35b4d4d03784526982ee60009ceb918`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:27.465439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:27.629667 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3152/6313] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:27.649452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:27.651637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:27.704719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:27.706861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:27.785930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94cc41-4eee-4417-bd09-88d6e17f8db5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:27.796772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:27.797437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:27.797738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:27.798013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:27.842777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78189f0-ec07-463d-b07d-481ec2f99871 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:27.854752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:27.856673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:27.857128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:27.858557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:27.862390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:27.981740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:27.985677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:28.028174 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3162/6314] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:28.069892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:28.072345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:28.090646 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3153/6315] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:34.510570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:34.524938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215cd23-5234-4682-b0cc-ce866482afb4 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:34.528919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:34.529588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:34.529866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:34.530134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:34.542335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:34.544426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:34.572221 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3163/6316] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:38.083207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:44.554519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:47.261353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:47.262936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:47.276551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f416711-5cae-4a3e-b7da-a42297e5e0f8 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:47.281956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:47.282606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:47.283075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:47.283349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] PATH_INFO: `/v3/users/2db8955202fe42d2a4c86a15b193787f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:47.285429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db8955202fe42d2a4c86a15b193787f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:47.287426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:47.373222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db8955202fe42d2a4c86a15b193787f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:47.395584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db8955202fe42d2a4c86a15b193787f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:47.408466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db8955202fe42d2a4c86a15b193787f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:47.409898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db8955202fe42d2a4c86a15b193787f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:47.411040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70ee84ea-f3f1-4f01-9e75-1167d556dbf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db8955202fe42d2a4c86a15b193787f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:47.438981 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3154/6317] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:47 2022] DELETE /identity/v3/users/2db8955202fe42d2a4c86a15b193787f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:48.517360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:48.535054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fffb26-071c-4d8f-8a78-c9de5bc49dd3 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:48.679324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:48.680320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:48.680774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:48.681301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] PATH_INFO: `/v3/projects/ee1481a8ca9c4dd590cd6c7c53179df8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:48.683063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1481a8ca9c4dd590cd6c7c53179df8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:48.685209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:48.783709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1481a8ca9c4dd590cd6c7c53179df8', 'request_id': 'req-0018d958-e0c6-46f3-913d-375d61ac7a61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:48.800006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1481a8ca9c4dd590cd6c7c53179df8', 'request_id': 'req-0018d958-e0c6-46f3-913d-375d61ac7a61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:48.823856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1481a8ca9c4dd590cd6c7c53179df8', 'request_id': 'req-0018d958-e0c6-46f3-913d-375d61ac7a61'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:48.848335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] The token cache is being invalidate because project ee1481a8ca9c4dd590cd6c7c53179df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:48.848335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:48.851540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0018d958-e0c6-46f3-913d-375d61ac7a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:48.853445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3164/6318] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:48 2022] DELETE /identity/v3/projects/ee1481a8ca9c4dd590cd6c7c53179df8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:48.859480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c5b19-74a2-40ed-ba51-56bacbd4a982 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:48.860023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c5b19-74a2-40ed-ba51-56bacbd4a982 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:48.860023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c5b19-74a2-40ed-ba51-56bacbd4a982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:48.893827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c94c5b19-74a2-40ed-ba51-56bacbd4a982 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:48.895070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c94c5b19-74a2-40ed-ba51-56bacbd4a982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:49.045120 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3155/6319] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:49.053251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134c3d-4a33-49be-b129-31f294203474 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.053556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134c3d-4a33-49be-b129-31f294203474 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.054178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134c3d-4a33-49be-b129-31f294203474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.088555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e8134c3d-4a33-49be-b129-31f294203474 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:49.089841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e8134c3d-4a33-49be-b129-31f294203474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:49.110979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3165/6320] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:49.115734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c5b19-74a2-40ed-ba51-56bacbd4a982 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:49.133144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c5b19-74a2-40ed-ba51-56bacbd4a982 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:49.145817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:49.147894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.148530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.149375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.151413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:49.152391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:49.156676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:49.185528 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3156/6321] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:49.190152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134c3d-4a33-49be-b129-31f294203474 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:49.202932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134c3d-4a33-49be-b129-31f294203474 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:49.214929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:49.216141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.216618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.217084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.217977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:49.222864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:49.294007 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3166/6322] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:49 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:49.298272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:49.311151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9abe50-822b-469d-8df0-3b081652c70d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:49.389165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:49.390708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.394820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.395210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.396474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:49.398517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:49.421310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:49.455484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:49.457589 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3157/6323] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:49 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:49.474654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf3a7d-771d-46f1-a33d-1261b9bcaf5e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:49.487848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:49.488622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.488961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.489503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.490183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:49.501952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:49.527391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3167/6324] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:49.533101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:49.547837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4279be-b844-48a0-9228-a23fb3cc5c98 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:49.556722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f76a4-2eff-4d89-8f83-19e8c622ce7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:49.558396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f76a4-2eff-4d89-8f83-19e8c622ce7e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.561444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f76a4-2eff-4d89-8f83-19e8c622ce7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.562751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f76a4-2eff-4d89-8f83-19e8c622ce7e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.564040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f76a4-2eff-4d89-8f83-19e8c622ce7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:49.567253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f76a4-2eff-4d89-8f83-19e8c622ce7e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:49.583008 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3158/6325] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:49.587307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:49.605792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e028f-95d0-4e17-8090-86f829b48896 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:49.626161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419e44b-911f-4189-ac56-cfe0c622d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:49.631235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419e44b-911f-4189-ac56-cfe0c622d179 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.631817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419e44b-911f-4189-ac56-cfe0c622d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.632196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419e44b-911f-4189-ac56-cfe0c622d179 admin admin] PATH_INFO: `/v3/projects/aedb1becfa4b42deb13b265c9377e368/users/c777e5af74e64cb3805ef28d1d0ed776/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.676598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419e44b-911f-4189-ac56-cfe0c622d179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aedb1becfa4b42deb13b265c9377e368, user_id=c777e5af74e64cb3805ef28d1d0ed776, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:49.682828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419e44b-911f-4189-ac56-cfe0c622d179 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:49.720348 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3168/6326] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:49 2022] PUT /identity/v3/projects/aedb1becfa4b42deb13b265c9377e368/users/c777e5af74e64cb3805ef28d1d0ed776/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:49.730505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.730789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.731066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:49.786544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] MFA Rules not processed for user `c777e5af74e64cb3805ef28d1d0ed776`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:49.787912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:49.933550 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3159/6327] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:49.941302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:49.941812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:49.942230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:50.002228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] MFA Rules not processed for user `c777e5af74e64cb3805ef28d1d0ed776`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:50.003706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:50.020244 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3169/6328] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:50.032474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:50.034281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:50.183640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9298ad-3702-4ed6-9fe7-8cf9f52a29f8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:50.190531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:50.191833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:50.192202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:50.192500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:50.207451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:50.209201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:50.248545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3160/6329] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:57.428488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:57.430082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:57.485424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:57.597040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4d0b0-e4ca-453b-a014-7b01cad90f2c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:57.603280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:57.604223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:57.604513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:57.604790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] PATH_INFO: `/v3/users/7a13dc0c95a64d499bd05f808565fccd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:57.608473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a13dc0c95a64d499bd05f808565fccd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:57.613686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:57.719618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a13dc0c95a64d499bd05f808565fccd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.720577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a13dc0c95a64d499bd05f808565fccd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.742659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a13dc0c95a64d499bd05f808565fccd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.742999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a13dc0c95a64d499bd05f808565fccd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.770577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a13dc0c95a64d499bd05f808565fccd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.789441 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3170/6330] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:57 2022] DELETE /identity/v3/users/7a13dc0c95a64d499bd05f808565fccd => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:59.052220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.076438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294cd3d-ea61-4190-b1af-986882f6818b service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.177560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:59.185018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.187664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:59.196443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.198530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.201863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.202183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] PATH_INFO: `/v3/projects/c4a4d7fd12784db69cf699e7e58d4d6a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.204078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a4d7fd12784db69cf699e7e58d4d6a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.207100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.209926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126463e-5fb6-40ea-bb98-c5925c3be910 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.219842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.220612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.220893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.221925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] PATH_INFO: `/v3/users/0042863b26ca408a9ab608edba9951ac` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.243594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0042863b26ca408a9ab608edba9951ac)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.245686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.292320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a4d7fd12784db69cf699e7e58d4d6a', 'request_id': 'req-a9107a29-6416-43d7-85b1-03e0ba0b9607'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.308644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a4d7fd12784db69cf699e7e58d4d6a', 'request_id': 'req-a9107a29-6416-43d7-85b1-03e0ba0b9607'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.317988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a4d7fd12784db69cf699e7e58d4d6a', 'request_id': 'req-a9107a29-6416-43d7-85b1-03e0ba0b9607'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.347229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] The token cache is being invalidate because project c4a4d7fd12784db69cf699e7e58d4d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:59.347811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.349184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.352567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3161/6331] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:59 2022] DELETE /identity/v3/projects/c4a4d7fd12784db69cf699e7e58d4d6a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:59.357746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.360344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:59.376854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0042863b26ca408a9ab608edba9951ac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.379941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0042863b26ca408a9ab608edba9951ac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.403444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0042863b26ca408a9ab608edba9951ac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.404551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0042863b26ca408a9ab608edba9951ac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.435814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1e646a89-0460-4c39-843c-6b10b5d8cf86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0042863b26ca408a9ab608edba9951ac'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.466124 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3171/6332] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:59 2022] DELETE /identity/v3/users/0042863b26ca408a9ab608edba9951ac => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:59.470308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.470600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.470875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.506436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9107a29-6416-43d7-85b1-03e0ba0b9607 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.528970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:59.530280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:59.546751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:59.644026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.644793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.645697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.645697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] PATH_INFO: `/v3/users/0b0991772b84482bbb1a6ae35629558d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.648603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b0991772b84482bbb1a6ae35629558d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.650761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.732815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:59.757903 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3172/6333] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:59.763118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b0991772b84482bbb1a6ae35629558d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.789299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b0991772b84482bbb1a6ae35629558d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.805593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b0991772b84482bbb1a6ae35629558d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.806638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b0991772b84482bbb1a6ae35629558d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.807522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b0991772b84482bbb1a6ae35629558d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:59.825200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.827238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:59.847770 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3162/6334] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:59 2022] DELETE /identity/v3/users/0b0991772b84482bbb1a6ae35629558d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:59.912713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a77513-c354-4b86-977a-b6cd27a4cc01 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:00.029244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110de1e-7bc3-4623-b7c9-8b1486c19e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:00.030014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110de1e-7bc3-4623-b7c9-8b1486c19e47 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:00.030359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110de1e-7bc3-4623-b7c9-8b1486c19e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:00.030739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110de1e-7bc3-4623-b7c9-8b1486c19e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:00.037889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6110de1e-7bc3-4623-b7c9-8b1486c19e47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:00.131648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110de1e-7bc3-4623-b7c9-8b1486c19e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:00.133204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110de1e-7bc3-4623-b7c9-8b1486c19e47 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:00.147779 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3173/6335] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:00.647134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:00.652460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:00.708619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b7089-eed0-40d4-aa07-e0ccab5022e9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:00.725050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:00.728102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:00.728816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:00.729445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] PATH_INFO: `/v3/projects/3271ad542baf4788824fb264cbc5d0d8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:00.731439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3271ad542baf4788824fb264cbc5d0d8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:00.733935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:00.854332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3271ad542baf4788824fb264cbc5d0d8', 'request_id': 'req-14708098-3e56-492f-b548-c6678380ccd1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:00.867954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3271ad542baf4788824fb264cbc5d0d8', 'request_id': 'req-14708098-3e56-492f-b548-c6678380ccd1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:00.881747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3271ad542baf4788824fb264cbc5d0d8', 'request_id': 'req-14708098-3e56-492f-b548-c6678380ccd1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:00.912652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] The token cache is being invalidate because project 3271ad542baf4788824fb264cbc5d0d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:00.913680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:00.914528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:00.917537 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3163/6336] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:00 2022] DELETE /identity/v3/projects/3271ad542baf4788824fb264cbc5d0d8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:00.922655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83fb59-7cf5-45f6-ae86-43a50933adbb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:00.923494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83fb59-7cf5-45f6-ae86-43a50933adbb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:00.924202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83fb59-7cf5-45f6-ae86-43a50933adbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:00.952105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8d83fb59-7cf5-45f6-ae86-43a50933adbb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:00.954292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8d83fb59-7cf5-45f6-ae86-43a50933adbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:00.969028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:01.079258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:01.080842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:01.093958 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3174/6337] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:01.104919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced96584-1469-40d0-bce2-2f43b52a4418 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.105209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced96584-1469-40d0-bce2-2f43b52a4418 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.105493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced96584-1469-40d0-bce2-2f43b52a4418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.128516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14708098-3e56-492f-b548-c6678380ccd1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:01.138484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ced96584-1469-40d0-bce2-2f43b52a4418 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:01.141845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ced96584-1469-40d0-bce2-2f43b52a4418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:01.144395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:01.149317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.154264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.154264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] PATH_INFO: `/v3/projects/7e0169496f8740ca83bc6f92594ca95d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.160954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0169496f8740ca83bc6f92594ca95d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:01.169879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:01.172059 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3175/6338] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:01.176767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced96584-1469-40d0-bce2-2f43b52a4418 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:01.210416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced96584-1469-40d0-bce2-2f43b52a4418 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:01.238214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:01.239021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.239399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.239968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.240594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:01.241197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:01.247494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:01.281394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0169496f8740ca83bc6f92594ca95d', 'request_id': 'req-192ad388-d59a-4296-ae06-ed1de66ee35d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.297503 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3176/6339] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:15:01.303043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:01.310078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0169496f8740ca83bc6f92594ca95d', 'request_id': 'req-192ad388-d59a-4296-ae06-ed1de66ee35d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.326780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0169496f8740ca83bc6f92594ca95d', 'request_id': 'req-192ad388-d59a-4296-ae06-ed1de66ee35d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.328478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d2475-3a4f-4685-8be2-db7d899a4f35 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:01.358012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] The token cache is being invalidate because project 7e0169496f8740ca83bc6f92594ca95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:01.359109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.359908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.374717 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3164/6340] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:01 2022] DELETE /identity/v3/projects/7e0169496f8740ca83bc6f92594ca95d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:01.390013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:01.391708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:01.426875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:01.457817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:01.458891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.459401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.459887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.460742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:01.463030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:01.562702 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3177/6341] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:01 2022] POST /identity/v3/projects => generated 381 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:01.564273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:01.569848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:01.580153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ad388-d59a-4296-ae06-ed1de66ee35d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:01.766491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:01.777988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:01.779809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.780183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.780571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] PATH_INFO: `/v3/users/f35b4d4d03784526982ee60009ceb918` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.787917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35b4d4d03784526982ee60009ceb918)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:01.791335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6e94f-e788-4cc6-b147-f84086dcf00f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:01.793851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:01.804790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:01.805483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.805756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.806061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] PATH_INFO: `/v3/users/caa2c2f4848e497f80d4a439c785c2b0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.808447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa2c2f4848e497f80d4a439c785c2b0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:01.819948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:01.931328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35b4d4d03784526982ee60009ceb918'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.954786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa2c2f4848e497f80d4a439c785c2b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.955728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa2c2f4848e497f80d4a439c785c2b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.956611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35b4d4d03784526982ee60009ceb918'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.966282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa2c2f4848e497f80d4a439c785c2b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.967065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa2c2f4848e497f80d4a439c785c2b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.970244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35b4d4d03784526982ee60009ceb918'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.971797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35b4d4d03784526982ee60009ceb918'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:01.973284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35b4d4d03784526982ee60009ceb918'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:02.000803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa2c2f4848e497f80d4a439c785c2b0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:02.008341 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3165/6342] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:01 2022] DELETE /identity/v3/users/f35b4d4d03784526982ee60009ceb918 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:02.009304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.018599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.022335 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3178/6343] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:01 2022] DELETE /identity/v3/users/caa2c2f4848e497f80d4a439c785c2b0 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:02.025369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.027938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.070621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:02.160943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3458c0f-cead-4bb1-85ca-571655baa268 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.167416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afde28a-007a-4a43-929b-79c8e6eecaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:02.168162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afde28a-007a-4a43-929b-79c8e6eecaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.168555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afde28a-007a-4a43-929b-79c8e6eecaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.168920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afde28a-007a-4a43-929b-79c8e6eecaa6 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.170009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afde28a-007a-4a43-929b-79c8e6eecaa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:02.172955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afde28a-007a-4a43-929b-79c8e6eecaa6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:02.187160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85863987-23c1-4686-ae5f-863dcfe9e3a1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.197888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:02.199281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.199741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.200096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] PATH_INFO: `/v3/users/1e44336ce1da4cd6b10da246e4ab8469` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.201736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e44336ce1da4cd6b10da246e4ab8469)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:02.205291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:02.222349 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3166/6344] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:02 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:02.224028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26b732-7e7a-4fae-9298-74de17495c19 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.224317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26b732-7e7a-4fae-9298-74de17495c19 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.224640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26b732-7e7a-4fae-9298-74de17495c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.248308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ec26b732-7e7a-4fae-9298-74de17495c19 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:02.249607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ec26b732-7e7a-4fae-9298-74de17495c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.265082 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3167/6345] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:02.265082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26b732-7e7a-4fae-9298-74de17495c19 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.287596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26b732-7e7a-4fae-9298-74de17495c19 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.292761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:02.293434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.293766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.294040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.294407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e44336ce1da4cd6b10da246e4ab8469'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:02.294939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:02.295435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e44336ce1da4cd6b10da246e4ab8469'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:02.296916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:02.316167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e44336ce1da4cd6b10da246e4ab8469'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:02.316878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e44336ce1da4cd6b10da246e4ab8469'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:02.322153 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3168/6346] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:02.323113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.325114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.338787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e44336ce1da4cd6b10da246e4ab8469'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:02.340670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:02.363914 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3179/6347] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:02 2022] DELETE /identity/v3/users/1e44336ce1da4cd6b10da246e4ab8469 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:02.364902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.368982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.471532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e20961-78a0-4240-aa99-d61da8941c93 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.472651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b3196-8da7-4af2-8e60-dc82fa94732d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.479096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce4665-b81b-4917-80b5-0022bfc60e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:02.479719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce4665-b81b-4917-80b5-0022bfc60e89 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.480085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce4665-b81b-4917-80b5-0022bfc60e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.480508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce4665-b81b-4917-80b5-0022bfc60e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.481160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce4665-b81b-4917-80b5-0022bfc60e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:02.482820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce4665-b81b-4917-80b5-0022bfc60e89 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:02.498691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3180/6348] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:02.510933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056d84-b251-4930-ae72-36c62d0fc2ad None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.511239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056d84-b251-4930-ae72-36c62d0fc2ad None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.511515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056d84-b251-4930-ae72-36c62d0fc2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.546269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:02.547308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.547826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.548331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.550754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.555492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0f056d84-b251-4930-ae72-36c62d0fc2ad None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:02.557144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f056d84-b251-4930-ae72-36c62d0fc2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.576831 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3181/6349] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:02.577723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056d84-b251-4930-ae72-36c62d0fc2ad None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.583666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:02.588574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:02.599037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056d84-b251-4930-ae72-36c62d0fc2ad None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.603860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210f381-15b0-4836-b13b-f51d8bd01751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:02.604662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210f381-15b0-4836-b13b-f51d8bd01751 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.604940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210f381-15b0-4836-b13b-f51d8bd01751 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.605207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210f381-15b0-4836-b13b-f51d8bd01751 admin admin] PATH_INFO: `/v3/projects/bdfb56572d9c4d559ad4fbce7cb6d527/users/753144834d1a418897a600589137fcbc/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.615542 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3169/6350] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:02.616464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.632756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969bceb-ed4a-4338-87ff-a6f7a12debe8 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.638471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210f381-15b0-4836-b13b-f51d8bd01751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfb56572d9c4d559ad4fbce7cb6d527, user_id=753144834d1a418897a600589137fcbc, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:02.642359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210f381-15b0-4836-b13b-f51d8bd01751 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:02.651177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:02.656142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.656430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.656699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.685676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:02.688828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:02.708517 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3182/6351] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:02 2022] PUT /identity/v3/projects/bdfb56572d9c4d559ad4fbce7cb6d527/users/753144834d1a418897a600589137fcbc/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:02.714916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:02.715573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:02.716554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:02.720776 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3170/6352] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:02.761003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:02.762848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.882475 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3183/6353] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:02.912579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:02.913867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:02.942156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38602e1b-ce3f-4e69-86e4-cff1a4587fb5 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:02.978578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:03.019228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.021330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.047147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7735ac5-2d2f-4ca2-9768-7cb723277fda None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.049833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.050982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.052024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.052587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.059717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.063058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.063500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.063928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] PATH_INFO: `/v3/projects/af958f07b8bd442cb3a41ddac03c1f2b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.066881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af958f07b8bd442cb3a41ddac03c1f2b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.073930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.079377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.082912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.106946 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3171/6354] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:03.148124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.149612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.152802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af958f07b8bd442cb3a41ddac03c1f2b', 'request_id': 'req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.156062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:03.164106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af958f07b8bd442cb3a41ddac03c1f2b', 'request_id': 'req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.171748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad858d4c-12bb-468e-b874-d757355010dc service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.181762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af958f07b8bd442cb3a41ddac03c1f2b', 'request_id': 'req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.215697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] The token cache is being invalidate because project af958f07b8bd442cb3a41ddac03c1f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:03.216126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.218241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce40a306-b43a-4ab5-9f8c-0a21f4f80da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.231670 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3184/6355] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/projects/af958f07b8bd442cb3a41ddac03c1f2b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:03.233505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9218-4e6f-46f4-9fe6-c5dba894a7ae None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.238922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9218-4e6f-46f4-9fe6-c5dba894a7ae None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.239458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9218-4e6f-46f4-9fe6-c5dba894a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.266257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-50dc9218-4e6f-46f4-9fe6-c5dba894a7ae None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:03.267797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-50dc9218-4e6f-46f4-9fe6-c5dba894a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.293225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.293945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.294705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.294705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] PATH_INFO: `/v3/projects/cfc26054d4a34fb2b2aef34fb4891b4c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.306512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc26054d4a34fb2b2aef34fb4891b4c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.308507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.367234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3185/6356] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:03.379294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.379926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.380279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.401769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc26054d4a34fb2b2aef34fb4891b4c', 'request_id': 'req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.410855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:03.414476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.422700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc26054d4a34fb2b2aef34fb4891b4c', 'request_id': 'req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.429443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc26054d4a34fb2b2aef34fb4891b4c', 'request_id': 'req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.457205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] The token cache is being invalidate because project cfc26054d4a34fb2b2aef34fb4891b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:03.457741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.460795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.471265 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3172/6357] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/projects/cfc26054d4a34fb2b2aef34fb4891b4c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:03.475704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.480570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.534396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3186/6358] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:03.535463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.539005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.558062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfcc0a-7b11-479a-b6c6-f99ed7e63d25 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.565287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.566080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.566368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.566735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] PATH_INFO: `/v3/users/2e1a3388bac14ababdbc8e5e048e3263` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.568457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1a3388bac14ababdbc8e5e048e3263)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.571001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.580498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:03.619575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b974f-6f8d-42ce-af8d-0c1bd41f88a7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.628950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.629804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.630146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.630552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] PATH_INFO: `/v3/projects/79eff6f3f2aa46eeb2c7f8c7bb55fad7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.632157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79eff6f3f2aa46eeb2c7f8c7bb55fad7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.635174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.673647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1a3388bac14ababdbc8e5e048e3263'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.674794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1a3388bac14ababdbc8e5e048e3263'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.713288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1a3388bac14ababdbc8e5e048e3263'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.715815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1a3388bac14ababdbc8e5e048e3263'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.719071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79eff6f3f2aa46eeb2c7f8c7bb55fad7', 'request_id': 'req-c0b8e58c-b708-4bbf-91e7-fd030ff32814'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.737964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79eff6f3f2aa46eeb2c7f8c7bb55fad7', 'request_id': 'req-c0b8e58c-b708-4bbf-91e7-fd030ff32814'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.757725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79eff6f3f2aa46eeb2c7f8c7bb55fad7', 'request_id': 'req-c0b8e58c-b708-4bbf-91e7-fd030ff32814'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.763646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1a3388bac14ababdbc8e5e048e3263'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.788177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:03.815349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] The token cache is being invalidate because project 79eff6f3f2aa46eeb2c7f8c7bb55fad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:03.818083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.821733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c0b8e58c-b708-4bbf-91e7-fd030ff32814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.824548 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3187/6359] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/users/2e1a3388bac14ababdbc8e5e048e3263 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:03.825771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.829717 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3173/6360] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/projects/79eff6f3f2aa46eeb2c7f8c7bb55fad7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:03.832286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893976-5fce-494f-9f7f-c260a1afb157 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.833499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893976-5fce-494f-9f7f-c260a1afb157 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.833867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.834254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff893976-5fce-494f-9f7f-c260a1afb157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.886624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ff893976-5fce-494f-9f7f-c260a1afb157 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:03.888053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ff893976-5fce-494f-9f7f-c260a1afb157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.022149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3174/6361] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:04.035906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fd235-8664-49cc-a0ae-ac71c83ef91a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.037825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fd235-8664-49cc-a0ae-ac71c83ef91a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.038349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fd235-8664-49cc-a0ae-ac71c83ef91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.043169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9a4da-0a8e-499d-9191-544181f3da86 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.049654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.058014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.058571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.059053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.060611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:04.061090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.063227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.068618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d47fd235-8664-49cc-a0ae-ac71c83ef91a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:04.069892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d47fd235-8664-49cc-a0ae-ac71c83ef91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.079246 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3188/6362] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:04.079831 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3175/6363] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:04.084035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.084401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fd235-8664-49cc-a0ae-ac71c83ef91a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.088286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.111928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85264cf8-9fc6-4ab5-861b-b622bcd894b6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.123225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.124171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.124463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.124738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.127049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.127750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fd235-8664-49cc-a0ae-ac71c83ef91a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.129804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.138178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.139130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.140241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.140685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.141644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:04.142102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.144077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.167329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3176/6364] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:04.172182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.183147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4b7c0-5040-4909-a09b-d2e659873e24 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.187041 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3189/6365] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:04.188113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.274368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.275274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.275860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.276371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.278815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.279338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.287101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cf8fa-dea7-4294-bf01-049a9d91e82a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.290627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:04.308691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.312388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.312388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.312388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.314212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.316550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.342570 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3177/6366] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:04.343647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.367574 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3190/6367] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:04.370049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.382040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f31c12-9404-4ea2-a64e-ccec79a04037 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.454331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:04.464350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.465100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.465502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.465896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.466675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.468773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.480503 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3191/6368] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:04.483040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6ce39-a6ed-461b-816f-0bf0af00ea41 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.484868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.495692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.496432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.496779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.497682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.498469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.500367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38788e-cb57-4cbe-bdd5-9a93515d7179 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.500664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.507779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.508525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.509441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.509760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.511191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.514567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.538209 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3192/6369] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:04.545954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.547081 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3178/6370] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:04.549269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.571464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f57e49-dd7c-490d-bafe-6a39e89a1e78 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.575314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b6c92-2aa1-4d2e-9981-5437d1016087 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.586507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68f009-37f2-4167-b6a5-526ceae9cfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.587247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68f009-37f2-4167-b6a5-526ceae9cfda admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.587565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68f009-37f2-4167-b6a5-526ceae9cfda admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.587981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68f009-37f2-4167-b6a5-526ceae9cfda admin admin] PATH_INFO: `/v3/projects/dbeef98f1fb44c26a13741e637a819d7/users/0e63bded930641d8a15958628cf9a7aa/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.590166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.591146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.591571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.591849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.594026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.597422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.623246 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3179/6371] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:04.624266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.630815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68f009-37f2-4167-b6a5-526ceae9cfda admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbeef98f1fb44c26a13741e637a819d7, user_id=0e63bded930641d8a15958628cf9a7aa, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.637389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68f009-37f2-4167-b6a5-526ceae9cfda admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.645224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee2e16-0460-4627-af12-2d23aa165be7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.654273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.654996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.655279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.655638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.656294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.658395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.671965 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3180/6372] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:04.676390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.691220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f279a-adaf-4d79-9007-7102f8fc3303 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.700081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724e73b-540c-4b03-a4f4-85b7330cf1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.700829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724e73b-540c-4b03-a4f4-85b7330cf1c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.701115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724e73b-540c-4b03-a4f4-85b7330cf1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.701421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724e73b-540c-4b03-a4f4-85b7330cf1c0 admin admin] PATH_INFO: `/v3/projects/0068cbc531084f16b79007a63f869fa6/users/547a92bdb2db4116a4ea03e257a52f96/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.726541 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3193/6373] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:04 2022] PUT /identity/v3/projects/dbeef98f1fb44c26a13741e637a819d7/users/0e63bded930641d8a15958628cf9a7aa/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:04.728691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.729268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.729626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.791884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724e73b-540c-4b03-a4f4-85b7330cf1c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0068cbc531084f16b79007a63f869fa6, user_id=547a92bdb2db4116a4ea03e257a52f96, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.817304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724e73b-540c-4b03-a4f4-85b7330cf1c0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.829488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] MFA Rules not processed for user `0e63bded930641d8a15958628cf9a7aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:04.843416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.886073 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3181/6374] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:04 2022] PUT /identity/v3/projects/0068cbc531084f16b79007a63f869fa6/users/547a92bdb2db4116a4ea03e257a52f96/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:04.896865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.898470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.898757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.943463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] MFA Rules not processed for user `547a92bdb2db4116a4ea03e257a52f96`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:04.947211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.001386 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3194/6375] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:05.044702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.049609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.063472 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3182/6376] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:05.096212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.099732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.148056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:05.214206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade33979-d9e6-46fb-89aa-d4b99fcc37c1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.219834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.220497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.220747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.221021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.232470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.234399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.244596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ab6c-5980-48f7-9635-9f85adfacca6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.248218 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3195/6377] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:05.260157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.260954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.261572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.261741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.290247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.293035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.320313 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3183/6378] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:05.612483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.614327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.731445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:05.776401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d312d7-7061-47f0-9d9d-60a1636fe3aa service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.797535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.798353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.798686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.798992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] PATH_INFO: `/v3/projects/b2cf5878e6d4483b9d978ff5cb045a24` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.810681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2cf5878e6d4483b9d978ff5cb045a24)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.814597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.952079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2cf5878e6d4483b9d978ff5cb045a24', 'request_id': 'req-2d3e7722-fdae-4eee-bf47-03d46134a161'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.970803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2cf5878e6d4483b9d978ff5cb045a24', 'request_id': 'req-2d3e7722-fdae-4eee-bf47-03d46134a161'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.995950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2cf5878e6d4483b9d978ff5cb045a24', 'request_id': 'req-2d3e7722-fdae-4eee-bf47-03d46134a161'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.015961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] The token cache is being invalidate because project b2cf5878e6d4483b9d978ff5cb045a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:06.016399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.017433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.021234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3196/6379] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:05 2022] DELETE /identity/v3/projects/b2cf5878e6d4483b9d978ff5cb045a24 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:06.025964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.044315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad8a36-4004-4c2e-8f4a-b92d05a6221c service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.179610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.180473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.180751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.181068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] PATH_INFO: `/v3/users/e22b2eb3624d4950b0defc614e5a4476` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.209050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e22b2eb3624d4950b0defc614e5a4476)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.211145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.364399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e22b2eb3624d4950b0defc614e5a4476'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.382448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e22b2eb3624d4950b0defc614e5a4476'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.396757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e22b2eb3624d4950b0defc614e5a4476'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.398245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e22b2eb3624d4950b0defc614e5a4476'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.399398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e22b2eb3624d4950b0defc614e5a4476'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.426304 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3184/6380] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:06 2022] DELETE /identity/v3/users/e22b2eb3624d4950b0defc614e5a4476 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:06.824357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.825869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.844580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e7722-fdae-4eee-bf47-03d46134a161 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.933166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.935682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.968569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.969645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.970132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.971042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] PATH_INFO: `/v3/users/547a92bdb2db4116a4ea03e257a52f96` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.973288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=547a92bdb2db4116a4ea03e257a52f96)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.976918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:07.035031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:07.087052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcfab6-dbeb-4e79-872c-07fec022a6f9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:07.089097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547a92bdb2db4116a4ea03e257a52f96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.090133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547a92bdb2db4116a4ea03e257a52f96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.100665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:07.102367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.103124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.103522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] PATH_INFO: `/v3/projects/fb24a60f519e4ec38579b42bd14d81fd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.106074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb24a60f519e4ec38579b42bd14d81fd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:07.108154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:07.112065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547a92bdb2db4116a4ea03e257a52f96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.113160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547a92bdb2db4116a4ea03e257a52f96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.136525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c568c933-19e9-4c18-8db7-c266a2427961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547a92bdb2db4116a4ea03e257a52f96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.149863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:07.162171 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3197/6381] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:06 2022] DELETE /identity/v3/users/547a92bdb2db4116a4ea03e257a52f96 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:07.163932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.164442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.164767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.206767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:07.206767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:07.257709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb24a60f519e4ec38579b42bd14d81fd', 'request_id': 'req-c271231a-a509-4a35-bcf0-8309fa1460d9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.268355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb24a60f519e4ec38579b42bd14d81fd', 'request_id': 'req-c271231a-a509-4a35-bcf0-8309fa1460d9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.279660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb24a60f519e4ec38579b42bd14d81fd', 'request_id': 'req-c271231a-a509-4a35-bcf0-8309fa1460d9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.296683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] The token cache is being invalidate because project fb24a60f519e4ec38579b42bd14d81fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:07.299410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.300706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.307353 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3185/6382] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:06 2022] DELETE /identity/v3/projects/fb24a60f519e4ec38579b42bd14d81fd => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:07.355647 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3198/6383] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:07.402465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:07.404721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:07.751792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271231a-a509-4a35-bcf0-8309fa1460d9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:07.756345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cb0a7-4ade-4152-9b50-9e370e02210d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:07.757116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb0a7-4ade-4152-9b50-9e370e02210d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.757511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb0a7-4ade-4152-9b50-9e370e02210d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.757846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb0a7-4ade-4152-9b50-9e370e02210d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.950850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cb0a7-4ade-4152-9b50-9e370e02210d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:07.952494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cb0a7-4ade-4152-9b50-9e370e02210d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:07.969989 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3186/6384] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:10.109358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.110774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:10.132857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9974c33-1219-4cb7-9209-db064f9e4af8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.148724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.149915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.150110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.150417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] PATH_INFO: `/v3/projects/0068cbc531084f16b79007a63f869fa6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.156345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0068cbc531084f16b79007a63f869fa6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.159249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.275602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0068cbc531084f16b79007a63f869fa6', 'request_id': 'req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.284003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0068cbc531084f16b79007a63f869fa6', 'request_id': 'req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.307547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0068cbc531084f16b79007a63f869fa6', 'request_id': 'req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.339602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] The token cache is being invalidate because project 0068cbc531084f16b79007a63f869fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:10.340180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.341593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ce5bf36b-393e-4f1b-8272-0c2eef80a38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.349787 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3199/6385] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:10 2022] DELETE /identity/v3/projects/0068cbc531084f16b79007a63f869fa6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:10.351476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a41a8-cc33-42bc-b34a-7b2a328d267a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.351984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a41a8-cc33-42bc-b34a-7b2a328d267a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.352344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a41a8-cc33-42bc-b34a-7b2a328d267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.404884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-849a41a8-cc33-42bc-b34a-7b2a328d267a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:10.410832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-849a41a8-cc33-42bc-b34a-7b2a328d267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:10.455798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:10.556982 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3187/6386] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:10.564963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c464219-e16d-4385-877e-ccf010ad2952 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.565316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c464219-e16d-4385-877e-ccf010ad2952 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.565733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c464219-e16d-4385-877e-ccf010ad2952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.614068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9c464219-e16d-4385-877e-ccf010ad2952 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:10.622241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c464219-e16d-4385-877e-ccf010ad2952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:10.651429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3200/6387] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:10.660721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a41a8-cc33-42bc-b34a-7b2a328d267a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.675236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a41a8-cc33-42bc-b34a-7b2a328d267a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.692564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.692564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.692564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.692564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.693095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:10.693095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.693095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.721165 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3188/6388] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:10.723041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c464219-e16d-4385-877e-ccf010ad2952 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.758354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c464219-e16d-4385-877e-ccf010ad2952 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.790544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.791191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.791471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.791748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.792378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.807067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.893775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3201/6389] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/projects => generated 351 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:10.897983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.926395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233534-22dc-4eec-b9bc-dea495e0df2e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.141984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.144766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:11.238218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.239252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.239541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.239819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.242229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.249316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.306903 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3189/6390] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/users => generated 276 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:11.307640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:11.308540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.335621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b663d-fcaa-4fa3-be1e-2f1ec3fafc80 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.346890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11531c-0936-48a4-89c6-994673559beb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.348661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.349348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.351674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.351957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] PATH_INFO: `/v3/users/0e63bded930641d8a15958628cf9a7aa` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.353695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e63bded930641d8a15958628cf9a7aa)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.355679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.362092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.363832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.364118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.364389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.366207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.368293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.387361 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3190/6391] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:11.388706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.430158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53987236-307f-45cf-aa32-ff4070f73fc7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.438093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.438757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.439081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.439368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.440051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.442533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.461948 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3191/6392] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:11.463008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.465344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e63bded930641d8a15958628cf9a7aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:11.466622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e63bded930641d8a15958628cf9a7aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:11.478069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf45b1-02ba-4269-9e79-0cc4ac3c5fbf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.483437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e63bded930641d8a15958628cf9a7aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:11.484068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e63bded930641d8a15958628cf9a7aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:11.491113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dd142-2864-4de2-829c-9a5f34ecd715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.492461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dd142-2864-4de2-829c-9a5f34ecd715 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.496017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dd142-2864-4de2-829c-9a5f34ecd715 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.496324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dd142-2864-4de2-829c-9a5f34ecd715 admin admin] PATH_INFO: `/v3/projects/bb8a8c3cb8cf4380add19163afd9c4c6/users/a10da30b69dd41cd9558714d9509564f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.512201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3df39f98-42e7-47cb-972f-4a67be68e13e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e63bded930641d8a15958628cf9a7aa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:11.545110 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3202/6393] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:11 2022] DELETE /identity/v3/users/0e63bded930641d8a15958628cf9a7aa => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:11.548844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dd142-2864-4de2-829c-9a5f34ecd715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb8a8c3cb8cf4380add19163afd9c4c6, user_id=a10da30b69dd41cd9558714d9509564f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.549944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.550465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.550948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.551972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dd142-2864-4de2-829c-9a5f34ecd715 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.595920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:11.598606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:11.608987 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3192/6394] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:11 2022] PUT /identity/v3/projects/bb8a8c3cb8cf4380add19163afd9c4c6/users/a10da30b69dd41cd9558714d9509564f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:11.611608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.611975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.612293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.705173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] MFA Rules not processed for user `a10da30b69dd41cd9558714d9509564f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:11.708547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:11.765162 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3203/6395] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:11.775906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:11.788645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.789999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:11.917045 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3193/6396] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:11.932491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.934059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:11.974277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe9aab-b306-4571-8214-23d5ce92eede None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.983806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.985374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.986668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.987103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:12.007398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:12.010124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:12.033347 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3204/6397] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:12.045510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963871d3-78ea-47fa-834a-a721633de6d5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:12.049504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22648f-bbff-4d8c-b7ed-4a6ee96de166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:12.050374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22648f-bbff-4d8c-b7ed-4a6ee96de166 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:12.050754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22648f-bbff-4d8c-b7ed-4a6ee96de166 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:12.051252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22648f-bbff-4d8c-b7ed-4a6ee96de166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:12.064156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22648f-bbff-4d8c-b7ed-4a6ee96de166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:12.065928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22648f-bbff-4d8c-b7ed-4a6ee96de166 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:12.076934 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3194/6398] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:13.156105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:13.161079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:13.179189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:13.197597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97adbd-1f6a-4e34-ba96-61e764601677 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:13.204589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:13.205791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:13.206346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:13.206853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] PATH_INFO: `/v3/projects/dbeef98f1fb44c26a13741e637a819d7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:13.211766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbeef98f1fb44c26a13741e637a819d7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:13.217849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:13.293315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbeef98f1fb44c26a13741e637a819d7', 'request_id': 'req-818f5eb6-5bad-4ce0-b219-6b43ddd75474'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:13.300336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbeef98f1fb44c26a13741e637a819d7', 'request_id': 'req-818f5eb6-5bad-4ce0-b219-6b43ddd75474'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:13.323233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbeef98f1fb44c26a13741e637a819d7', 'request_id': 'req-818f5eb6-5bad-4ce0-b219-6b43ddd75474'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:13.339761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] The token cache is being invalidate because project dbeef98f1fb44c26a13741e637a819d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:13.340733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:13.341052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-818f5eb6-5bad-4ce0-b219-6b43ddd75474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:13.343836 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3205/6399] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:13 2022] DELETE /identity/v3/projects/dbeef98f1fb44c26a13741e637a819d7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:13.347817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719ff5-a727-4c66-a025-5e203b285152 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:13.348680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719ff5-a727-4c66-a025-5e203b285152 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:13.349299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719ff5-a727-4c66-a025-5e203b285152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:13.381429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-52719ff5-a727-4c66-a025-5e203b285152 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:13.383343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52719ff5-a727-4c66-a025-5e203b285152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:13.488064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3195/6400] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:13.492695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e5b78-473a-4578-b062-99e529aad4b0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:13.493888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e5b78-473a-4578-b062-99e529aad4b0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:13.494795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e5b78-473a-4578-b062-99e529aad4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:13.538929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2e8e5b78-473a-4578-b062-99e529aad4b0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:13.542636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2e8e5b78-473a-4578-b062-99e529aad4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:13.571107 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3206/6401] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:13.575568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719ff5-a727-4c66-a025-5e203b285152 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:13.600513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719ff5-a727-4c66-a025-5e203b285152 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:13.608478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:13.610187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:13.610669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:13.611507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:13.613927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:13.615500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:13.620811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:13.653972 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3196/6402] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:13.655829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e5b78-473a-4578-b062-99e529aad4b0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:13.671838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e5b78-473a-4578-b062-99e529aad4b0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:13.696376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:13.697718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:13.697718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:13.697718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:13.698304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:13.700235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:13.767654 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3207/6403] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:13 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:13.769613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:13.786045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73719b47-5fc7-4a2c-b57d-17cc1fb1199c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:13.924772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:13.938258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:13.940329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:13.941037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:13.941316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:13.944148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:13.948397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:14.008140 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3197/6404] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:13 2022] POST /identity/v3/users => generated 288 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:14.012527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:14.025945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c17e36-b154-4024-9bd8-436efa9b0f9d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:14.035858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:14.037429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:14.038721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:14.039292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:14.040991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:14.044629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:14.060592 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3208/6405] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:14.066011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:14.084505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6388-bf01-4813-a048-6f12fc2b30c7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:14.118772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359850e5-e0ec-4472-8a3f-d10bf0bb54ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:14.127596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359850e5-e0ec-4472-8a3f-d10bf0bb54ba admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:14.129571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359850e5-e0ec-4472-8a3f-d10bf0bb54ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:14.129948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359850e5-e0ec-4472-8a3f-d10bf0bb54ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:14.131749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359850e5-e0ec-4472-8a3f-d10bf0bb54ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:14.133923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359850e5-e0ec-4472-8a3f-d10bf0bb54ba admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:14.165983 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3198/6406] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:14.169506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:14.183507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920eaa3b-a954-4f72-9eb7-726fd5903f12 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:14.195585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb97173-3b80-4fc4-b572-a682c0d837cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:14.196400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb97173-3b80-4fc4-b572-a682c0d837cd admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:14.196737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb97173-3b80-4fc4-b572-a682c0d837cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:14.197011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb97173-3b80-4fc4-b572-a682c0d837cd admin admin] PATH_INFO: `/v3/projects/a7cfffdc05394baaa8a064b5570772c6/users/17bdd318ee7a4f6ba3092804c2ff40bf/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:14.290271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb97173-3b80-4fc4-b572-a682c0d837cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7cfffdc05394baaa8a064b5570772c6, user_id=17bdd318ee7a4f6ba3092804c2ff40bf, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:14.293049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb97173-3b80-4fc4-b572-a682c0d837cd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:14.335453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbb9b9-35cc-45d2-90d2-10444f88635d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:14.337279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbb9b9-35cc-45d2-90d2-10444f88635d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:14.343430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbb9b9-35cc-45d2-90d2-10444f88635d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:14.348466 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3209/6407] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:14 2022] PUT /identity/v3/projects/a7cfffdc05394baaa8a064b5570772c6/users/17bdd318ee7a4f6ba3092804c2ff40bf/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:14.365154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df1c7c-73a1-4c28-8263-9fe4d9c42597 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:14.365515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df1c7c-73a1-4c28-8263-9fe4d9c42597 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:14.365831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df1c7c-73a1-4c28-8263-9fe4d9c42597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:14.426418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b5df1c7c-73a1-4c28-8263-9fe4d9c42597 None None] MFA Rules not processed for user `17bdd318ee7a4f6ba3092804c2ff40bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:14.427803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b5df1c7c-73a1-4c28-8263-9fe4d9c42597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:14.448812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-55bbb9b9-35cc-45d2-90d2-10444f88635d None None] MFA Rules not processed for user `753144834d1a418897a600589137fcbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:14.453158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-55bbb9b9-35cc-45d2-90d2-10444f88635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:14.562113 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3210/6408] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:14.563542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df1c7c-73a1-4c28-8263-9fe4d9c42597 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:14.584470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:14.628834 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3199/6409] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:14.638502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:14.638939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:14.639319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:14.681216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] MFA Rules not processed for user `753144834d1a418897a600589137fcbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:14.682346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:14.701426 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3200/6410] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:14.736682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:14.738463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:14.750485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df1c7c-73a1-4c28-8263-9fe4d9c42597 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:14.771537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:14.773520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:14.774004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:14.774470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:14.775552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:14.778121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:14.846254 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3211/6411] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:14 2022] POST /identity/v3/projects => generated 375 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:14.851518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:14.969228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502ee2c-de89-40a4-8e35-6b491fbaf34e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.009066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cdda-b99f-40ee-a69c-98045c112522 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.024317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.025508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.026354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.026800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.027628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.030814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.086092 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3212/6412] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:14 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:15.091752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.097825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:15.131874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77f893-7e2f-493d-bafa-a1a654e65f27 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.144367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.147503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.148145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.148679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.149116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.150548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.151032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.151472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.152338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.154835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.185121 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3213/6413] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:15.188646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.212903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27617897-dfdb-4fcb-8938-9ec66af4a9c7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.218442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.219133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.219409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.219739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] PATH_INFO: `/v3/projects/031ce56ae1a74294bd34e40a75ec828c/users/9c1967f692f9425d8dbcf9c1cf395036/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.248623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031ce56ae1a74294bd34e40a75ec828c, user_id=9c1967f692f9425d8dbcf9c1cf395036, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.252197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.295437 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3214/6414] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:15 2022] PUT /identity/v3/projects/031ce56ae1a74294bd34e40a75ec828c/users/9c1967f692f9425d8dbcf9c1cf395036/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:15.299556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.371613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.374225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.394187 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3201/6415] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:15.443401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:15.473833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c38fb-5384-4a3d-93a6-54605f79c1c6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.488489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b56e4-cb16-44a9-8fc7-eeda46d239e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.489238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b56e4-cb16-44a9-8fc7-eeda46d239e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.489638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b56e4-cb16-44a9-8fc7-eeda46d239e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.490308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b56e4-cb16-44a9-8fc7-eeda46d239e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.491159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b56e4-cb16-44a9-8fc7-eeda46d239e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.497099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b56e4-cb16-44a9-8fc7-eeda46d239e7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.524466 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3215/6416] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:15.529032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.547771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd54383-8fa6-4615-81c3-299510b07d40 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.559750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.560598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.560880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.561154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] PATH_INFO: `/v3/projects/031ce56ae1a74294bd34e40a75ec828c/users/9c1967f692f9425d8dbcf9c1cf395036/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.567556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=031ce56ae1a74294bd34e40a75ec828c, user_id=9c1967f692f9425d8dbcf9c1cf395036, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.570664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.610664 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3202/6417] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:15 2022] PUT /identity/v3/projects/031ce56ae1a74294bd34e40a75ec828c/users/9c1967f692f9425d8dbcf9c1cf395036/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:15.613078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.613783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.614388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.676239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] MFA Rules not processed for user `9c1967f692f9425d8dbcf9c1cf395036`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:15.677920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:15.819258 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3216/6418] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:15.835548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.838594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:16.001223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20edfc9-a785-46f1-902b-673e19bb7f3f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:16.007100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b073c-0288-408f-a836-526643c54e76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:16.007840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b073c-0288-408f-a836-526643c54e76 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:16.008313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b073c-0288-408f-a836-526643c54e76 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:16.008685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b073c-0288-408f-a836-526643c54e76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:16.179844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b073c-0288-408f-a836-526643c54e76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:16.182124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b073c-0288-408f-a836-526643c54e76 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:16.195371 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3203/6419] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:17.060999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:17.062518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:17.302006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530a41b-4da5-4782-963c-c731ff2b9170 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:17.311027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:17.311705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:17.312225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:17.312354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] PATH_INFO: `/v3/users/a10da30b69dd41cd9558714d9509564f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:17.313654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10da30b69dd41cd9558714d9509564f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:17.315397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:17.480955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10da30b69dd41cd9558714d9509564f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:17.483403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10da30b69dd41cd9558714d9509564f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:17.511326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10da30b69dd41cd9558714d9509564f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:17.511978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10da30b69dd41cd9558714d9509564f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:17.543461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10da30b69dd41cd9558714d9509564f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:17.558755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:17.591961 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3217/6420] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:17 2022] DELETE /identity/v3/users/a10da30b69dd41cd9558714d9509564f => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:17.596891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:17.597823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:17.598536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:17.662659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:17.667633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:17.719602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:17.833673 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3204/6421] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:17.881114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:17.882658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:18.102127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b3127-4d6c-46ca-a7b2-c2a1e52978f1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:18.107678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:18.108628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:18.109293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:18.110213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:18.131333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:18.136192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:18.156833 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3218/6422] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:19.306361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:19.309757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:19.329778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71b45-6cde-4da6-96bf-dc88be15cb03 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:19.347133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:19.347797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:19.348120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:19.348399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] PATH_INFO: `/v3/projects/bb8a8c3cb8cf4380add19163afd9c4c6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:19.350736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb8a8c3cb8cf4380add19163afd9c4c6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:19.353120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:19.473182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb8a8c3cb8cf4380add19163afd9c4c6', 'request_id': 'req-084850e6-1d22-454a-ab86-c13d33073185'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.485621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb8a8c3cb8cf4380add19163afd9c4c6', 'request_id': 'req-084850e6-1d22-454a-ab86-c13d33073185'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.488588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:19.492485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:19.522309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9e511-f152-487f-b32a-d81923b4af9b service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:19.527182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb8a8c3cb8cf4380add19163afd9c4c6', 'request_id': 'req-084850e6-1d22-454a-ab86-c13d33073185'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.564529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] The token cache is being invalidate because project bb8a8c3cb8cf4380add19163afd9c4c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:19.566452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.570488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-084850e6-1d22-454a-ab86-c13d33073185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.578829 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3205/6423] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:19 2022] DELETE /identity/v3/projects/bb8a8c3cb8cf4380add19163afd9c4c6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:19.583603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693d565-dd90-41a4-b817-6b50ab49c5f6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:19.583889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693d565-dd90-41a4-b817-6b50ab49c5f6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:19.584164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693d565-dd90-41a4-b817-6b50ab49c5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:19.646568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:19.647739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:19.648548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:19.649091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] PATH_INFO: `/v3/users/17bdd318ee7a4f6ba3092804c2ff40bf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:19.651009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=17bdd318ee7a4f6ba3092804c2ff40bf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:19.654846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:19.700944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3693d565-dd90-41a4-b817-6b50ab49c5f6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:19.707073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3693d565-dd90-41a4-b817-6b50ab49c5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:19.724269 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3206/6424] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:19.732967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bae0bb-3597-4902-91fa-0646d3262f4d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:19.733247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bae0bb-3597-4902-91fa-0646d3262f4d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:19.733787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bae0bb-3597-4902-91fa-0646d3262f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:19.768015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-20bae0bb-3597-4902-91fa-0646d3262f4d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:19.770450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-20bae0bb-3597-4902-91fa-0646d3262f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:19.783157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17bdd318ee7a4f6ba3092804c2ff40bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.784871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17bdd318ee7a4f6ba3092804c2ff40bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.791208 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3207/6425] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:19.800583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bae0bb-3597-4902-91fa-0646d3262f4d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:19.811617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17bdd318ee7a4f6ba3092804c2ff40bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.812808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17bdd318ee7a4f6ba3092804c2ff40bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.816990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bae0bb-3597-4902-91fa-0646d3262f4d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:19.839789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf9672-4c87-4220-a7fc-7a317384b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:19.840475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf9672-4c87-4220-a7fc-7a317384b92e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:19.840761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf9672-4c87-4220-a7fc-7a317384b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:19.841046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf9672-4c87-4220-a7fc-7a317384b92e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:19.841799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf9672-4c87-4220-a7fc-7a317384b92e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:19.842108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf9672-4c87-4220-a7fc-7a317384b92e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:19.843297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17bdd318ee7a4f6ba3092804c2ff40bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:19.843816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf9672-4c87-4220-a7fc-7a317384b92e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:19.870676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3208/6426] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:19.871118 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3219/6427] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:19 2022] DELETE /identity/v3/users/17bdd318ee7a4f6ba3092804c2ff40bf => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:19.877628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:19.879653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:19.880720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:19.884603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:19.886393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:19.913410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:19.939738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:19.943233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:20.090720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:20.102749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1c998-19b3-425d-9fdd-5a808d896aaf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:20.117354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:20.118186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.118633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.119002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.119748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:20.122507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:20.125906 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3209/6428] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:20.184233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:20.186238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:20.216430 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3220/6429] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:19 2022] POST /identity/v3/projects => generated 365 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:20.220305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:20.352310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85d1f6-c2e5-4d14-93b8-f309d03cba05 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:20.361715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe05b5-e4b9-4b97-9099-3530a9f627a0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:20.366465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:20.367116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.367392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.367668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.368376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:20.370375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:20.425543 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3221/6430] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:20 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:20.429268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:20.447275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:20.447904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.448181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.448450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.448829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d5d06-9fe4-4d37-bf66-174e90230691 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:20.464501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:20.466180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:20.466376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:20.466938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.467221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.467496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.468159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:20.469819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:20.483706 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3210/6431] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:20.491841 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3222/6432] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:20.495790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:20.508271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e08d-ce0e-45c6-ab5c-769d6f6a34c7 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:20.516766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848a5ef-bf2e-4693-a781-f617b39d110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:20.517754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848a5ef-bf2e-4693-a781-f617b39d110d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.518104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848a5ef-bf2e-4693-a781-f617b39d110d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.518493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848a5ef-bf2e-4693-a781-f617b39d110d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.519259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848a5ef-bf2e-4693-a781-f617b39d110d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:20.521130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848a5ef-bf2e-4693-a781-f617b39d110d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:20.542565 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3211/6433] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:20.546467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:20.559623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56347ab2-6edd-49d0-91fe-8cb9966d71f3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:20.572044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:20.572956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.573569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.574036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] PATH_INFO: `/v3/projects/bf380d9339814239b9e1078b99c4905a/users/9b96d4611d66470082a5574fde431b3c/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.603844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf380d9339814239b9e1078b99c4905a, user_id=9b96d4611d66470082a5574fde431b3c, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:20.607112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:20.659873 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3223/6434] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:20 2022] PUT /identity/v3/projects/bf380d9339814239b9e1078b99c4905a/users/9b96d4611d66470082a5574fde431b3c/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:20.660085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cd917-f345-47d4-92f8-6ae7ddafb7e1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.660424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cd917-f345-47d4-92f8-6ae7ddafb7e1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.660771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cd917-f345-47d4-92f8-6ae7ddafb7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.735800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-219cd917-f345-47d4-92f8-6ae7ddafb7e1 None None] MFA Rules not processed for user `9b96d4611d66470082a5574fde431b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:20.737216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-219cd917-f345-47d4-92f8-6ae7ddafb7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:20.855181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3212/6435] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:20.855601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.004869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:21.046606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc4fb6-d5c6-42dd-9032-23a72dd102ea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.058635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.059432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.062244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.062655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.063936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.065929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.132173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3224/6436] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:20 2022] POST /identity/v3/projects => generated 365 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:21.136029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cd917-f345-47d4-92f8-6ae7ddafb7e1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.151112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cd917-f345-47d4-92f8-6ae7ddafb7e1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.225390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:21.308197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.309047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.309415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.309883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.311496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.313724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.369186 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3213/6437] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:21 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:21.374325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.389709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec36c7-0d69-4f08-86e7-d7c267394890 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.404709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.405131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.405415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.406326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.407076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.408814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.432458 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3225/6438] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:21.437320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.457176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9121f-e592-4af4-a731-d6199e0caa67 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.474817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.475990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.476371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.476528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] PATH_INFO: `/v3/projects/1cfaa56865c040638694461bad7a1f5e/users/e52b3c140ba5428cb541914c4c61689f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.512020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cfaa56865c040638694461bad7a1f5e, user_id=e52b3c140ba5428cb541914c4c61689f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.516678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.584780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.585436 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3214/6439] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:21 2022] PUT /identity/v3/projects/1cfaa56865c040638694461bad7a1f5e/users/e52b3c140ba5428cb541914c4c61689f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:21.603382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699e4a-87bb-4867-8ea6-8b17dadd0fe7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.698076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.698798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.699151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.699494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.700223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.704612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.714402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.716483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:21.741142 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3226/6440] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:21.744117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.763975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c5953-3f71-4bf0-8834-9675d58a9e54 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.778923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.779586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.779861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.780129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] PATH_INFO: `/v3/projects/a7cfffdc05394baaa8a064b5570772c6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.782572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7cfffdc05394baaa8a064b5570772c6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.784520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.788683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a67652-2868-4579-9ae7-61026957e3c3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.799344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35c4b7-dab7-470b-acd9-5da5728ee31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.800217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35c4b7-dab7-470b-acd9-5da5728ee31e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.800491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35c4b7-dab7-470b-acd9-5da5728ee31e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.800759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35c4b7-dab7-470b-acd9-5da5728ee31e admin admin] PATH_INFO: `/v3/projects/1cfaa56865c040638694461bad7a1f5e/users/e52b3c140ba5428cb541914c4c61689f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.804494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35c4b7-dab7-470b-acd9-5da5728ee31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cfaa56865c040638694461bad7a1f5e, user_id=e52b3c140ba5428cb541914c4c61689f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.807393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35c4b7-dab7-470b-acd9-5da5728ee31e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.845923 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3227/6441] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:21 2022] PUT /identity/v3/projects/1cfaa56865c040638694461bad7a1f5e/users/e52b3c140ba5428cb541914c4c61689f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:21.856240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.856516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.856786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.918237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7cfffdc05394baaa8a064b5570772c6', 'request_id': 'req-1eae58d0-f1f2-460c-8089-0c14e00cca50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:21.926689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] MFA Rules not processed for user `e52b3c140ba5428cb541914c4c61689f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:21.928207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:21.941624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7cfffdc05394baaa8a064b5570772c6', 'request_id': 'req-1eae58d0-f1f2-460c-8089-0c14e00cca50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:21.948280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7cfffdc05394baaa8a064b5570772c6', 'request_id': 'req-1eae58d0-f1f2-460c-8089-0c14e00cca50'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:21.988322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] The token cache is being invalidate because project a7cfffdc05394baaa8a064b5570772c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:21.988659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:21.992127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:21.992696 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3215/6442] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:21 2022] DELETE /identity/v3/projects/a7cfffdc05394baaa8a064b5570772c6 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:21.997098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:22.005215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:22.130862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3228/6443] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:22.143354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:22.144824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:22.214822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae58d0-f1f2-460c-8089-0c14e00cca50 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:22.231449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:22.233082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:22.236535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:22.236948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] PATH_INFO: `/v3/users/9c1967f692f9425d8dbcf9c1cf395036` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:22.238197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:22.241979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1967f692f9425d8dbcf9c1cf395036)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:22.250818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:22.302129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:22.335733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038a2d-5c7d-4e4a-9bcd-12b75ba3f664 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:22.345255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:22.347622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:22.347886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:22.348422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:22.373059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1967f692f9425d8dbcf9c1cf395036'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:22.379970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:22.385210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:22.406140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1967f692f9425d8dbcf9c1cf395036'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:22.416637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3229/6444] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:22.433902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1967f692f9425d8dbcf9c1cf395036'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:22.434679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1967f692f9425d8dbcf9c1cf395036'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:22.435653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1967f692f9425d8dbcf9c1cf395036'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:22.463015 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3216/6445] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:21 2022] DELETE /identity/v3/users/9c1967f692f9425d8dbcf9c1cf395036 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:22.549667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:22.573492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf9ec7-4253-44a5-a72b-82f8673176ca service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:22.713979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4e908-95b2-4cfa-9f90-cbc990ee9eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:22.717586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e908-95b2-4cfa-9f90-cbc990ee9eae service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:22.718833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e908-95b2-4cfa-9f90-cbc990ee9eae service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:22.719949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e908-95b2-4cfa-9f90-cbc990ee9eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:22.722902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3f4e908-95b2-4cfa-9f90-cbc990ee9eae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:22.833429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4e908-95b2-4cfa-9f90-cbc990ee9eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:22.835300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4e908-95b2-4cfa-9f90-cbc990ee9eae service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:22.857141 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3230/6446] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:23.347173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:23.350229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:23.490623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dd86a-11e9-4b85-8d84-e169e73e5d22 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:23.498209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:23.499031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:23.499374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:23.499712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] PATH_INFO: `/v3/projects/031ce56ae1a74294bd34e40a75ec828c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:23.502581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031ce56ae1a74294bd34e40a75ec828c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:23.504695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:23.586488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031ce56ae1a74294bd34e40a75ec828c', 'request_id': 'req-b290a1e6-0aec-4275-aa0d-9e289f380833'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:23.618953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031ce56ae1a74294bd34e40a75ec828c', 'request_id': 'req-b290a1e6-0aec-4275-aa0d-9e289f380833'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:23.644324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031ce56ae1a74294bd34e40a75ec828c', 'request_id': 'req-b290a1e6-0aec-4275-aa0d-9e289f380833'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:23.664508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] The token cache is being invalidate because project 031ce56ae1a74294bd34e40a75ec828c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:23.665006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:23.666083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b290a1e6-0aec-4275-aa0d-9e289f380833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:23.668220 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3217/6447] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:23 2022] DELETE /identity/v3/projects/031ce56ae1a74294bd34e40a75ec828c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:23.672990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f115d-ad44-4419-b882-807cc11d2e28 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:23.673213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f115d-ad44-4419-b882-807cc11d2e28 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:23.676212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f115d-ad44-4419-b882-807cc11d2e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:23.727516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b52f115d-ad44-4419-b882-807cc11d2e28 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:23.729368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b52f115d-ad44-4419-b882-807cc11d2e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:23.871950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff33457-f2bb-4de7-9969-f452d5d671f7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:23.872306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff33457-f2bb-4de7-9969-f452d5d671f7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:23.872640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff33457-f2bb-4de7-9969-f452d5d671f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:23.878417 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3231/6448] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:23.902964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0ff33457-f2bb-4de7-9969-f452d5d671f7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:23.904656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ff33457-f2bb-4de7-9969-f452d5d671f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:23.919656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3218/6449] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:23.924143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f115d-ad44-4419-b882-807cc11d2e28 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:23.935298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f115d-ad44-4419-b882-807cc11d2e28 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:23.940445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:23.941438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:23.941745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:23.942012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:23.942670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:23.942967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:23.944559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:23.961067 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3232/6450] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:23.962999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff33457-f2bb-4de7-9969-f452d5d671f7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:23.974057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff33457-f2bb-4de7-9969-f452d5d671f7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:23.980161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:23.981068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:23.981406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:23.981886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:23.982727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:23.984693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:24.004375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:24.027032 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3219/6451] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:23 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:24.028075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:24.048529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ca4e9-d445-4378-9f95-13827a0ddd4b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:24.060711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:24.128791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:24.129690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:24.129690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:24.130069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:24.130720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:24.133070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:24.179358 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3233/6452] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:24 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:24.183698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:24.197470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fede50b-3682-48c2-9d68-38762c66fdba admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:24.208265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:24.208895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:24.209175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:24.209447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:24.210134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:24.211782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:24.228451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:24.239454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f3781-d8c2-45e7-bff7-0303cbc56557 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:24.240076 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3220/6453] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:24.244834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386727b-8828-4289-8dc3-26739bf702fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:24.245487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386727b-8828-4289-8dc3-26739bf702fa admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:24.245793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386727b-8828-4289-8dc3-26739bf702fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:24.246065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386727b-8828-4289-8dc3-26739bf702fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:24.247076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386727b-8828-4289-8dc3-26739bf702fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:24.249011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386727b-8828-4289-8dc3-26739bf702fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:24.262234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3234/6454] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:24.266585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:24.282875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461f9b-1b39-49c6-9a98-3c2202d03672 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:24.296352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:24.297048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:24.297329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:24.297648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] PATH_INFO: `/v3/projects/d485f1219d7f471cba8ebface38c8cec/users/5f861462391648188d43f16bd9348979/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:24.327005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d485f1219d7f471cba8ebface38c8cec, user_id=5f861462391648188d43f16bd9348979, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:24.332415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:24.366797 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3221/6455] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:24 2022] PUT /identity/v3/projects/d485f1219d7f471cba8ebface38c8cec/users/5f861462391648188d43f16bd9348979/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:24.377614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affac80-9672-4b99-b61f-487b46a619ee None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:24.377893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affac80-9672-4b99-b61f-487b46a619ee None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:24.378163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affac80-9672-4b99-b61f-487b46a619ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:24.458213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7affac80-9672-4b99-b61f-487b46a619ee None None] MFA Rules not processed for user `5f861462391648188d43f16bd9348979`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:24.458213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7affac80-9672-4b99-b61f-487b46a619ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:24.637686 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3235/6456] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:24.641957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:24.788386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb4236-b3e1-4fc6-ae99-be5ce8b9f1ca admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:24.798946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:24.800786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:24.801525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:24.802991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:24.804416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:24.819057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:24.930804 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3222/6457] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:24 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:24.934474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affac80-9672-4b99-b61f-487b46a619ee None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:24.948882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affac80-9672-4b99-b61f-487b46a619ee None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:25.134619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:25.134619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:25.134619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:25.134619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:25.135567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:25.135567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:25.235731 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3236/6458] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:24 2022] POST /identity/v3/users => generated 292 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:25.240020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:25.263081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed725a0-b784-4b0b-b8cf-34c680050800 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:25.290635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:25.291283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:25.291566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:25.291846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:25.292802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:25.294183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:25.327236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3223/6459] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:25.330873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:25.364575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357238f3-ae2b-42d2-ae21-a8291a354a24 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:25.376292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d0af2-9581-40b9-9bc0-b83c28c5cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:25.376930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d0af2-9581-40b9-9bc0-b83c28c5cd6c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:25.377210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d0af2-9581-40b9-9bc0-b83c28c5cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:25.380084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d0af2-9581-40b9-9bc0-b83c28c5cd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:25.380858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d0af2-9581-40b9-9bc0-b83c28c5cd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:25.382554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d0af2-9581-40b9-9bc0-b83c28c5cd6c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:25.387382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:25.425201 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3237/6460] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:25.429777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:25.466458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fbc59-6601-4a93-b908-3f8b5a995e98 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:25.482610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:25.484907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:25.485325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:25.493347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] PATH_INFO: `/v3/projects/07b56059667641feac379a67ec2e3d5f/users/2c6a5e73b1204e19a3924dff9c5ea88e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:25.543472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b56059667641feac379a67ec2e3d5f, user_id=2c6a5e73b1204e19a3924dff9c5ea88e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:25.546730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:25.575395 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3224/6461] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:25 2022] PUT /identity/v3/projects/07b56059667641feac379a67ec2e3d5f/users/2c6a5e73b1204e19a3924dff9c5ea88e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:25.586962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525a32-0a2b-44cf-91f1-d064c89c1953 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:25.587775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525a32-0a2b-44cf-91f1-d064c89c1953 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:25.588257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525a32-0a2b-44cf-91f1-d064c89c1953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:25.640829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4a525a32-0a2b-44cf-91f1-d064c89c1953 None None] MFA Rules not processed for user `2c6a5e73b1204e19a3924dff9c5ea88e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:25.642466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a525a32-0a2b-44cf-91f1-d064c89c1953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:25.833506 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3238/6462] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:25.837995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:26.008115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c0b83-011f-4202-bce0-52f5923d7da9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:26.020413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:26.021311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:26.021631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:26.021902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:26.022546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:26.024617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:26.060073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:26.108174 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3225/6463] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:25 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:26.111054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a525a32-0a2b-44cf-91f1-d064c89c1953 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:26.128931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a525a32-0a2b-44cf-91f1-d064c89c1953 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:26.253950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:26.256945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:26.257297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:26.257679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:26.258549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:26.262001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:26.313861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3239/6464] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:26 2022] POST /identity/v3/users => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:26.318011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:26.333246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d02f1e-0151-4bbc-8d21-f09fc0b168b9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:26.349670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:26.350313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:26.350595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:26.350865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:26.351631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:26.369337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:26.403603 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3226/6465] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:26.407855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:26.424173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84f3eb-5e25-4071-9c2a-f51183f79b00 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:26.433993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:26.435132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:26.436809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:26.437130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] PATH_INFO: `/v3/projects/5179ac2aa1324664980bdca51d02137c/users/0180cb0f20e54d00928f651e63be0076/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:26.474428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5179ac2aa1324664980bdca51d02137c, user_id=0180cb0f20e54d00928f651e63be0076, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:26.481961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:26.516278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:26.516722 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3240/6466] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:26 2022] PUT /identity/v3/projects/5179ac2aa1324664980bdca51d02137c/users/0180cb0f20e54d00928f651e63be0076/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:26.530147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f609785-c0bf-41f7-af0f-0d5d58485adc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:26.718975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325ce49-4eac-4cb9-81fe-7660e5ff839b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:26.719698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325ce49-4eac-4cb9-81fe-7660e5ff839b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:26.719976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325ce49-4eac-4cb9-81fe-7660e5ff839b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:26.720310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325ce49-4eac-4cb9-81fe-7660e5ff839b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:26.721022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325ce49-4eac-4cb9-81fe-7660e5ff839b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:26.727984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325ce49-4eac-4cb9-81fe-7660e5ff839b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:26.749501 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3227/6467] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:26.753962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:26.771388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a538d-cb31-4f13-8e0f-7262e73f76a1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:26.795051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:26.796330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:26.796748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:26.797181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] PATH_INFO: `/v3/projects/5179ac2aa1324664980bdca51d02137c/users/0180cb0f20e54d00928f651e63be0076/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:26.803087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5179ac2aa1324664980bdca51d02137c, user_id=0180cb0f20e54d00928f651e63be0076, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:26.812719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:26.845395 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3241/6468] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:26 2022] PUT /identity/v3/projects/5179ac2aa1324664980bdca51d02137c/users/0180cb0f20e54d00928f651e63be0076/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:26.856234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:26.857131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:26.857650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:26.925644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] MFA Rules not processed for user `0180cb0f20e54d00928f651e63be0076`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:26.927166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:27.075514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3228/6469] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:27.084503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:27.087447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:27.228131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:27.270490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfd5b1-e15d-4a12-998d-f8d31e11c45d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:27.288416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a6ce3-b0b6-406e-b0ce-eb4bc0f7b300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:27.291313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a6ce3-b0b6-406e-b0ce-eb4bc0f7b300 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:27.291608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a6ce3-b0b6-406e-b0ce-eb4bc0f7b300 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:27.291887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a6ce3-b0b6-406e-b0ce-eb4bc0f7b300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:27.424129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a6ce3-b0b6-406e-b0ce-eb4bc0f7b300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:27.427834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a6ce3-b0b6-406e-b0ce-eb4bc0f7b300 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:27.448158 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3242/6470] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:29.624473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:29.626345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:29.674143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c76aa-a7a4-4123-bb85-e8ae71f56d1b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:29.683957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:29.685703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:29.685995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:29.686279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:29.846315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:29.847952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:29.876229 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3229/6471] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:30.194465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978636b4-6649-4fa0-88a8-919b9cb91c39 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:30.195017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978636b4-6649-4fa0-88a8-919b9cb91c39 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:30.195462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978636b4-6649-4fa0-88a8-919b9cb91c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:30.258397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-978636b4-6649-4fa0-88a8-919b9cb91c39 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:30.259769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-978636b4-6649-4fa0-88a8-919b9cb91c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:30.397821 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3243/6472] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:30.478344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:30.479731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:30.486596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:30.626312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338e098-1354-45ff-ab83-9416d028a143 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:30.641188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:30.645724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:30.646447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:30.647891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:30.666539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:30.669898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:30.703323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3230/6473] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:15:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:15:31.815771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978636b4-6649-4fa0-88a8-919b9cb91c39 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:31.870767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978636b4-6649-4fa0-88a8-919b9cb91c39 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:31.882192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd419db-8196-47c6-8725-22d080d063ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:31.883091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd419db-8196-47c6-8725-22d080d063ff service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:31.883576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd419db-8196-47c6-8725-22d080d063ff service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:31.884082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd419db-8196-47c6-8725-22d080d063ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:31.905163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd419db-8196-47c6-8725-22d080d063ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:31.906855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd419db-8196-47c6-8725-22d080d063ff service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:31.931665 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3244/6474] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:32.259255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:32.261862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:32.438070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60893a1-21ee-4a48-ab18-444dafce078f service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:32.452691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:32.453358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:32.453713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:32.454146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] PATH_INFO: `/v3/users/c777e5af74e64cb3805ef28d1d0ed776` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:32.459881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c777e5af74e64cb3805ef28d1d0ed776)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:32.462313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:32.583302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c777e5af74e64cb3805ef28d1d0ed776'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:32.611843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c777e5af74e64cb3805ef28d1d0ed776'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:32.635319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c777e5af74e64cb3805ef28d1d0ed776'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:32.636203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c777e5af74e64cb3805ef28d1d0ed776'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:32.636957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c777e5af74e64cb3805ef28d1d0ed776'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:32.676330 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3231/6475] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:32 2022] DELETE /identity/v3/users/c777e5af74e64cb3805ef28d1d0ed776 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:32.677789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364534d-9196-49f0-8775-605c1c81f2cd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:32.678077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364534d-9196-49f0-8775-605c1c81f2cd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:32.678402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364534d-9196-49f0-8775-605c1c81f2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:32.750162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d364534d-9196-49f0-8775-605c1c81f2cd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:32.751411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d364534d-9196-49f0-8775-605c1c81f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:32.991819 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3245/6476] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:33.035158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:33.036813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:33.042731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:33.242349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe1c50-7ce0-4a13-9208-93cbfb497bc5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:33.249567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:33.250228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:33.250507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:33.250778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:33.268472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:33.272009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:33.300472 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3232/6477] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:33.471791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:33.472322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364534d-9196-49f0-8775-605c1c81f2cd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:33.487025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad2db6-b5eb-485d-95e9-3263263c1b07 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:33.496328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364534d-9196-49f0-8775-605c1c81f2cd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:33.500089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12e01c-0679-4659-a0d8-610b211c3cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:33.503351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12e01c-0679-4659-a0d8-610b211c3cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:33.503633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12e01c-0679-4659-a0d8-610b211c3cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:33.504011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:33.505472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:33.505597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12e01c-0679-4659-a0d8-610b211c3cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:33.506262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:33.506726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:33.509352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:33.514320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6d12e01c-0679-4659-a0d8-610b211c3cb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:33.531237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:33.630860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:33.632554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:33.656196 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3246/6478] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:33.823176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12e01c-0679-4659-a0d8-610b211c3cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:33.833304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12e01c-0679-4659-a0d8-610b211c3cb6 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:33.894891 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3233/6479] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:34.070599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.079563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.126903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43710a3-d0cc-4c5e-805a-e0f2f18c14b5 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.143245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.144081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.144504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.144803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] PATH_INFO: `/v3/users/9b96d4611d66470082a5574fde431b3c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.149888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b96d4611d66470082a5574fde431b3c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.152153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.271131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b96d4611d66470082a5574fde431b3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.271898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b96d4611d66470082a5574fde431b3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.292848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b96d4611d66470082a5574fde431b3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.293596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b96d4611d66470082a5574fde431b3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.322641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b96d4611d66470082a5574fde431b3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.356757 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3247/6480] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:34 2022] DELETE /identity/v3/users/9b96d4611d66470082a5574fde431b3c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:34.361498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.362644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.363763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.394014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:34.395941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.515853 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3234/6481] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:34.547819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.549671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.557380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.559684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.588595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f0eb3-6f58-4d32-9d3c-bffcc9cf6710 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.603297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.603974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.604290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.607575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] PATH_INFO: `/v3/projects/aedb1becfa4b42deb13b265c9377e368` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.612557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aedb1becfa4b42deb13b265c9377e368)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.618241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.657768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:34.756861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b310da-02cb-4e47-8e48-e11e82e579db None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.762183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b344d-96dd-40bb-8a08-4ce3e99fe42c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.762819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b344d-96dd-40bb-8a08-4ce3e99fe42c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.763154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b344d-96dd-40bb-8a08-4ce3e99fe42c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.763429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b344d-96dd-40bb-8a08-4ce3e99fe42c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.776004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b344d-96dd-40bb-8a08-4ce3e99fe42c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.778110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b344d-96dd-40bb-8a08-4ce3e99fe42c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.791076 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3235/6482] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:34.797823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aedb1becfa4b42deb13b265c9377e368', 'request_id': 'req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.811034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aedb1becfa4b42deb13b265c9377e368', 'request_id': 'req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.834715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aedb1becfa4b42deb13b265c9377e368', 'request_id': 'req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.860648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:34.869864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] The token cache is being invalidate because project aedb1becfa4b42deb13b265c9377e368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:34.870387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.872954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fdcb0cdf-f3dd-4f0c-92da-23b43df221d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.884290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72940b4-ca5d-4e7b-9e84-bc34772cea57 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.884591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72940b4-ca5d-4e7b-9e84-bc34772cea57 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.884882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72940b4-ca5d-4e7b-9e84-bc34772cea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.887688 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3248/6483] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:34 2022] DELETE /identity/v3/projects/aedb1becfa4b42deb13b265c9377e368 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:34.914147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e72940b4-ca5d-4e7b-9e84-bc34772cea57 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:34.915708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e72940b4-ca5d-4e7b-9e84-bc34772cea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:35.014371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a884-e317-4429-ac08-f8eb5fd2865f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.014666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a884-e317-4429-ac08-f8eb5fd2865f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.014988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a884-e317-4429-ac08-f8eb5fd2865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.018971 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3236/6484] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:35.044581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-faf1a884-e317-4429-ac08-f8eb5fd2865f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:35.046243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-faf1a884-e317-4429-ac08-f8eb5fd2865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:35.062603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72940b4-ca5d-4e7b-9e84-bc34772cea57 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.070861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3249/6485] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:35.083815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72940b4-ca5d-4e7b-9e84-bc34772cea57 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.092965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.092965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.092965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.092965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.094626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:35.094940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.099684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.120463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a884-e317-4429-ac08-f8eb5fd2865f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.130091 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3237/6486] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:35.134834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a884-e317-4429-ac08-f8eb5fd2865f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.143212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.145165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.146234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.146583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.147427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.151865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.217643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.218899 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3250/6487] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:35 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:35.240100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b800e-e786-43ad-a7a6-e164630f4f1d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.390448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.391227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.391604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.391987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.393261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.399183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.468865 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3238/6488] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:35 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:35.474252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.511891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2673e2-fe73-4366-82e2-ba5d0c8af5a1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.526211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.530078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.530366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.530597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.531301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.533010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.575958 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3251/6489] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:35.577993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.600337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56626fca-4adc-4e90-8537-abe0c6c9e14e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.620136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a52c8-0bbc-4931-8e1d-e611cc4bf32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.621068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a52c8-0bbc-4931-8e1d-e611cc4bf32a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.622093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a52c8-0bbc-4931-8e1d-e611cc4bf32a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.622499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a52c8-0bbc-4931-8e1d-e611cc4bf32a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.623322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a52c8-0bbc-4931-8e1d-e611cc4bf32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.626375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a52c8-0bbc-4931-8e1d-e611cc4bf32a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.662205 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3239/6490] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:35.665221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.691439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88450d33-0de3-4a30-9f72-54e095fc977d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.706751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.707711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.708130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.710245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] PATH_INFO: `/v3/projects/eda7ef86237546a9837904167e6f3bf5/users/664ea04b2259430ab73bb6495826578f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.796492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda7ef86237546a9837904167e6f3bf5, user_id=664ea04b2259430ab73bb6495826578f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.805154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.834127 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3252/6491] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:35 2022] PUT /identity/v3/projects/eda7ef86237546a9837904167e6f3bf5/users/664ea04b2259430ab73bb6495826578f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:35.839660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.840309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.840595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.883587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:35.889096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:35.965298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:36.069329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3240/6492] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:36.191913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:36.193890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:36.318532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afed76-2459-49ec-831e-013ae9d1f7d0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:36.333134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:36.333699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:36.334008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:36.334247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:36.354231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:36.356275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:36.388085 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3253/6493] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:36.835643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:36.839701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:36.883162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cd6ec-5c9f-4f08-9126-362f1d6dc3f3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:36.910689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:36.923878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:36.924344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:36.924756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] PATH_INFO: `/v3/projects/bf380d9339814239b9e1078b99c4905a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:36.930968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf380d9339814239b9e1078b99c4905a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:36.945228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:36.997641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:36.999649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:37.034129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf380d9339814239b9e1078b99c4905a', 'request_id': 'req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.050690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf380d9339814239b9e1078b99c4905a', 'request_id': 'req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.056225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57512fdd-7e02-46b4-bdd4-786dbdb1a723 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:37.061256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf380d9339814239b9e1078b99c4905a', 'request_id': 'req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.092005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] The token cache is being invalidate because project bf380d9339814239b9e1078b99c4905a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:37.092315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.097188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.101983 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3241/6494] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:36 2022] DELETE /identity/v3/projects/bf380d9339814239b9e1078b99c4905a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:37.106459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:37.108070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:37.252432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:37.253140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:37.253389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:37.253689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] PATH_INFO: `/v3/users/5f861462391648188d43f16bd9348979` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:37.254970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f861462391648188d43f16bd9348979)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:37.256937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:37.326019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a9519-a9df-402a-858d-5e0b5af3f7ec admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:37.343604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:37.348784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:37.349554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:37.349841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:37.350285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] PATH_INFO: `/v3/users/e52b3c140ba5428cb541914c4c61689f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:37.351635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e52b3c140ba5428cb541914c4c61689f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:37.354667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:37.383400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f861462391648188d43f16bd9348979'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.385406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f861462391648188d43f16bd9348979'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.415938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f861462391648188d43f16bd9348979'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.416682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f861462391648188d43f16bd9348979'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.495892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c09002ba-573d-4d07-93c8-f0715c31ce0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f861462391648188d43f16bd9348979'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.515848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e52b3c140ba5428cb541914c4c61689f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.536934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:37.552943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e52b3c140ba5428cb541914c4c61689f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.560061 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3254/6495] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:36 2022] DELETE /identity/v3/users/5f861462391648188d43f16bd9348979 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:37.575242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e52b3c140ba5428cb541914c4c61689f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.575892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e52b3c140ba5428cb541914c4c61689f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.576638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e52b3c140ba5428cb541914c4c61689f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:37.580895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57450d-bd65-4646-a543-53860d91089f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:37.581679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57450d-bd65-4646-a543-53860d91089f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:37.582193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57450d-bd65-4646-a543-53860d91089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:37.675306 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3242/6496] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:37 2022] DELETE /identity/v3/users/e52b3c140ba5428cb541914c4c61689f => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:37.709436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2a57450d-bd65-4646-a543-53860d91089f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:37.710617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a57450d-bd65-4646-a543-53860d91089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:37.854457 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3255/6497] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:37.902878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:37.904445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:38.047852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd116c-4680-42ed-aaa6-7aca8558821c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:38.051918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c097a-f888-4573-842c-281235585680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:38.052541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c097a-f888-4573-842c-281235585680 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:38.052820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c097a-f888-4573-842c-281235585680 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:38.053103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c097a-f888-4573-842c-281235585680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:38.064629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c097a-f888-4573-842c-281235585680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:38.066148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c097a-f888-4573-842c-281235585680 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:38.085567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3243/6498] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:38.375920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:38.375920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:38.375920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:38.486945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] MFA Rules not processed for user `664ea04b2259430ab73bb6495826578f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:38.488289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:38.634136 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3256/6499] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:38.645625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:38.645902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:38.646178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:38.684876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] MFA Rules not processed for user `664ea04b2259430ab73bb6495826578f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:38.692709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:38.701013 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3244/6500] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:38.711002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:38.712794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:38.856703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:38.926844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51361cb8-a564-4a7d-b270-ebc1f992dd1c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:38.936461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:38.937293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:38.937839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:38.938223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:38.959411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:38.961382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:38.994237 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3257/6501] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:39.420373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:39.422439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:39.467364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959476e-38a6-40c5-9e87-2d8eece4fa07 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:39.487608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:39.488365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:39.488705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:39.489046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] PATH_INFO: `/v3/projects/d485f1219d7f471cba8ebface38c8cec` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:39.492811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d485f1219d7f471cba8ebface38c8cec)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:39.495125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:39.623526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:39.624491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:39.627338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd485f1219d7f471cba8ebface38c8cec', 'request_id': 'req-508b2d87-d601-48d8-b569-21970ff237b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:39.658957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba23341-8cea-4fc7-8029-a4ac5a85f21b service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:39.659318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd485f1219d7f471cba8ebface38c8cec', 'request_id': 'req-508b2d87-d601-48d8-b569-21970ff237b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:39.687016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd485f1219d7f471cba8ebface38c8cec', 'request_id': 'req-508b2d87-d601-48d8-b569-21970ff237b3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:39.722763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] The token cache is being invalidate because project d485f1219d7f471cba8ebface38c8cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:39.723106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:39.726539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:39.729642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3245/6502] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:39 2022] DELETE /identity/v3/projects/d485f1219d7f471cba8ebface38c8cec => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:39.735804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:39.737363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:39.828385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:39.831107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:39.832143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:39.833607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] PATH_INFO: `/v3/projects/1cfaa56865c040638694461bad7a1f5e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:39.838426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cfaa56865c040638694461bad7a1f5e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:39.844837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:39.854366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:39.879986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b2d87-d601-48d8-b569-21970ff237b3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:39.894451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:39.895541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:39.896187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:39.896515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] PATH_INFO: `/v3/users/2c6a5e73b1204e19a3924dff9c5ea88e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:39.897800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6a5e73b1204e19a3924dff9c5ea88e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:39.899930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:40.014458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cfaa56865c040638694461bad7a1f5e', 'request_id': 'req-c5d193c0-e1c9-408a-a5ba-058f7f00203c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.023933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cfaa56865c040638694461bad7a1f5e', 'request_id': 'req-c5d193c0-e1c9-408a-a5ba-058f7f00203c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.034580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6a5e73b1204e19a3924dff9c5ea88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.046406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cfaa56865c040638694461bad7a1f5e', 'request_id': 'req-c5d193c0-e1c9-408a-a5ba-058f7f00203c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.111167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] The token cache is being invalidate because project 1cfaa56865c040638694461bad7a1f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:40.111519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.142926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5d193c0-e1c9-408a-a5ba-058f7f00203c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.163558 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3258/6503] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:39 2022] DELETE /identity/v3/projects/1cfaa56865c040638694461bad7a1f5e => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:40.165512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6a5e73b1204e19a3924dff9c5ea88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.168657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b8e0-fc55-4427-9e38-ab777db6294b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:40.168963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b8e0-fc55-4427-9e38-ab777db6294b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:40.169245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b8e0-fc55-4427-9e38-ab777db6294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:40.199559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6a5e73b1204e19a3924dff9c5ea88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.200434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6a5e73b1204e19a3924dff9c5ea88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.201081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62cd6637-d029-4fb7-8caf-8848560248ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6a5e73b1204e19a3924dff9c5ea88e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:40.223653 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3246/6504] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:39 2022] DELETE /identity/v3/users/2c6a5e73b1204e19a3924dff9c5ea88e => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:40.226842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6929b8e0-fc55-4427-9e38-ab777db6294b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:40.227994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6929b8e0-fc55-4427-9e38-ab777db6294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:40.386673 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3259/6505] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:40.395526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431e70-0b04-4f3e-ae57-ea98a2ccf057 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:40.395870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431e70-0b04-4f3e-ae57-ea98a2ccf057 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:40.396081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431e70-0b04-4f3e-ae57-ea98a2ccf057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:40.456107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b1431e70-0b04-4f3e-ae57-ea98a2ccf057 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:40.458810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b1431e70-0b04-4f3e-ae57-ea98a2ccf057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:40.472470 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3247/6506] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:40.478934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b8e0-fc55-4427-9e38-ab777db6294b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:40.494562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:40.504074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b8e0-fc55-4427-9e38-ab777db6294b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:40.518276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:40.522905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:40.523352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:40.525520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:40.526386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:40.526837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:40.547007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:40.587779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431e70-0b04-4f3e-ae57-ea98a2ccf057 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:40.597666 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3260/6507] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:40.600410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431e70-0b04-4f3e-ae57-ea98a2ccf057 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:40.619626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:40.620507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:40.625343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:40.625842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:40.626714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:40.628859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:40.702897 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3248/6508] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:40 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:40.706942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:40.718469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac32b3-fa84-4214-868a-0411a4283e06 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:40.879978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:40.880672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:40.880957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:40.881255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:40.884518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:40.892763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:40.952424 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3261/6509] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:40 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:40.956363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:40.970898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8b216-5291-4777-afa1-f8cf111ac8cd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:40.975738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:40.976363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:40.976640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:40.976910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:40.977689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:40.980238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:40.996945 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3249/6510] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:41.001170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:41.016028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bb10a-a976-4e73-9126-48e13710da4c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:41.022851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519dd92-3bb8-421d-a45e-09eb1bada752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:41.023559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519dd92-3bb8-421d-a45e-09eb1bada752 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:41.023911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519dd92-3bb8-421d-a45e-09eb1bada752 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:41.024546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519dd92-3bb8-421d-a45e-09eb1bada752 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:41.025484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519dd92-3bb8-421d-a45e-09eb1bada752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:41.027330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519dd92-3bb8-421d-a45e-09eb1bada752 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:41.049431 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3262/6511] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:41.053844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:41.075434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5381a-1ee4-41b9-a697-3fff7dda4859 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:41.097774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:41.098436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:41.098716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:41.098990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] PATH_INFO: `/v3/projects/a58223645aa741b4af2ab6ffba072ddd/users/79e2fa81dbe94025bf959b35a1fd99be/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:41.140227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58223645aa741b4af2ab6ffba072ddd, user_id=79e2fa81dbe94025bf959b35a1fd99be, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:41.142996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:41.190861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366034ee-2192-4575-ada6-265364386ed9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:41.191204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366034ee-2192-4575-ada6-265364386ed9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:41.192226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366034ee-2192-4575-ada6-265364386ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:41.193268 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3250/6512] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:41 2022] PUT /identity/v3/projects/a58223645aa741b4af2ab6ffba072ddd/users/79e2fa81dbe94025bf959b35a1fd99be/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:41.254215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-366034ee-2192-4575-ada6-265364386ed9 None None] MFA Rules not processed for user `79e2fa81dbe94025bf959b35a1fd99be`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:41.256055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-366034ee-2192-4575-ada6-265364386ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:41.484792 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3263/6513] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:41.485383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:41.611504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb106a64-13f2-467b-8b21-fe7feea51dc1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:41.617495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:41.618529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:41.619287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:41.620125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:41.622945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:41.625179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:41.654157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:41.709752 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3251/6514] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:41 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:41.723561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366034ee-2192-4575-ada6-265364386ed9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:41.728457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:41.735229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:41.760170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366034ee-2192-4575-ada6-265364386ed9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:41.865109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71630026-f9db-4051-9108-f9dcbe85c764 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:41.879880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:41.880859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:41.881339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:41.881858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] PATH_INFO: `/v3/users/664ea04b2259430ab73bb6495826578f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:41.883298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=664ea04b2259430ab73bb6495826578f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:41.885710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:41.946964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:41.960335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:41.961187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:41.961518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:41.963250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:41.963966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:41.966822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:42.023829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664ea04b2259430ab73bb6495826578f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:42.029168 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3264/6515] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:41 2022] POST /identity/v3/users => generated 300 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:42.032153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:42.050933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcc1e7-a3ea-43da-8b98-85baa54ac899 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:42.057996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664ea04b2259430ab73bb6495826578f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:42.066070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:42.066715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:42.066989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:42.067273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:42.067923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:42.069620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:42.096137 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3265/6516] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:42.097494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:42.098176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664ea04b2259430ab73bb6495826578f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:42.100152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664ea04b2259430ab73bb6495826578f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:42.102222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664ea04b2259430ab73bb6495826578f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:42.142025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b7378-461e-40e6-b078-237457e35df0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:42.153198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:42.153444 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3252/6517] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:41 2022] DELETE /identity/v3/users/664ea04b2259430ab73bb6495826578f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:42.155919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:42.156609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:42.156609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] PATH_INFO: `/v3/projects/a756354e783c4298be135f23f8f0f516/users/3a7c3f82d94a4650adf5761e18b285a5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:42.208571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a756354e783c4298be135f23f8f0f516, user_id=3a7c3f82d94a4650adf5761e18b285a5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:42.214589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:42.243590 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3266/6518] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:42 2022] PUT /identity/v3/projects/a756354e783c4298be135f23f8f0f516/users/3a7c3f82d94a4650adf5761e18b285a5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:42.247748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:42.249447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:42.392164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d52e-1e09-4f04-ac43-42b658ae2583 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:42.430452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:42.494933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:42.497316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:42.517965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:42.519298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:42.520320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:42.520790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:42.522130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:42.524016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:42.542955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3253/6519] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:42.545711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:42.579091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499d5c4-fb15-48ce-a85d-592b4d246acf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:42.594428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233c948-4234-4eea-89eb-97f8f2962595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:42.594651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c948-4234-4eea-89eb-97f8f2962595 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:42.594859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c948-4234-4eea-89eb-97f8f2962595 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:42.595319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c948-4234-4eea-89eb-97f8f2962595 admin admin] PATH_INFO: `/v3/projects/a756354e783c4298be135f23f8f0f516/users/3a7c3f82d94a4650adf5761e18b285a5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:42.605039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233c948-4234-4eea-89eb-97f8f2962595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a756354e783c4298be135f23f8f0f516, user_id=3a7c3f82d94a4650adf5761e18b285a5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:42.607814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233c948-4234-4eea-89eb-97f8f2962595 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:42.657954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3254/6520] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:42 2022] PUT /identity/v3/projects/a756354e783c4298be135f23f8f0f516/users/3a7c3f82d94a4650adf5761e18b285a5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:42.661547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:42.661547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:42.661547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:42.730960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf87b6e-9354-4f8f-bd9c-d073d43c0883 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:42.731852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] MFA Rules not processed for user `3a7c3f82d94a4650adf5761e18b285a5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:42.733097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:42.939651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:42.940501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:42.940885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:42.941250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] PATH_INFO: `/v3/projects/07b56059667641feac379a67ec2e3d5f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:42.946243 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3255/6521] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:42.946427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b56059667641feac379a67ec2e3d5f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:42.946427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:42.956811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:42.958275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:43.083193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b56059667641feac379a67ec2e3d5f', 'request_id': 'req-04526356-9a9b-4aa3-b0c6-e2d245040314'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.096131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b56059667641feac379a67ec2e3d5f', 'request_id': 'req-04526356-9a9b-4aa3-b0c6-e2d245040314'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.107796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7336d-cfda-4d58-82b5-020008b09e2f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:43.120203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b56059667641feac379a67ec2e3d5f', 'request_id': 'req-04526356-9a9b-4aa3-b0c6-e2d245040314'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.122467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:43.164802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] The token cache is being invalidate because project 07b56059667641feac379a67ec2e3d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:43.167279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.169585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.174669 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3267/6522] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:42 2022] DELETE /identity/v3/projects/07b56059667641feac379a67ec2e3d5f => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:43.179691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:43.189346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:43.290214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:43.304841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:43.306126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:43.306126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:43.311311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:43.315865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:43.365668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04526356-9a9b-4aa3-b0c6-e2d245040314 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:43.372156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:43.373614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:43.373955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:43.374554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] PATH_INFO: `/v3/users/0180cb0f20e54d00928f651e63be0076` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:43.375997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0180cb0f20e54d00928f651e63be0076)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:43.378120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:43.526966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0180cb0f20e54d00928f651e63be0076'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.527986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0180cb0f20e54d00928f651e63be0076'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.543814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:43.544668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0180cb0f20e54d00928f651e63be0076'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.545532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0180cb0f20e54d00928f651e63be0076'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.548446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:43.568147 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3256/6523] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:43.569826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:43.571577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:43.603331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9132553-f50f-4d1e-bca0-3a10c34b0065 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:43.609950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67fcde59-1522-4d6d-accc-6a1044f5bea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0180cb0f20e54d00928f651e63be0076'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.645079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:43.646806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:43.647610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:43.647985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] PATH_INFO: `/v3/projects/eda7ef86237546a9837904167e6f3bf5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:43.653374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda7ef86237546a9837904167e6f3bf5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:43.657719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:43.662824 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3268/6524] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:43 2022] DELETE /identity/v3/users/0180cb0f20e54d00928f651e63be0076 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:43.782382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda7ef86237546a9837904167e6f3bf5', 'request_id': 'req-c83e6369-55c5-4c42-ada6-1972bf3406e8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.802866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda7ef86237546a9837904167e6f3bf5', 'request_id': 'req-c83e6369-55c5-4c42-ada6-1972bf3406e8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.813291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda7ef86237546a9837904167e6f3bf5', 'request_id': 'req-c83e6369-55c5-4c42-ada6-1972bf3406e8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.848726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] The token cache is being invalidate because project eda7ef86237546a9837904167e6f3bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:43.850362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.862260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:43.879690 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3257/6525] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:43 2022] DELETE /identity/v3/projects/eda7ef86237546a9837904167e6f3bf5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:43.887309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817d5e8-1021-4949-9ad6-1499f358fb26 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:43.887889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817d5e8-1021-4949-9ad6-1499f358fb26 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:43.887889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817d5e8-1021-4949-9ad6-1499f358fb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:43.921655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:43.923214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:43.928731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8817d5e8-1021-4949-9ad6-1499f358fb26 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:43.930037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8817d5e8-1021-4949-9ad6-1499f358fb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:44.024961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e6369-55c5-4c42-ada6-1972bf3406e8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:44.032852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:44.033942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.034418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.035417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] PATH_INFO: `/v3/users/79e2fa81dbe94025bf959b35a1fd99be` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.037221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e2fa81dbe94025bf959b35a1fd99be)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:44.039417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:44.071027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:44.126943 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3269/6526] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:44.135679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d674a-c66b-49d3-832d-147ca3117783 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.135964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d674a-c66b-49d3-832d-147ca3117783 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.136238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d674a-c66b-49d3-832d-147ca3117783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.139054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e2fa81dbe94025bf959b35a1fd99be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:44.158899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e2fa81dbe94025bf959b35a1fd99be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:44.169859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e2fa81dbe94025bf959b35a1fd99be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:44.170889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e2fa81dbe94025bf959b35a1fd99be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:44.171903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d01d3875-1e8e-48e8-85de-4105c5208675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e2fa81dbe94025bf959b35a1fd99be'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:44.182741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-187d674a-c66b-49d3-832d-147ca3117783 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:44.185730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-187d674a-c66b-49d3-832d-147ca3117783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:44.191918 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3258/6527] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:43 2022] DELETE /identity/v3/users/79e2fa81dbe94025bf959b35a1fd99be => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:44.193187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:44.194878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.195264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.195594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.246280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:44.247901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:44.315992 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3270/6528] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:44.317077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d674a-c66b-49d3-832d-147ca3117783 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:44.342740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d674a-c66b-49d3-832d-147ca3117783 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:44.352102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:44.352812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.353130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.353382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.356892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:44.358908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:44.372789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:44.384224 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3259/6529] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:44.395699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:44.399136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:44.404273 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3271/6530] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:44.413015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:44.448711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93a971-7cb2-42cb-a3ab-e1998dad882d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:44.458408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:44.460222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.461052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.461348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.464204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:44.468189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:44.549547 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3272/6531] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:44 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:44.550550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:44.642602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b801d90-99d8-401a-9b2b-372cdf820aa2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:44.786072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f495e4-087b-49cb-bd50-d91a212bafd1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:44.791814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:44.792550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.792827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.793227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.794362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:44.795077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:44.795947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.796334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.796563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:44.796842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.808410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:44.812219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:44.814496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:44.831002 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3260/6532] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:44.847915 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3273/6533] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:44 2022] POST /identity/v3/users => generated 288 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:44.852072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:44.862927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a70e0d-1fbd-4e03-9471-6e147e540a33 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:44.868847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:44.869888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.870274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.870870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.872254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:44.874081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:44.890735 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3261/6534] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:44.894874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:44.921973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc86cf-64c9-4bde-afa9-5461d813a594 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:44.948343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8915c01-9711-499b-8404-73871aabfa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:44.950791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8915c01-9711-499b-8404-73871aabfa53 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:44.951225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8915c01-9711-499b-8404-73871aabfa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:44.959826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8915c01-9711-499b-8404-73871aabfa53 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:44.961525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8915c01-9711-499b-8404-73871aabfa53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:44.967871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8915c01-9711-499b-8404-73871aabfa53 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:45.006755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:45.017075 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3274/6535] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:45.022854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e94f1e-b2b4-40bd-9108-cfe0dc17e176 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:45.031203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:45.032083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:45.032485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:45.032875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] PATH_INFO: `/v3/projects/8d3fab3adf0c425680c91af8afcc516f/users/7ff5b886208a44c0891c2d1ba65dc083/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:45.055345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3fab3adf0c425680c91af8afcc516f, user_id=7ff5b886208a44c0891c2d1ba65dc083, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:45.058564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:45.085968 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3262/6536] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:45 2022] PUT /identity/v3/projects/8d3fab3adf0c425680c91af8afcc516f/users/7ff5b886208a44c0891c2d1ba65dc083/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:45.089126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:45.089972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:45.090330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:45.125716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:45.128996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:45.250083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:45.277554 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3275/6537] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:45.284306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:45.286499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:45.445803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802000d-9a47-4818-9533-d4903d84ae15 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:45.462998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc27a11-1502-43fe-8957-7d9e659e4119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:45.463645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc27a11-1502-43fe-8957-7d9e659e4119 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:45.463929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc27a11-1502-43fe-8957-7d9e659e4119 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:45.464373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc27a11-1502-43fe-8957-7d9e659e4119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:45.503875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc27a11-1502-43fe-8957-7d9e659e4119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:45.511878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc27a11-1502-43fe-8957-7d9e659e4119 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:45.551041 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3263/6538] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:45.788772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:45.790361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:45.825749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91406-a66b-4822-a160-3bfda9a6802c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:45.848429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:45.851746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:45.855338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:45.855671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] PATH_INFO: `/v3/projects/5179ac2aa1324664980bdca51d02137c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:45.859244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5179ac2aa1324664980bdca51d02137c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:45.871756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:46.043685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5179ac2aa1324664980bdca51d02137c', 'request_id': 'req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.057366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5179ac2aa1324664980bdca51d02137c', 'request_id': 'req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.074788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5179ac2aa1324664980bdca51d02137c', 'request_id': 'req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.092115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] The token cache is being invalidate because project 5179ac2aa1324664980bdca51d02137c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:46.093067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.094012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3f6fa62-3eb7-4f85-bf82-750b53ba9548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.096151 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3276/6539] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:45 2022] DELETE /identity/v3/projects/5179ac2aa1324664980bdca51d02137c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:46.100982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06b60-34f3-4b77-aa4f-c91b2ce19d78 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.103415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06b60-34f3-4b77-aa4f-c91b2ce19d78 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.103942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06b60-34f3-4b77-aa4f-c91b2ce19d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.134775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b4b06b60-34f3-4b77-aa4f-c91b2ce19d78 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:46.136936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b4b06b60-34f3-4b77-aa4f-c91b2ce19d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:46.306251 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3264/6540] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:46.311175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75034b-2887-4810-9697-adf87303350b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.313523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75034b-2887-4810-9697-adf87303350b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.313914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75034b-2887-4810-9697-adf87303350b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.366985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fb75034b-2887-4810-9697-adf87303350b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:46.369711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fb75034b-2887-4810-9697-adf87303350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:46.385179 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3277/6541] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:46.389270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06b60-34f3-4b77-aa4f-c91b2ce19d78 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.408510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06b60-34f3-4b77-aa4f-c91b2ce19d78 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.435522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.439467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.441844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.442207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.443942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:46.448112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.455930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:46.506848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3265/6542] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:46.509002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75034b-2887-4810-9697-adf87303350b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.527085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75034b-2887-4810-9697-adf87303350b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.556890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.559661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.559961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.560236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.561900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.570918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:46.654468 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3278/6543] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:46.656070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.672576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b04cc0-f6d0-40e0-b12a-d2bd2bd90555 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.807640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:46.842326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.843429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.845751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.846768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.847777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.850169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:46.907681 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3266/6544] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:46.911937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.971216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf16b0-29a4-417b-b2c3-da9c8748f203 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.980260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.981279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.995033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.995701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.997315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.999378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.029444 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3279/6545] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:47.033507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.052000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee76f-814d-466c-b843-9d477664663e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.068883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.074772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.075201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.075479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.076203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.080437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.118472 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3267/6546] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:47.122210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.144037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cafd4-defd-46db-b476-d7facdb82f4e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.175480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ec7f6-c6e3-4add-baec-7ade42e297f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.176638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ec7f6-c6e3-4add-baec-7ade42e297f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.177971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ec7f6-c6e3-4add-baec-7ade42e297f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.178569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ec7f6-c6e3-4add-baec-7ade42e297f8 admin admin] PATH_INFO: `/v3/projects/5b03de630e9147ea956fc60c721ac029/users/af02475a88b6401d97b31b7c5563fe99/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.252045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ec7f6-c6e3-4add-baec-7ade42e297f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b03de630e9147ea956fc60c721ac029, user_id=af02475a88b6401d97b31b7c5563fe99, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.267813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ec7f6-c6e3-4add-baec-7ade42e297f8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.279996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.282579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:47.315755 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3280/6547] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:47 2022] PUT /identity/v3/projects/5b03de630e9147ea956fc60c721ac029/users/af02475a88b6401d97b31b7c5563fe99/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:47.328461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.328831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.329165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.361312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f4b95-6b97-4897-ab4f-d59b36cbda00 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.444192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] MFA Rules not processed for user `af02475a88b6401d97b31b7c5563fe99`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:47.447469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:47.586771 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3281/6548] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:47.602441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.603115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.603244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.603407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.603673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] PATH_INFO: `/v3/projects/a58223645aa741b4af2ab6ffba072ddd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.605447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58223645aa741b4af2ab6ffba072ddd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.605659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:47.607423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.625157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:47.797130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58223645aa741b4af2ab6ffba072ddd', 'request_id': 'req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:47.863290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58223645aa741b4af2ab6ffba072ddd', 'request_id': 'req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:47.866287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0f5e6-28be-4f29-9935-cc22c475ed3e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.944342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58223645aa741b4af2ab6ffba072ddd', 'request_id': 'req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:47.985505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] The token cache is being invalidate because project a58223645aa741b4af2ab6ffba072ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:47.985969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:47.988742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.011521 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3268/6549] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:47 2022] DELETE /identity/v3/projects/a58223645aa741b4af2ab6ffba072ddd => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:48.013403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:48.017840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.053433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:48.053433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:48.053433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:48.053433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:48.065622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.193684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee63cb1-a1c2-4b5f-a093-7d835638dc24 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:48.212641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:48.213312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:48.213619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:48.213899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] PATH_INFO: `/v3/users/3a7c3f82d94a4650adf5761e18b285a5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:48.219185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7c3f82d94a4650adf5761e18b285a5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:48.221183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:48.223240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:48.259351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:48.260849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:48.298412 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3282/6550] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:48.354328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7c3f82d94a4650adf5761e18b285a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.385763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7c3f82d94a4650adf5761e18b285a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.408356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7c3f82d94a4650adf5761e18b285a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.409003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7c3f82d94a4650adf5761e18b285a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.409662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4ba12cc5-a05e-45a8-80d5-d7833b5dc0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7c3f82d94a4650adf5761e18b285a5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.445000 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3269/6551] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:48 2022] DELETE /identity/v3/users/3a7c3f82d94a4650adf5761e18b285a5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:51.013961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:51.038000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1fc1-6879-4af9-b37f-d84a56a0d4ed service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:51.188279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:51.188936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.189378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.189818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] PATH_INFO: `/v3/projects/a756354e783c4298be135f23f8f0f516` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.191228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a756354e783c4298be135f23f8f0f516)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:51.193323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:51.239922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:51.305818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a756354e783c4298be135f23f8f0f516', 'request_id': 'req-344b6800-2f61-4099-a8ed-a52ba0aa8115'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.313977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a756354e783c4298be135f23f8f0f516', 'request_id': 'req-344b6800-2f61-4099-a8ed-a52ba0aa8115'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.335194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a756354e783c4298be135f23f8f0f516', 'request_id': 'req-344b6800-2f61-4099-a8ed-a52ba0aa8115'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.379069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] The token cache is being invalidate because project a756354e783c4298be135f23f8f0f516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:51.379591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.382789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.387976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3283/6552] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:51 2022] DELETE /identity/v3/projects/a756354e783c4298be135f23f8f0f516 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:51.395024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030e16-ca87-468e-9d9a-7978b2ced287 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.395568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030e16-ca87-468e-9d9a-7978b2ced287 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.395725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030e16-ca87-468e-9d9a-7978b2ced287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.445922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f4030e16-ca87-468e-9d9a-7978b2ced287 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:51.448387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f4030e16-ca87-468e-9d9a-7978b2ced287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:51.510179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:51.513636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:51.660007 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3270/6553] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:51.673530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00344a6e-6096-41bb-871b-5d7d01fd7f3e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.673530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00344a6e-6096-41bb-871b-5d7d01fd7f3e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.673530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00344a6e-6096-41bb-871b-5d7d01fd7f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.725650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-00344a6e-6096-41bb-871b-5d7d01fd7f3e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:51.730134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-00344a6e-6096-41bb-871b-5d7d01fd7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:51.734558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b6800-2f61-4099-a8ed-a52ba0aa8115 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:51.751046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:51.752054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.753224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.753883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] PATH_INFO: `/v3/users/af02475a88b6401d97b31b7c5563fe99` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.756627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af02475a88b6401d97b31b7c5563fe99)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:51.762014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:51.771258 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3271/6554] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:51.773874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00344a6e-6096-41bb-871b-5d7d01fd7f3e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:51.788429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00344a6e-6096-41bb-871b-5d7d01fd7f3e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:51.811209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:51.811797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.812036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.812314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.813532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:51.813532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:51.814952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:51.859365 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3272/6555] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:15:51.861249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:51.889339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8286ef-ffcb-4ea5-98e0-16d340c7fa61 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:51.889804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af02475a88b6401d97b31b7c5563fe99'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.893878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af02475a88b6401d97b31b7c5563fe99'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.913624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:51.914203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.914489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.914790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.915521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:51.917121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af02475a88b6401d97b31b7c5563fe99'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.918170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af02475a88b6401d97b31b7c5563fe99'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.928629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:51.947489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e695f7e0-a7f5-48cd-9d7b-c6447e85b0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af02475a88b6401d97b31b7c5563fe99'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:51.979297 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3284/6556] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:51 2022] DELETE /identity/v3/users/af02475a88b6401d97b31b7c5563fe99 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:51.984234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.985892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.986450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.026423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:52.027201 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3273/6557] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:51 2022] POST /identity/v3/projects => generated 409 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:52.028822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:52.028934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.037202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:52.065279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:52.166949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:52.198898 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3285/6558] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:52.214156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6df2a-7d3b-48de-a970-4a0704ec6f87 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.220208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.220942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.221432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.221901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.222658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:52.225075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:52.227824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.229495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:52.286917 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3274/6559] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:52 2022] POST /identity/v3/users => generated 305 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:52.292656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.323250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11534a-1377-411a-b109-88cfcc7e96cb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.336605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.337765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.338710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.339122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.339880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff18026-1418-475f-ab94-dcb3ca4652e4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.341649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:52.344284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.345191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.345610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.345982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.348397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:52.367799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:52.370759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:52.385852 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3275/6560] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:52.391604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.397098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3286/6561] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:52.418707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904efb9-420d-4ec4-9869-34e069074e34 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.431751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee527ed-81ad-4a33-bd10-6309ed69ec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.432391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee527ed-81ad-4a33-bd10-6309ed69ec7b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.433341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee527ed-81ad-4a33-bd10-6309ed69ec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.433643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee527ed-81ad-4a33-bd10-6309ed69ec7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.434305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee527ed-81ad-4a33-bd10-6309ed69ec7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:52.444845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee527ed-81ad-4a33-bd10-6309ed69ec7b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:52.483611 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3276/6562] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:52.487712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.502428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236aed-8b41-4824-a79c-6db5622281d6 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.524429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.525431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.525822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.526187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] PATH_INFO: `/v3/projects/a98f75b3241841cdaa283fa4075d7593/users/e494b95db6df4b9b8d119c6f70e2f81a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.550554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98f75b3241841cdaa283fa4075d7593, user_id=e494b95db6df4b9b8d119c6f70e2f81a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:52.555235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:52.583083 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3287/6563] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:52 2022] PUT /identity/v3/projects/a98f75b3241841cdaa283fa4075d7593/users/e494b95db6df4b9b8d119c6f70e2f81a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:52.595405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8a2ed-2949-410c-9a06-bd468284b5fd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.596299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8a2ed-2949-410c-9a06-bd468284b5fd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.596773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8a2ed-2949-410c-9a06-bd468284b5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.664885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c3b8a2ed-2949-410c-9a06-bd468284b5fd None None] MFA Rules not processed for user `e494b95db6df4b9b8d119c6f70e2f81a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:52.679419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c3b8a2ed-2949-410c-9a06-bd468284b5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:52.844769 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3277/6564] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:52.849148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.980401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4addad0-96c4-4ca4-a40d-38dd1a1df5d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.994255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.994909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.996726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.997000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.998968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.003247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.071844 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3288/6565] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:52 2022] POST /identity/v3/projects => generated 409 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:53.073384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8a2ed-2949-410c-9a06-bd468284b5fd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:53.089491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8a2ed-2949-410c-9a06-bd468284b5fd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:53.119564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:53.122121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:53.182743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:53.237171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:53.237929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:53.239293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:53.239681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.243722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.247040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.258727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303f867-99fb-4528-8665-90b2a347b25e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:53.270254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:53.271159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:53.272104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:53.273774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] PATH_INFO: `/v3/projects/5b03de630e9147ea956fc60c721ac029` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.276046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b03de630e9147ea956fc60c721ac029)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.278405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.297599 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3278/6566] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:53 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:53.299018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:53.319599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c623-e84b-4e45-a728-96251e0fd46c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:53.344643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:53.346919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:53.349902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:53.350885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.352052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.360489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.396819 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3279/6567] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:53.398087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:53.417253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b03de630e9147ea956fc60c721ac029', 'request_id': 'req-138a3a50-d766-46a9-a408-277a8c29cb59'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.419135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d9ad0-dde6-4efd-a775-494db5c50dea admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:53.433037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b03de630e9147ea956fc60c721ac029', 'request_id': 'req-138a3a50-d766-46a9-a408-277a8c29cb59'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.442619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:53.455970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b03de630e9147ea956fc60c721ac029', 'request_id': 'req-138a3a50-d766-46a9-a408-277a8c29cb59'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.472088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] The token cache is being invalidate because project 5b03de630e9147ea956fc60c721ac029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:53.473670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.476748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.482600 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3289/6568] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:53 2022] DELETE /identity/v3/projects/5b03de630e9147ea956fc60c721ac029 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:53.631284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:53.632689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:53.632981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:53.633258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] PATH_INFO: `/v3/projects/beb609b6edbd457184bcabb5cb297f0d/users/0890d53f29c94926a2aad393c888ca44/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.678179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb609b6edbd457184bcabb5cb297f0d, user_id=0890d53f29c94926a2aad393c888ca44, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.685765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.724833 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3280/6569] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:53 2022] PUT /identity/v3/projects/beb609b6edbd457184bcabb5cb297f0d/users/0890d53f29c94926a2aad393c888ca44/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:53.727215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:53.729082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:53.755360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3a50-d766-46a9-a408-277a8c29cb59 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:53.966897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fedd0-461a-4957-8c71-a9f568c45a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:53.967569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fedd0-461a-4957-8c71-a9f568c45a06 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:53.967853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fedd0-461a-4957-8c71-a9f568c45a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:53.968121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fedd0-461a-4957-8c71-a9f568c45a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.968839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fedd0-461a-4957-8c71-a9f568c45a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.970605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fedd0-461a-4957-8c71-a9f568c45a06 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.020212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.023610 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3290/6570] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:54.038749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e050a-96a0-4ee0-bd90-0f9f1adb80b0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.060308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.061285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.064638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.066522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] PATH_INFO: `/v3/projects/beb609b6edbd457184bcabb5cb297f0d/users/0890d53f29c94926a2aad393c888ca44/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.075424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb609b6edbd457184bcabb5cb297f0d, user_id=0890d53f29c94926a2aad393c888ca44, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.086383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.108134 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3281/6571] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:54 2022] PUT /identity/v3/projects/beb609b6edbd457184bcabb5cb297f0d/users/0890d53f29c94926a2aad393c888ca44/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:54.119900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1621f-d683-431d-be1c-5bcbe9d49a4c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.120445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1621f-d683-431d-be1c-5bcbe9d49a4c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.120445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1621f-d683-431d-be1c-5bcbe9d49a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.184391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-62c1621f-d683-431d-be1c-5bcbe9d49a4c None None] MFA Rules not processed for user `0890d53f29c94926a2aad393c888ca44`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:54.185701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-62c1621f-d683-431d-be1c-5bcbe9d49a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:54.299596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:54.367304 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3291/6572] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:54.383722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.387693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:54.607757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5db7c-ef4a-4c47-b15c-50743f1caea6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.612731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.613443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.615710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.616439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.644498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.646304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.699714 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3282/6573] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:54 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:55.109007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1621f-d683-431d-be1c-5bcbe9d49a4c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.122691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1621f-d683-431d-be1c-5bcbe9d49a4c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.137191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc84ee-b352-4634-8061-826c8beb513c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.137878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc84ee-b352-4634-8061-826c8beb513c service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.138172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc84ee-b352-4634-8061-826c8beb513c service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.138422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc84ee-b352-4634-8061-826c8beb513c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.140310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8dc84ee-b352-4634-8061-826c8beb513c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:55.318478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc84ee-b352-4634-8061-826c8beb513c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.319902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc84ee-b352-4634-8061-826c8beb513c service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.332183 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3292/6574] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:55 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:02.135941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:02.307815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809848d-56bc-4c3b-b57c-290f487e425e service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:02.319802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:02.320914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:02.321190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:02.321509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] PATH_INFO: `/v3/users/e494b95db6df4b9b8d119c6f70e2f81a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:02.323663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e494b95db6df4b9b8d119c6f70e2f81a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:02.329418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:02.403551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:02.450548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e494b95db6df4b9b8d119c6f70e2f81a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:02.465285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e494b95db6df4b9b8d119c6f70e2f81a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:02.486424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e494b95db6df4b9b8d119c6f70e2f81a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:02.487248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e494b95db6df4b9b8d119c6f70e2f81a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:02.488228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e494b95db6df4b9b8d119c6f70e2f81a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:02.508285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3283/6575] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:02 2022] DELETE /identity/v3/users/e494b95db6df4b9b8d119c6f70e2f81a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:16:02.513241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:02.514083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:02.514504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:02.547188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:02.550153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:02.727816 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3293/6576] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:02.742457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:02.744077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:02.820701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21aea5-61f4-4c32-8210-5494d9fa6db1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:02.824654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:02.825320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:02.825640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:02.825911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:02.836119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:02.837877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:02.849773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3284/6577] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:04.974966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:04.981580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:04.995007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cef50-25d5-4fcc-aab0-5ec787b2beab None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:05.008461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:05.009125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:05.009403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:05.009723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] PATH_INFO: `/v3/projects/a98f75b3241841cdaa283fa4075d7593` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:05.010864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98f75b3241841cdaa283fa4075d7593)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:05.012817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:05.130837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98f75b3241841cdaa283fa4075d7593', 'request_id': 'req-727eef0c-dbb7-4bf1-a70b-83eeff543555'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.170789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98f75b3241841cdaa283fa4075d7593', 'request_id': 'req-727eef0c-dbb7-4bf1-a70b-83eeff543555'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.206624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98f75b3241841cdaa283fa4075d7593', 'request_id': 'req-727eef0c-dbb7-4bf1-a70b-83eeff543555'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.216812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:05.225378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] The token cache is being invalidate because project a98f75b3241841cdaa283fa4075d7593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:05.226982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.228444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.232046 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3294/6578] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:04 2022] DELETE /identity/v3/projects/a98f75b3241841cdaa283fa4075d7593 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:05.236476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:05.256875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fca81-5c07-449f-a87e-c0b13784fd27 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:05.439194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:05.441266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:05.441816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:05.442356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] PATH_INFO: `/v3/users/0890d53f29c94926a2aad393c888ca44` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:05.446511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0890d53f29c94926a2aad393c888ca44)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:05.449217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:05.459236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:05.464051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:05.540493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eef0c-dbb7-4bf1-a70b-83eeff543555 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:05.561113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:05.562213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:05.563497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:05.563853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] PATH_INFO: `/v3/users/753144834d1a418897a600589137fcbc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:05.571106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=753144834d1a418897a600589137fcbc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:05.573276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:05.645547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0890d53f29c94926a2aad393c888ca44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.649617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:05.706128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0890d53f29c94926a2aad393c888ca44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.710287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753144834d1a418897a600589137fcbc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.714057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753144834d1a418897a600589137fcbc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.739801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0890d53f29c94926a2aad393c888ca44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.743734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0890d53f29c94926a2aad393c888ca44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.744794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0890d53f29c94926a2aad393c888ca44'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.751278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753144834d1a418897a600589137fcbc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.752465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753144834d1a418897a600589137fcbc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.777105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753144834d1a418897a600589137fcbc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.784867 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3285/6579] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:05 2022] DELETE /identity/v3/users/0890d53f29c94926a2aad393c888ca44 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:05.785833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:05.790642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:05.827970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3295/6580] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:05 2022] DELETE /identity/v3/users/753144834d1a418897a600589137fcbc => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:05.971028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc680c0-4b99-4fac-9ed3-a758265f6958 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.059893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f12856-c2c4-4701-944b-f57bea1015e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.061727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12856-c2c4-4701-944b-f57bea1015e2 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.062072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12856-c2c4-4701-944b-f57bea1015e2 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.062438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12856-c2c4-4701-944b-f57bea1015e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.065238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1f12856-c2c4-4701-944b-f57bea1015e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.220248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f12856-c2c4-4701-944b-f57bea1015e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.222479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f12856-c2c4-4701-944b-f57bea1015e2 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.270920 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3286/6581] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:06.432818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.434315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.503010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.503293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.505931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.538923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72b21-62ba-4bcf-9c7a-d555c0d19423 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.543226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.544169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.544799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.545228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.547372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.562458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.565259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.573044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] MFA Rules not processed for user `7ff5b886208a44c0891c2d1ba65dc083`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:06.576241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.584680 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3296/6582] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:06.599770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:06.693691 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3287/6583] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:06.713273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.715014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.734438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.739352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.777533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a3a36-2f81-4bcc-88d5-da2022aa07b5 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.785715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.786345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.786623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.786903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.805430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.807069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.829349 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3297/6584] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:06.833005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.834759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.868161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d17bc-a274-4da3-aa49-88479b8efd74 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.876446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85113b-81ef-4582-ab5f-f176241942e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.877691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85113b-81ef-4582-ab5f-f176241942e8 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.878935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85113b-81ef-4582-ab5f-f176241942e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.882535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85113b-81ef-4582-ab5f-f176241942e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.885900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f85113b-81ef-4582-ab5f-f176241942e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.896656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bcb4b-0436-4e99-bf28-80bcbb53f50f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.906462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.907112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.907394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.907668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] PATH_INFO: `/v3/projects/bdfb56572d9c4d559ad4fbce7cb6d527` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.908907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfb56572d9c4d559ad4fbce7cb6d527)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.910885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.931150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85113b-81ef-4582-ab5f-f176241942e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.934030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85113b-81ef-4582-ab5f-f176241942e8 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.962564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3298/6585] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:16:07.034797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfb56572d9c4d559ad4fbce7cb6d527', 'request_id': 'req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.044423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfb56572d9c4d559ad4fbce7cb6d527', 'request_id': 'req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.051366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfb56572d9c4d559ad4fbce7cb6d527', 'request_id': 'req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.065533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] The token cache is being invalidate because project bdfb56572d9c4d559ad4fbce7cb6d527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:07.066398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.067318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.070660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3288/6586] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:06 2022] DELETE /identity/v3/projects/bdfb56572d9c4d559ad4fbce7cb6d527 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:07.077438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.077943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.078458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.107709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:07.109498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.187746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.189931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.283835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:07.318033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3299/6587] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:16:07.319169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.324295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.331176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:07.347819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6e53d-962d-4e32-ae2b-ccf08bdf958b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.364692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed679e-c06b-4d1d-b3db-ff858ae8d64d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.374319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.374578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22608d64-fca7-4f31-9d4c-297bd4ff585c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.375282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608d64-fca7-4f31-9d4c-297bd4ff585c service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.375535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.375775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608d64-fca7-4f31-9d4c-297bd4ff585c service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.376105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.376213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608d64-fca7-4f31-9d4c-297bd4ff585c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.376594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] PATH_INFO: `/v3/projects/beb609b6edbd457184bcabb5cb297f0d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.378318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-22608d64-fca7-4f31-9d4c-297bd4ff585c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.378564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb609b6edbd457184bcabb5cb297f0d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.380605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.395963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22608d64-fca7-4f31-9d4c-297bd4ff585c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.398525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22608d64-fca7-4f31-9d4c-297bd4ff585c service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.431040 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3289/6588] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:07.433048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99991-2424-44c4-ba59-11704cff6649 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.435628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99991-2424-44c4-ba59-11704cff6649 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.436032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd99991-2424-44c4-ba59-11704cff6649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.511794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6cd99991-2424-44c4-ba59-11704cff6649 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:07.513115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6cd99991-2424-44c4-ba59-11704cff6649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.515212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb609b6edbd457184bcabb5cb297f0d', 'request_id': 'req-a6ef19be-a803-4fbf-8865-08b1fd2132c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.541068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb609b6edbd457184bcabb5cb297f0d', 'request_id': 'req-a6ef19be-a803-4fbf-8865-08b1fd2132c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.596920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb609b6edbd457184bcabb5cb297f0d', 'request_id': 'req-a6ef19be-a803-4fbf-8865-08b1fd2132c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.628352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] The token cache is being invalidate because project beb609b6edbd457184bcabb5cb297f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:07.630557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.633453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ef19be-a803-4fbf-8865-08b1fd2132c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.643938 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3300/6589] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:07 2022] DELETE /identity/v3/projects/beb609b6edbd457184bcabb5cb297f0d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:07.649741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32a3e7-f6af-4d8c-bda1-f3086aa06022 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.650334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32a3e7-f6af-4d8c-bda1-f3086aa06022 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.650808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32a3e7-f6af-4d8c-bda1-f3086aa06022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.657699 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3290/6590] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:07.660750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd99991-2424-44c4-ba59-11704cff6649 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.675663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9a32a3e7-f6af-4d8c-bda1-f3086aa06022 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:07.680860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9a32a3e7-f6af-4d8c-bda1-f3086aa06022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.716398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd99991-2424-44c4-ba59-11704cff6649 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.722494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3301/6591] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:07.729082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.729812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.730263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.730367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.732408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2aa21-1d10-4dc4-b3a0-d19f0d0734b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.732695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2aa21-1d10-4dc4-b3a0-d19f0d0734b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.732986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2aa21-1d10-4dc4-b3a0-d19f0d0734b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.737184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:16:07.737538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.742593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.780268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-edc2aa21-1d10-4dc4-b3a0-d19f0d0734b1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:07.782065 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3291/6592] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:16:07.782439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-edc2aa21-1d10-4dc4-b3a0-d19f0d0734b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.785033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.796250 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3302/6593] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:07.797489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2aa21-1d10-4dc4-b3a0-d19f0d0734b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.808034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd5b97-7b12-4285-8a9b-d1d401fe4014 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.811253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2aa21-1d10-4dc4-b3a0-d19f0d0734b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.836972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.839511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.839809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.840091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.840842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:16:07.841149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.842837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.844859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.845514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.845799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.847778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.848566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.851245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.875669 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3303/6594] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:16:07.882106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.910307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6318de7-b18e-4738-a6fe-0a4e7809d8cd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.918629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.932630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.933124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.933831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.934597 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3292/6595] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/projects => generated 389 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:16:07.936158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.938458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.940507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.994272 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3304/6596] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:07.999463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.149281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f6380-73ba-4831-843f-a337e096c261 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.158586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab2b0e-d8a4-4d8a-9d1b-7ffcd4769f1a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.165511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.171856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.172294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.172663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.176537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.178762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.185679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.186335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.186618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.187042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.187704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.189927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.236376 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3305/6597] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:08.240193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.249818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3293/6598] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/users => generated 295 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:08.253594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.272284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58cd4-8a5e-41cc-a724-2c5e44d6d840 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.274561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860545b-706b-4d42-b700-e3d4f2f73676 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.290589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.291658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.291658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.294711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.294888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.295663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.295920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.297410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.297930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.298814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.301985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.302849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.335695 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3294/6599] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:16:08.336728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.362720 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3306/6600] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:16:08.363933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.368676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4aace-c959-4d1e-a6a2-4e4a0489349d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.378146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:08.385488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.386173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.386450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.387010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.387836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.392453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71674a7-99b6-4873-b1e7-eaf80d77b6ac admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.392903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.417180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.419422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.419931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.421842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.426880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.428966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.450645 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3295/6601] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:08.453437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.461005 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3307/6602] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:08.464902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.488487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af762559-e56c-461e-90f8-c68b9b1fae5c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.494289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912890b8-4d74-4a0a-bbcb-0589d0cd4709 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.507476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda27a-f71e-45d9-88ea-a9f0975ffc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.509285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda27a-f71e-45d9-88ea-a9f0975ffc4a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.509693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda27a-f71e-45d9-88ea-a9f0975ffc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.510539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda27a-f71e-45d9-88ea-a9f0975ffc4a admin admin] PATH_INFO: `/v3/projects/7ee0ba8c8f5441edbc5ee5b926186df2/users/4431e9b573e44d119f615cd2146e2f0d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.515640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a50f7-5ebe-420b-9466-73dcdd1f1b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.516417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a50f7-5ebe-420b-9466-73dcdd1f1b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.516698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a50f7-5ebe-420b-9466-73dcdd1f1b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.517042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a50f7-5ebe-420b-9466-73dcdd1f1b71 admin admin] PATH_INFO: `/v3/projects/f082a1525e7a400b9248faddccccad57/users/a19f3d81b25a452997230984e774d3a8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.523163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:08.549582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda27a-f71e-45d9-88ea-a9f0975ffc4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee0ba8c8f5441edbc5ee5b926186df2, user_id=4431e9b573e44d119f615cd2146e2f0d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.554420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda27a-f71e-45d9-88ea-a9f0975ffc4a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.567155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a50f7-5ebe-420b-9466-73dcdd1f1b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f082a1525e7a400b9248faddccccad57, user_id=a19f3d81b25a452997230984e774d3a8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.570483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a50f7-5ebe-420b-9466-73dcdd1f1b71 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.586183 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3308/6603] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:08 2022] PUT /identity/v3/projects/7ee0ba8c8f5441edbc5ee5b926186df2/users/4431e9b573e44d119f615cd2146e2f0d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:08.597057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f7d04-a922-44b0-baba-9b158e3ec703 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.597796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f7d04-a922-44b0-baba-9b158e3ec703 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.598286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f7d04-a922-44b0-baba-9b158e3ec703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.617176 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3296/6604] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:08 2022] PUT /identity/v3/projects/f082a1525e7a400b9248faddccccad57/users/a19f3d81b25a452997230984e774d3a8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:08.622657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.622967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.623389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.662666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-052f7d04-a922-44b0-baba-9b158e3ec703 None None] MFA Rules not processed for user `4431e9b573e44d119f615cd2146e2f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:08.664130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-052f7d04-a922-44b0-baba-9b158e3ec703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:08.674027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:08.677505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:08.810122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3297/6605] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:08.852739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.855193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:08.892299 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3309/6606] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:16:08.897007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f7d04-a922-44b0-baba-9b158e3ec703 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.918459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f7d04-a922-44b0-baba-9b158e3ec703 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.929360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.930367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.930651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.930922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.931562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.933514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.043318 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3310/6607] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:16:08 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:16:09.045283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a40742-a43e-4a88-95cf-2c0f80cc2c38 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:09.047429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:09.207682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:09.210347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.210747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.211682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.251827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2f66-cd0d-468e-b21b-b2a5466550de admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:09.265376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:09.266042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.266317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.266585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.267222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:09.270479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.320526 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3311/6608] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:16:09 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:16:09.324267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:09.345040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001accce-b69e-4be3-88a8-f116b7c1be41 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:09.360502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:09.361168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.361444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.361734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.362445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:09.364070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:09.364559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.389854 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3312/6609] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:16:09.390741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:09.391870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:09.394141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.422364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758ce82-84e8-4581-92b9-021954a12a53 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:09.428281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:09.428974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.429542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.429542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] PATH_INFO: `/v3/projects/6b5c774e20314baf93e74a7adaf8db38/users/ea39314a127f4a7184e9af6e835d03c1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.438488 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3298/6610] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:16:09.457034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:09.465393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5c774e20314baf93e74a7adaf8db38, user_id=ea39314a127f4a7184e9af6e835d03c1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:09.471850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.498286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3313/6611] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:09 2022] PUT /identity/v3/projects/6b5c774e20314baf93e74a7adaf8db38/users/ea39314a127f4a7184e9af6e835d03c1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:09.502742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:09.519045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363ceb8-8bca-4009-94c7-b4bc6b7b2e01 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:09.708005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b900ce4-d335-4141-8b11-372a5b1de7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:09.708746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900ce4-d335-4141-8b11-372a5b1de7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.709034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900ce4-d335-4141-8b11-372a5b1de7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.710393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b900ce4-d335-4141-8b11-372a5b1de7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.712207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b900ce4-d335-4141-8b11-372a5b1de7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:09.718164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b900ce4-d335-4141-8b11-372a5b1de7c5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.754480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3299/6612] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:09.758180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:09.772548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b292-05ae-4cb9-86d8-ea9e857b7861 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:09.785781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:09.786475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.786770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.787367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] PATH_INFO: `/v3/projects/6b5c774e20314baf93e74a7adaf8db38/users/ea39314a127f4a7184e9af6e835d03c1/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.789005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5c774e20314baf93e74a7adaf8db38, user_id=ea39314a127f4a7184e9af6e835d03c1, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:09.796466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.835559 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3314/6613] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:09 2022] PUT /identity/v3/projects/6b5c774e20314baf93e74a7adaf8db38/users/ea39314a127f4a7184e9af6e835d03c1/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:09.846180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c5f8-b197-4a85-a703-57d23f90ef26 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.846643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c5f8-b197-4a85-a703-57d23f90ef26 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.846920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206c5f8-b197-4a85-a703-57d23f90ef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.941793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2206c5f8-b197-4a85-a703-57d23f90ef26 None None] MFA Rules not processed for user `ea39314a127f4a7184e9af6e835d03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:09.943353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2206c5f8-b197-4a85-a703-57d23f90ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:10.139170 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3300/6614] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:10.152416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:10.154251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:10.298222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84c6a2-5c5d-49b1-a3a4-5d7e2c1a39e4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:10.304885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a340a-6a05-4f88-9aa4-f84a40d33dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:10.305750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a340a-6a05-4f88-9aa4-f84a40d33dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:10.306030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a340a-6a05-4f88-9aa4-f84a40d33dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:10.306302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a340a-6a05-4f88-9aa4-f84a40d33dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:10.331866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a340a-6a05-4f88-9aa4-f84a40d33dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:10.333382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a340a-6a05-4f88-9aa4-f84a40d33dd3 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:10.373763 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3315/6615] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:11.226821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c5f8-b197-4a85-a703-57d23f90ef26 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:11.378955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206c5f8-b197-4a85-a703-57d23f90ef26 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:11.392705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:11.392705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:11.392705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:11.392705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] PATH_INFO: `/v3/users/4431e9b573e44d119f615cd2146e2f0d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:11.394241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4431e9b573e44d119f615cd2146e2f0d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:11.398169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:11.443542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:11.529381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4431e9b573e44d119f615cd2146e2f0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:11.558052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4431e9b573e44d119f615cd2146e2f0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:11.577567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4431e9b573e44d119f615cd2146e2f0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:11.578484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4431e9b573e44d119f615cd2146e2f0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:11.579252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4431e9b573e44d119f615cd2146e2f0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:11.621563 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3301/6616] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:11 2022] DELETE /identity/v3/users/4431e9b573e44d119f615cd2146e2f0d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:11.623025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:11.623467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:11.623567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:11.674968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:11.676640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:11.852079 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3316/6617] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:16:11.921560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:11.926030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:11.955387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:11.958984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:12.117631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea834ec3-087a-40ee-8c8b-1e8ac691d993 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:12.130659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c0b92-97da-4973-b151-7a68a5d245e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:12.131481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c0b92-97da-4973-b151-7a68a5d245e5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:12.133202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c0b92-97da-4973-b151-7a68a5d245e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:12.134708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c0b92-97da-4973-b151-7a68a5d245e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:12.135392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa14a6-b200-44a0-b55a-54a3cba0113e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:12.147512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:12.151220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:12.152635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:12.153050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:12.159383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:12.168433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c0b92-97da-4973-b151-7a68a5d245e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:12.173870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c0b92-97da-4973-b151-7a68a5d245e5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:12.190268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:12.194474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:12.207581 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3302/6618] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:16:12.227233 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3317/6619] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:13.243019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e159a0-3e83-496f-bb01-c7bfa2939461 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:13.243484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e159a0-3e83-496f-bb01-c7bfa2939461 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:13.243742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e159a0-3e83-496f-bb01-c7bfa2939461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:13.283152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f7e159a0-3e83-496f-bb01-c7bfa2939461 None None] MFA Rules not processed for user `7ff5b886208a44c0891c2d1ba65dc083`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:13.284415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f7e159a0-3e83-496f-bb01-c7bfa2939461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:13.313585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:13.315088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:13.338660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cceb2-331b-4ee0-8510-5067067ce0ef service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:13.352431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:13.353097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:13.353370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:13.353703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] PATH_INFO: `/v3/projects/7ee0ba8c8f5441edbc5ee5b926186df2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:13.361585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee0ba8c8f5441edbc5ee5b926186df2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:13.363731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:13.403500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:13.423319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3303/6620] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:13.467981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee0ba8c8f5441edbc5ee5b926186df2', 'request_id': 'req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.480674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee0ba8c8f5441edbc5ee5b926186df2', 'request_id': 'req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.501715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee0ba8c8f5441edbc5ee5b926186df2', 'request_id': 'req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.560895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] The token cache is being invalidate because project 7ee0ba8c8f5441edbc5ee5b926186df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:13.561198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.564388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.567791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3318/6621] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:13 2022] DELETE /identity/v3/projects/7ee0ba8c8f5441edbc5ee5b926186df2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:13.572067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e159a0-3e83-496f-bb01-c7bfa2939461 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:13.595542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e159a0-3e83-496f-bb01-c7bfa2939461 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:13.660932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:13.733640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:13.735853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:13.736257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:13.736663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] PATH_INFO: `/v3/users/ea39314a127f4a7184e9af6e835d03c1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:13.738164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea39314a127f4a7184e9af6e835d03c1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:13.740661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:13.843467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea39314a127f4a7184e9af6e835d03c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.868308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea39314a127f4a7184e9af6e835d03c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.889001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea39314a127f4a7184e9af6e835d03c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.889679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea39314a127f4a7184e9af6e835d03c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.905671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a53eccbb-7ab2-45b0-9a40-6de750555d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea39314a127f4a7184e9af6e835d03c1'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:13.913548 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3304/6622] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:13 2022] DELETE /identity/v3/users/ea39314a127f4a7184e9af6e835d03c1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:16:15.257682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:15.259263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:15.282624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ac42e-dd30-4251-b0a7-2839231ecbc6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:15.393087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:15.394228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:15.394579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:15.394973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] PATH_INFO: `/v3/projects/6b5c774e20314baf93e74a7adaf8db38` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:15.396357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5c774e20314baf93e74a7adaf8db38)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:15.398865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:15.472260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5c774e20314baf93e74a7adaf8db38', 'request_id': 'req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:15.484629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5c774e20314baf93e74a7adaf8db38', 'request_id': 'req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:15.496759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5c774e20314baf93e74a7adaf8db38', 'request_id': 'req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:15.511285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] The token cache is being invalidate because project 6b5c774e20314baf93e74a7adaf8db38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:15.511653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:15.512516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-17e0b1e4-b02b-45ef-9af1-d0f57fe04401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:15.514794 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3319/6623] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:15 2022] DELETE /identity/v3/projects/6b5c774e20314baf93e74a7adaf8db38 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:15.519467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aeeb3-b02f-4334-9d13-76fc79933206 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:15.520327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aeeb3-b02f-4334-9d13-76fc79933206 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:15.520760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73aeeb3-b02f-4334-9d13-76fc79933206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:15.559330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a73aeeb3-b02f-4334-9d13-76fc79933206 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:15.562435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a73aeeb3-b02f-4334-9d13-76fc79933206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:15.692251 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3305/6624] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:15.694751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a7d26-a916-4ef0-bc09-697361219e6d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:15.695250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a7d26-a916-4ef0-bc09-697361219e6d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:15.695882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a7d26-a916-4ef0-bc09-697361219e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:15.735640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-796a7d26-a916-4ef0-bc09-697361219e6d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:15.741165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-796a7d26-a916-4ef0-bc09-697361219e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:15.762266 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3320/6625] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:15.764084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73aeeb3-b02f-4334-9d13-76fc79933206 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:15.777880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73aeeb3-b02f-4334-9d13-76fc79933206 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:15.785584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:15.786472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:15.786907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:15.787401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:15.788280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:16:15.788777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:15.791794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:15.810005 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3306/6626] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:16:15.814346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a7d26-a916-4ef0-bc09-697361219e6d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:15.829804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a7d26-a916-4ef0-bc09-697361219e6d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:15.834947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:15.835729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:15.836157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:15.836556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:15.837285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:15.839598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:15.901054 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3321/6627] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:16:15 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:15.901989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:15.916371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adaf75c-7ae5-4a94-a55b-33485c9e88e8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:15.988672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:16.016962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:16.017625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:16.019063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:16.019367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:16.021410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:16.023432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:16.075766 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3307/6628] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:16:15 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:16:16.080347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:16.091418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8bc2-c14e-4ab0-b005-6358477ae687 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:16.096702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:16.098776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:16.098999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:16.099376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:16.100150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:16.102052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:16.104073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:16.120460 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3322/6629] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:16.121657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:16.135368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb1dcc-713f-42a8-8918-a3a186f53436 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:16.140786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400560e4-0399-41db-8d0a-aa0da93c5aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:16.141370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400560e4-0399-41db-8d0a-aa0da93c5aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:16.141930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400560e4-0399-41db-8d0a-aa0da93c5aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:16.142352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400560e4-0399-41db-8d0a-aa0da93c5aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:16.143241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400560e4-0399-41db-8d0a-aa0da93c5aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:16.144845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400560e4-0399-41db-8d0a-aa0da93c5aa7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:16.157264 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3308/6630] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:16:16.161286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:16.172370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f6b7-a9d1-4971-9d1e-34ba3670ad1e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:16.185006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:16.185776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:16.186052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:16.186322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] PATH_INFO: `/v3/projects/3fe15d4415344341b9d9d8f7694cddc3/users/262593ad1bf34506a4d7f73bf682c7d1/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:16.210593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe15d4415344341b9d9d8f7694cddc3, user_id=262593ad1bf34506a4d7f73bf682c7d1, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:16.214664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:16.245548 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3323/6631] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:16 2022] PUT /identity/v3/projects/3fe15d4415344341b9d9d8f7694cddc3/users/262593ad1bf34506a4d7f73bf682c7d1/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:16.254608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a2a00-436d-49ae-a9e8-8ffe4bc79af2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:16.255344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a2a00-436d-49ae-a9e8-8ffe4bc79af2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:16.256102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a2a00-436d-49ae-a9e8-8ffe4bc79af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:16.307114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4d9a2a00-436d-49ae-a9e8-8ffe4bc79af2 None None] MFA Rules not processed for user `262593ad1bf34506a4d7f73bf682c7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:16.309193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4d9a2a00-436d-49ae-a9e8-8ffe4bc79af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:16.479406 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3309/6632] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:16:16.493494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:16.495303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:16.699672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab3f0-b24a-4b30-b45c-ce79e5f8a803 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:16.707298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:16.708450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:16.708874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:16.709314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:16.725532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:16.727553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:16.759984 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3324/6633] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:20.820760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95df1-cd5d-43d6-a0ba-59f82cb814d3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:20.821315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95df1-cd5d-43d6-a0ba-59f82cb814d3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:20.821315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95df1-cd5d-43d6-a0ba-59f82cb814d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:20.875418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4ba95df1-cd5d-43d6-a0ba-59f82cb814d3 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:20.878957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4ba95df1-cd5d-43d6-a0ba-59f82cb814d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:21.078604 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3310/6634] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:16:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:21.095499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:21.097840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:21.283750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b782d-14bc-4f8f-b360-97bbbbfc7edd service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:21.290104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea582801-45f2-4cf7-828e-4164fd5572ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:21.291069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea582801-45f2-4cf7-828e-4164fd5572ac service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:21.291555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea582801-45f2-4cf7-828e-4164fd5572ac service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:21.292057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea582801-45f2-4cf7-828e-4164fd5572ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:21.316118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea582801-45f2-4cf7-828e-4164fd5572ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:21.318124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea582801-45f2-4cf7-828e-4164fd5572ac service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:21.370805 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3325/6635] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:16:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:16:22.857256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:22.857772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:22.857850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:22.924325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] MFA Rules not processed for user `a19f3d81b25a452997230984e774d3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:22.925629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:23.071075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:23.146956 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3311/6636] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:23.158347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:23.159366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:23.159790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:23.218759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] MFA Rules not processed for user `a19f3d81b25a452997230984e774d3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:23.223399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:23.260547 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3326/6637] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:23.335789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:23.337418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:23.508620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d687f-6d02-47e7-a76b-3a09f6d4a94b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:23.514286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:23.515126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:23.515542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:23.516097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:23.518722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:23.539560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:23.541982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:23.601835 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3312/6638] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:23.718096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:23.719652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:23.877665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:23.944357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b9e3-92f8-426a-9207-107f881efb63 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:23.960734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:23.964760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:23.965213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:23.965671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:24.002778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:24.005878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:24.031686 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3327/6639] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:26.803854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:26.806133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:26.841917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17ac74-5278-44ab-89ce-9f6242629951 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:26.872165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4224f0d-492f-4620-99cb-a8516cb91e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:26.872973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4224f0d-492f-4620-99cb-a8516cb91e6b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:26.873383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4224f0d-492f-4620-99cb-a8516cb91e6b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:26.873778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4224f0d-492f-4620-99cb-a8516cb91e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:26.880050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4224f0d-492f-4620-99cb-a8516cb91e6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:26.911262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4224f0d-492f-4620-99cb-a8516cb91e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:26.914593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4224f0d-492f-4620-99cb-a8516cb91e6b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:26.964043 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3313/6640] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:33.071326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:33.252310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e036e71-6a19-4f7c-ac8b-02f3c7c2c817 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:33.260764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:33.261699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:33.262074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:33.262439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] PATH_INFO: `/v3/users/262593ad1bf34506a4d7f73bf682c7d1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:33.264928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=262593ad1bf34506a4d7f73bf682c7d1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:33.269038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:33.380423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262593ad1bf34506a4d7f73bf682c7d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:33.381274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262593ad1bf34506a4d7f73bf682c7d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:33.405585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262593ad1bf34506a4d7f73bf682c7d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:33.406409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262593ad1bf34506a4d7f73bf682c7d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:33.451744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262593ad1bf34506a4d7f73bf682c7d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:33.496499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3328/6641] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:33 2022] DELETE /identity/v3/users/262593ad1bf34506a4d7f73bf682c7d1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:33.505523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:33.505523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:33.505523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:33.556921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:33.558470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:33.783055 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3314/6642] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:33.961391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:33.963179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:33.990679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:33.992101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:34.161930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff0c22-a66d-4f80-b46e-8e1af7854442 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:34.170395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:34.172224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:34.173321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:34.174701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:34.202274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:34.204212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:34.227734 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3329/6643] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:34.239102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1555897-c9cd-4109-bd99-68acc381da41 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:34.254016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e4a7a-79fe-4b48-b93d-9e9c32657eaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:34.254824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e4a7a-79fe-4b48-b93d-9e9c32657eaa service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:34.255681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e4a7a-79fe-4b48-b93d-9e9c32657eaa service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:34.256090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e4a7a-79fe-4b48-b93d-9e9c32657eaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:34.263215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1e8e4a7a-79fe-4b48-b93d-9e9c32657eaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:34.301206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e4a7a-79fe-4b48-b93d-9e9c32657eaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:34.305108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e4a7a-79fe-4b48-b93d-9e9c32657eaa service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:34.331862 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3315/6644] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:36.575221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:36.577017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:36.594441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:36.605911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685a7a5-44df-45f7-bd88-b39dc069da8b service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:36.619461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:36.621869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:36.622461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:36.622746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] PATH_INFO: `/v3/projects/3fe15d4415344341b9d9d8f7694cddc3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:36.624928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe15d4415344341b9d9d8f7694cddc3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:36.631470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:36.784119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe15d4415344341b9d9d8f7694cddc3', 'request_id': 'req-13aaae42-9bd8-4aa4-a747-21ca9400e782'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:36.797095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe15d4415344341b9d9d8f7694cddc3', 'request_id': 'req-13aaae42-9bd8-4aa4-a747-21ca9400e782'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:36.820606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe15d4415344341b9d9d8f7694cddc3', 'request_id': 'req-13aaae42-9bd8-4aa4-a747-21ca9400e782'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:36.839245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] The token cache is being invalidate because project 3fe15d4415344341b9d9d8f7694cddc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:36.839534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:36.844438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13aaae42-9bd8-4aa4-a747-21ca9400e782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:36.848773 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3330/6645] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:36 2022] DELETE /identity/v3/projects/3fe15d4415344341b9d9d8f7694cddc3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:36.854787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e305b-c9b7-422a-ad3f-1aa15d7b0957 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:36.855389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e305b-c9b7-422a-ad3f-1aa15d7b0957 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:36.855810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e305b-c9b7-422a-ad3f-1aa15d7b0957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:36.872341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:36.920948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9c1e305b-c9b7-422a-ad3f-1aa15d7b0957 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:36.934706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9c1e305b-c9b7-422a-ad3f-1aa15d7b0957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:37.139323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3316/6646] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:37.141903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999da64-8467-489e-b321-10219a2ca0f3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.141903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999da64-8467-489e-b321-10219a2ca0f3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:37.141903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999da64-8467-489e-b321-10219a2ca0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:37.189703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a999da64-8467-489e-b321-10219a2ca0f3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:37.191041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a999da64-8467-489e-b321-10219a2ca0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:37.234223 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3331/6647] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:37.235459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e305b-c9b7-422a-ad3f-1aa15d7b0957 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:37.250157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e305b-c9b7-422a-ad3f-1aa15d7b0957 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:37.264009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:37.264767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.266155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:37.266595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:37.267442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:16:37.268120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:37.270290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:37.284226 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3317/6648] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:16:37.289022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999da64-8467-489e-b321-10219a2ca0f3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:37.302952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999da64-8467-489e-b321-10219a2ca0f3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:37.312345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:37.313200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.313670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:37.314099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:37.315522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:37.319592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:37.378667 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3332/6649] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:16:37 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:37.379994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:37.396331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad149a6-6bc6-4213-8f18-da5008745636 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:37.524908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:37.525575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.526708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:37.527783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:37.530374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:37.535321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:37.636439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:37.640243 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3318/6650] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:16:37 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:37.652958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bfdc2-ad49-430d-baf0-b6ecce90262f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:37.700012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:37.700657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.700933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:37.701204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:37.708279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:37.711163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:37.759179 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3333/6651] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:16:37.760530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:37.783680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ddebda-c966-4171-9a5e-f176eb0e8db9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:37.805006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000724e-75e1-4597-b37f-30837b23d4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:37.810968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000724e-75e1-4597-b37f-30837b23d4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.811468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000724e-75e1-4597-b37f-30837b23d4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:37.811882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000724e-75e1-4597-b37f-30837b23d4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:37.812716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000724e-75e1-4597-b37f-30837b23d4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:37.817999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000724e-75e1-4597-b37f-30837b23d4f5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:37.853057 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3319/6652] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:37.859123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:37.874773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4ba7e-3f40-4103-917a-375ae5a20ba1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:37.882056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:37.883461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.883884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:37.884390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] PATH_INFO: `/v3/projects/09f16a815901466791a114b9a19cc763/users/df6cec0e04954a3eba096b48a7cd2189/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:37.927859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f16a815901466791a114b9a19cc763, user_id=df6cec0e04954a3eba096b48a7cd2189, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:37.930733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:37.978370 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3334/6653] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:37 2022] PUT /identity/v3/projects/09f16a815901466791a114b9a19cc763/users/df6cec0e04954a3eba096b48a7cd2189/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:37.983835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c874f-f837-4fe3-ad6a-10f72bc8e8cb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:37.999514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c874f-f837-4fe3-ad6a-10f72bc8e8cb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:38.002453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c874f-f837-4fe3-ad6a-10f72bc8e8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:38.034790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4c3c874f-f837-4fe3-ad6a-10f72bc8e8cb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:38.046631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4c3c874f-f837-4fe3-ad6a-10f72bc8e8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:38.176416 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3320/6654] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:38.194911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:38.197557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:38.343657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bb278-d365-48e7-a3bc-9663960b9ab6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:38.357187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:38.358154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:38.358525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:38.358902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:38.376159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:38.378130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:38.397237 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3335/6655] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:44.218338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebf417-5505-4b0a-b5bb-8d3a258a8ca1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:44.218875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebf417-5505-4b0a-b5bb-8d3a258a8ca1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:44.218954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebf417-5505-4b0a-b5bb-8d3a258a8ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:44.243332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-49ebf417-5505-4b0a-b5bb-8d3a258a8ca1 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:44.244986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49ebf417-5505-4b0a-b5bb-8d3a258a8ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:44.252359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:44.409554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3321/6656] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:44.430595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:44.432424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:44.497519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:44.568505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224d654-d885-423a-b6b1-b684ee15b0a1 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:44.577329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:44.578248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:44.578703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:44.579172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:44.589803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:44.591870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:44.601609 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3336/6657] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:16:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:16:54.378354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=21 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:54.462762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f365c-f4c9-48f6-bf1c-585fdb371f18 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:54.463072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f365c-f4c9-48f6-bf1c-585fdb371f18 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:54.463351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f365c-f4c9-48f6-bf1c-585fdb371f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:54.520170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-359f365c-f4c9-48f6-bf1c-585fdb371f18 None None] MFA Rules not processed for user `df6cec0e04954a3eba096b48a7cd2189`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:54.521440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-359f365c-f4c9-48f6-bf1c-585fdb371f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:54.597406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=20 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:54.640458 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3322/6658] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:54.644849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:54.685983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5ac83-28bf-4b8a-8c74-1e688b3deb7d service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:54.702755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:54.703531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:54.705451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:54.705783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:54.706553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:54.709341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:54.776337 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3337/6659] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:16:54 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:54.780483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f365c-f4c9-48f6-bf1c-585fdb371f18 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:54.799970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f365c-f4c9-48f6-bf1c-585fdb371f18 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:54.965178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:54.966215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:54.966635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:54.967082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:54.967873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:54.971196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:55.013387 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3323/6660] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:16:54 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:55.017342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:55.044251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f17ea-5258-4ac2-9f8e-1a5b332d2519 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:55.062391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:55.064634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:55.066521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:55.067543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:55.078334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:55.083887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:55.116659 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3338/6661] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:55.123994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:55.136235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864816-8874-4bfb-aae1-3b3a2c55a50e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:55.153969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:55.154844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:55.156321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:55.156676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] PATH_INFO: `/v3/projects/350d95806ad747439a4745ee873235d8/users/e5cff400a3ce426abe10ebc16d8c1bca/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:55.200824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350d95806ad747439a4745ee873235d8, user_id=e5cff400a3ce426abe10ebc16d8c1bca, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:55.208397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:55.240383 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3324/6662] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:55 2022] PUT /identity/v3/projects/350d95806ad747439a4745ee873235d8/users/e5cff400a3ce426abe10ebc16d8c1bca/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:55.244398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:55.259591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851e9af-e780-4ce9-b0c0-6e571aa95acb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:55.336046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace45090-af4b-4da7-a93d-c54ac7a582e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:55.336280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace45090-af4b-4da7-a93d-c54ac7a582e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:55.336514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace45090-af4b-4da7-a93d-c54ac7a582e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:55.336789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace45090-af4b-4da7-a93d-c54ac7a582e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:55.337449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace45090-af4b-4da7-a93d-c54ac7a582e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:55.339837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace45090-af4b-4da7-a93d-c54ac7a582e4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:55.355486 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3339/6663] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:55.356816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:55.369447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cda081-bb51-423d-8c1c-699920198191 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:55.381144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:55.381874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:55.382158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:55.382435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] PATH_INFO: `/v3/projects/350d95806ad747439a4745ee873235d8/users/e5cff400a3ce426abe10ebc16d8c1bca/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:55.409135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=350d95806ad747439a4745ee873235d8, user_id=e5cff400a3ce426abe10ebc16d8c1bca, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:55.422757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:55.438211 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3325/6664] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:55 2022] PUT /identity/v3/projects/350d95806ad747439a4745ee873235d8/users/e5cff400a3ce426abe10ebc16d8c1bca/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:05.348368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:05.432444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:06.073155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c521e3-7843-44fd-8500-d2f35ab8e202 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:06.073439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c521e3-7843-44fd-8500-d2f35ab8e202 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:06.073937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c521e3-7843-44fd-8500-d2f35ab8e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:06.138596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f5c521e3-7843-44fd-8500-d2f35ab8e202 None None] MFA Rules not processed for user `e5cff400a3ce426abe10ebc16d8c1bca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:06.139977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f5c521e3-7843-44fd-8500-d2f35ab8e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:06.236835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3340/6665] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:06.252194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:06.254229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:06.396812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2df7da-b766-4e4a-b5b3-a844068aa0ee admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:06.404084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:06.405252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:06.405735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:06.406169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:06.488278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:06.490830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:06.505609 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3326/6666] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:09.698192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917df9b-c531-41bd-bc9c-9d76464e78ac None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:09.698632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917df9b-c531-41bd-bc9c-9d76464e78ac None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:09.698737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917df9b-c531-41bd-bc9c-9d76464e78ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:09.741648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4917df9b-c531-41bd-bc9c-9d76464e78ac None None] MFA Rules not processed for user `df6cec0e04954a3eba096b48a7cd2189`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:09.745963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4917df9b-c531-41bd-bc9c-9d76464e78ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:09.766330 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3341/6667] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:17:16.498106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=56 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:19.724199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=37 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:23.520415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:23.522673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:23.628897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf872e0-62da-4e79-acd6-c18331fdb007 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:23.634945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:23.635653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:23.636083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:23.636530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] PATH_INFO: `/v3/users/6cee1c1229d847d284c4a5c9fbb754bd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:23.637990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cee1c1229d847d284c4a5c9fbb754bd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:23.640227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:23.716434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cee1c1229d847d284c4a5c9fbb754bd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:23.738413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cee1c1229d847d284c4a5c9fbb754bd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:23.754390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cee1c1229d847d284c4a5c9fbb754bd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:23.755504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cee1c1229d847d284c4a5c9fbb754bd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:23.756723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cee1c1229d847d284c4a5c9fbb754bd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:23.779604 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3327/6668] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:23 2022] DELETE /identity/v3/users/6cee1c1229d847d284c4a5c9fbb754bd => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:23.781362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:23.781920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:23.781920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:23.816865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:23.818512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:23.910689 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3342/6669] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:23.945092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:23.946820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:24.033647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763c5ac-d14d-4f08-bd03-c2d1fca58444 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:24.043924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec84235-647b-424a-8b58-abe49e2459d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:24.044105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec84235-647b-424a-8b58-abe49e2459d2 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:24.044739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec84235-647b-424a-8b58-abe49e2459d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:24.049308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec84235-647b-424a-8b58-abe49e2459d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:24.063280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec84235-647b-424a-8b58-abe49e2459d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:24.066301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec84235-647b-424a-8b58-abe49e2459d2 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:24.081432 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3328/6670] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:24.587078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:24.588816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:24.602669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffd54a-9720-40b7-bfab-dfea9b5e34a9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:24.611201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:24.612274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:24.612545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:24.613144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] PATH_INFO: `/v3/projects/0420c3486c474bc9a1ad1b7dcacf4e85` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:24.614694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0420c3486c474bc9a1ad1b7dcacf4e85)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:24.617223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:24.703405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0420c3486c474bc9a1ad1b7dcacf4e85', 'request_id': 'req-83b6de0e-4234-404a-95d4-60fed30f7db7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:24.719662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0420c3486c474bc9a1ad1b7dcacf4e85', 'request_id': 'req-83b6de0e-4234-404a-95d4-60fed30f7db7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:24.743735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0420c3486c474bc9a1ad1b7dcacf4e85', 'request_id': 'req-83b6de0e-4234-404a-95d4-60fed30f7db7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:24.764286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] The token cache is being invalidate because project 0420c3486c474bc9a1ad1b7dcacf4e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:24.764982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:24.766170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-83b6de0e-4234-404a-95d4-60fed30f7db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:24.772196 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3343/6671] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:24 2022] DELETE /identity/v3/projects/0420c3486c474bc9a1ad1b7dcacf4e85 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:24.777433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acec37-183a-42b2-9f88-ad2f52146a9d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:24.777961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acec37-183a-42b2-9f88-ad2f52146a9d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:24.778039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acec37-183a-42b2-9f88-ad2f52146a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:24.802816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-33acec37-183a-42b2-9f88-ad2f52146a9d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:24.804058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-33acec37-183a-42b2-9f88-ad2f52146a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:24.836861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:24.940893 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3329/6672] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:24.950319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ae55c-97b9-4680-8186-f1b1445dd30c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:24.951855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ae55c-97b9-4680-8186-f1b1445dd30c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:24.952344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ae55c-97b9-4680-8186-f1b1445dd30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.046550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-230ae55c-97b9-4680-8186-f1b1445dd30c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:25.048004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-230ae55c-97b9-4680-8186-f1b1445dd30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:25.073918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3344/6673] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:25.078421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acec37-183a-42b2-9f88-ad2f52146a9d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.097030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acec37-183a-42b2-9f88-ad2f52146a9d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.123731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.124684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.133705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.134161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.135596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:17:25.136954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.139047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.186414 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3330/6674] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:17:25.188058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ae55c-97b9-4680-8186-f1b1445dd30c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.202685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ae55c-97b9-4680-8186-f1b1445dd30c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.226886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.226886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.226886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.227652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.228590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.230611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.315627 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3345/6675] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:25 2022] POST /identity/v3/projects => generated 359 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:17:25.316533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.340296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f89b8a-1925-4904-8679-9db220b6c080 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.423357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.423982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.424264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.424529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.425168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.427203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.451212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3331/6676] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:25 2022] POST /identity/v3/users => generated 280 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:17:25.455215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.467174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab279c9-b16c-4297-bc81-795c364b5e27 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.490928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.492334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.492873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.493283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.494172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.496639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.537678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3346/6677] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:17:25.539011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.553619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44624fb-3a32-441b-9afc-22fde0ff9fd8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.561024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22023aeb-f807-4bb1-bcb0-017bd883c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.561477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22023aeb-f807-4bb1-bcb0-017bd883c9ae admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.561860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22023aeb-f807-4bb1-bcb0-017bd883c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.562116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22023aeb-f807-4bb1-bcb0-017bd883c9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.562776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22023aeb-f807-4bb1-bcb0-017bd883c9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.565059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22023aeb-f807-4bb1-bcb0-017bd883c9ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.577985 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3332/6678] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:17:25.582064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.592670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a9321-b7c1-48f9-9b1c-255d8cca1cbc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.598520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.599493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.599980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.600424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] PATH_INFO: `/v3/projects/b6ce06ccadc8438caa71573ab66a41c8/users/8570867020db4f99aa5a49a8e44b3b5b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.618708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ce06ccadc8438caa71573ab66a41c8, user_id=8570867020db4f99aa5a49a8e44b3b5b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.622516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.640603 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3347/6679] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:25 2022] PUT /identity/v3/projects/b6ce06ccadc8438caa71573ab66a41c8/users/8570867020db4f99aa5a49a8e44b3b5b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:25.651361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.651661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.652218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.690877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] MFA Rules not processed for user `8570867020db4f99aa5a49a8e44b3b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:25.692143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:25.790198 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3333/6680] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:25.801541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.803572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:25.838984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:25.923224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89723a1f-dec4-4570-bfb9-e97f6192c637 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.927775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.928661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.929203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.929710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.945101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.947239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.967105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3348/6681] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:26.032526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:26.034465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:26.063408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:26.127878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393eec9-ff31-45a5-bab2-5e1d4c8716b3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:26.133836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:26.134649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:26.134934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:26.135208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] PATH_INFO: `/v3/users/a19f3d81b25a452997230984e774d3a8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:26.136553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19f3d81b25a452997230984e774d3a8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:26.138702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:26.208930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19f3d81b25a452997230984e774d3a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:26.224030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19f3d81b25a452997230984e774d3a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:26.239025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19f3d81b25a452997230984e774d3a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:26.239758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19f3d81b25a452997230984e774d3a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:26.241357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-226d55ba-2b5a-4c0b-84da-2534a12e1beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19f3d81b25a452997230984e774d3a8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:26.263891 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3334/6682] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:26 2022] DELETE /identity/v3/users/a19f3d81b25a452997230984e774d3a8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:27.248450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.266404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e190f-b0b0-4455-a9ff-7f3f64e5c89c service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:27.373674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:27.374637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.375063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.375622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] PATH_INFO: `/v3/projects/f082a1525e7a400b9248faddccccad57` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.376946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f082a1525e7a400b9248faddccccad57)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:27.379276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:27.454307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f082a1525e7a400b9248faddccccad57', 'request_id': 'req-1596463f-4b7d-4869-92a7-2be8e48d6373'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.461126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f082a1525e7a400b9248faddccccad57', 'request_id': 'req-1596463f-4b7d-4869-92a7-2be8e48d6373'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.474062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f082a1525e7a400b9248faddccccad57', 'request_id': 'req-1596463f-4b7d-4869-92a7-2be8e48d6373'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.491169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] The token cache is being invalidate because project f082a1525e7a400b9248faddccccad57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:27.492560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.493599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1596463f-4b7d-4869-92a7-2be8e48d6373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.497935 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3349/6683] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:27 2022] DELETE /identity/v3/projects/f082a1525e7a400b9248faddccccad57 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:27.502832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.503677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.504113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.535038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:27.536674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:27.645088 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3335/6684] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:27.653620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1d87-9dd2-47b8-8189-f58dc9b4f52b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.654149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1d87-9dd2-47b8-8189-f58dc9b4f52b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.654424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1d87-9dd2-47b8-8189-f58dc9b4f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.704044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.712621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:27.736459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9c4f1d87-9dd2-47b8-8189-f58dc9b4f52b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:27.737737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c4f1d87-9dd2-47b8-8189-f58dc9b4f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:27.764381 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3350/6685] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:27.768235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1d87-9dd2-47b8-8189-f58dc9b4f52b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.787294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1d87-9dd2-47b8-8189-f58dc9b4f52b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:27.792708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:27.793590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.793947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.794223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.794894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:17:27.798596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:27.802535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:27.844211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:27.847039 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3351/6686] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:17:27.851630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.866837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3ec52-c29e-44c7-b93e-7e7983f95f85 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:27.868020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94524821-16ab-4ef1-b9bf-f1e3f68fbaec None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:27.872646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:27.873358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.873773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.874137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.875088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:27.876107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:27.876812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.877100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.877376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.878103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:27.879976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:27.944844 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3352/6687] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:27 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:27.950058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.965325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:27.975611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:27.977557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:27.991608 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3336/6688] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:28.032939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52954469-35e9-4cda-8e82-99a175b8b0e1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.037780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.038621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.038954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.039287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.039991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.042029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.071453 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3353/6689] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:27 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:28.075276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.087538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389c0d0-358d-41e7-b756-2ade348d0fe2 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.092561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.093809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.094165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.094534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.095922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.097764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.110679 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3337/6690] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:17:28.114700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.126596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efe577-bd5c-4cac-b25d-c46578521e91 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.132262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959e74-da6e-409a-8e32-2fe084b49335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.132996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959e74-da6e-409a-8e32-2fe084b49335 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.133394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959e74-da6e-409a-8e32-2fe084b49335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.133764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959e74-da6e-409a-8e32-2fe084b49335 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.134417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75959e74-da6e-409a-8e32-2fe084b49335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.136065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75959e74-da6e-409a-8e32-2fe084b49335 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.148345 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3354/6691] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:28.154304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.166393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067c2e9-7365-4655-bf60-2cbf2268f3f2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.171529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.172349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.172637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.172913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] PATH_INFO: `/v3/projects/4c0548e01f2249a98e28b618ca3ab8cc/users/47f724dca5d5409d9a3a02289c286d89/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.207812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0548e01f2249a98e28b618ca3ab8cc, user_id=47f724dca5d5409d9a3a02289c286d89, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.213929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.242189 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3338/6692] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:28 2022] PUT /identity/v3/projects/4c0548e01f2249a98e28b618ca3ab8cc/users/47f724dca5d5409d9a3a02289c286d89/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:28.248431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b446-1ced-43d3-a4eb-889f286e39a0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.248877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b446-1ced-43d3-a4eb-889f286e39a0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.249568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b446-1ced-43d3-a4eb-889f286e39a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.297151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9968b446-1ced-43d3-a4eb-889f286e39a0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:28.298634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9968b446-1ced-43d3-a4eb-889f286e39a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.434160 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3355/6693] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:28.528863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.531831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.676685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc43dae-ec58-44c5-b5b6-d31d988e91f8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.681506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.682620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.682843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.683386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.694791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.696444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.708053 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3339/6694] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:34.633615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ffe61-0dc3-44fd-86bc-7acd6d5d54dd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:34.634192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ffe61-0dc3-44fd-86bc-7acd6d5d54dd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:34.634294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ffe61-0dc3-44fd-86bc-7acd6d5d54dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:34.675832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-030ffe61-0dc3-44fd-86bc-7acd6d5d54dd None None] MFA Rules not processed for user `8ddb1d0a7bca457493d7ac9a58a01e64`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:34.678029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-030ffe61-0dc3-44fd-86bc-7acd6d5d54dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:34.829415 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3356/6695] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:36.005577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:36.016593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb06a9f-d085-4059-a29a-fb4142c3b406 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:36.020611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070ef8-2aa2-4ae8-914e-e184018d84b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:36.021498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070ef8-2aa2-4ae8-914e-e184018d84b4 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:36.021498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070ef8-2aa2-4ae8-914e-e184018d84b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:36.021812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070ef8-2aa2-4ae8-914e-e184018d84b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:36.024052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d0070ef8-2aa2-4ae8-914e-e184018d84b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:36.039259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0070ef8-2aa2-4ae8-914e-e184018d84b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:36.041442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0070ef8-2aa2-4ae8-914e-e184018d84b4 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:36.060782 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3340/6696] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:42.469399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:42.470223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:42.470638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:42.526071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] MFA Rules not processed for user `47f724dca5d5409d9a3a02289c286d89`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:42.528608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:42.607038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:42.700755 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3357/6697] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:42.713605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:42.714537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:42.715139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:42.765409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] MFA Rules not processed for user `47f724dca5d5409d9a3a02289c286d89`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:42.771812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:42.798376 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3341/6698] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:42.813132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:42.818315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:42.994583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a408-d4de-4b2f-afdb-6a8d42aeb1ba None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:43.003017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb8a72-4cfe-4fb9-b864-13af03e36b8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:43.003933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb8a72-4cfe-4fb9-b864-13af03e36b8b service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:43.004347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb8a72-4cfe-4fb9-b864-13af03e36b8b service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:43.004762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb8a72-4cfe-4fb9-b864-13af03e36b8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:43.017159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb8a72-4cfe-4fb9-b864-13af03e36b8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:43.020204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb8a72-4cfe-4fb9-b864-13af03e36b8b service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:43.033567 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3358/6699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:43.116912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:43.119799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:43.165537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d0c15-dc06-4107-98f5-7b80b96db17c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:43.176626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:43.176626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:43.176929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:43.180256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] PATH_INFO: `/v3/users/8570867020db4f99aa5a49a8e44b3b5b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:43.211257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8570867020db4f99aa5a49a8e44b3b5b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:43.216484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:43.290693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:43.316439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8570867020db4f99aa5a49a8e44b3b5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:43.333105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8570867020db4f99aa5a49a8e44b3b5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:43.343577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8570867020db4f99aa5a49a8e44b3b5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:43.344202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8570867020db4f99aa5a49a8e44b3b5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:43.344932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8570867020db4f99aa5a49a8e44b3b5b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:43.368251 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3342/6700] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:43 2022] DELETE /identity/v3/users/8570867020db4f99aa5a49a8e44b3b5b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:43.373695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:43.374040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:43.374387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:43.411309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:43.412631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:43.545993 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3359/6701] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:43.578682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:43.581031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:43.785883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f4a94-c61f-473f-abcf-a300f2cf5bbe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:43.797796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60b21b-2e23-4fd0-ab34-f0f377680096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:43.798872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b21b-2e23-4fd0-ab34-f0f377680096 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:43.799366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b21b-2e23-4fd0-ab34-f0f377680096 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:43.799852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b21b-2e23-4fd0-ab34-f0f377680096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:43.820854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60b21b-2e23-4fd0-ab34-f0f377680096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:43.823492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60b21b-2e23-4fd0-ab34-f0f377680096 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:43.835494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3343/6702] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:44.384072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:44.385540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:44.414288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6753e-d24c-4376-a8d5-4f59b35ccbde None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:44.428292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:44.429417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:44.432971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:44.433481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] PATH_INFO: `/v3/projects/b6ce06ccadc8438caa71573ab66a41c8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:44.437026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ce06ccadc8438caa71573ab66a41c8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:44.442125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:44.578547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:44.585241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ce06ccadc8438caa71573ab66a41c8', 'request_id': 'req-6917579b-db3e-4021-ae34-e1b46788eaf6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:44.597770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ce06ccadc8438caa71573ab66a41c8', 'request_id': 'req-6917579b-db3e-4021-ae34-e1b46788eaf6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:44.611389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ce06ccadc8438caa71573ab66a41c8', 'request_id': 'req-6917579b-db3e-4021-ae34-e1b46788eaf6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:44.636533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] The token cache is being invalidate because project b6ce06ccadc8438caa71573ab66a41c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:44.637279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:44.638578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6917579b-db3e-4021-ae34-e1b46788eaf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:44.640919 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3360/6703] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:44 2022] DELETE /identity/v3/projects/b6ce06ccadc8438caa71573ab66a41c8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:44.646078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f372b16-c7e2-4406-b89f-4aebbfb9cf3b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:44.646328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f372b16-c7e2-4406-b89f-4aebbfb9cf3b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:44.646544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f372b16-c7e2-4406-b89f-4aebbfb9cf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:44.689734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8f372b16-c7e2-4406-b89f-4aebbfb9cf3b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:44.691554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f372b16-c7e2-4406-b89f-4aebbfb9cf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:44.801585 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3344/6704] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:44.906485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe1805-d784-437a-8091-8aac71f51dd2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:44.906981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe1805-d784-437a-8091-8aac71f51dd2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:44.907400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe1805-d784-437a-8091-8aac71f51dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:44.935240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-36fe1805-d784-437a-8091-8aac71f51dd2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:44.936520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-36fe1805-d784-437a-8091-8aac71f51dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:44.950298 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3361/6705] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:17:44.951586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f372b16-c7e2-4406-b89f-4aebbfb9cf3b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:44.962336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f372b16-c7e2-4406-b89f-4aebbfb9cf3b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:44.968425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:44.969187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:44.969643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:44.970106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:44.971027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:17:44.971470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:44.973237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:44.992049 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3345/6706] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:17:44.992761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe1805-d784-437a-8091-8aac71f51dd2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:45.004413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe1805-d784-437a-8091-8aac71f51dd2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:45.021912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:45.022581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:45.022893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:45.023440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:45.024234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:45.031211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:45.108116 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3362/6707] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:44 2022] POST /identity/v3/projects => generated 403 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:45.112182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:45.125104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf201558-2481-4b70-89c3-646a90224473 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:45.207961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:45.264194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:45.269283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:45.270014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:45.271914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:45.275496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:45.294944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:45.353181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3346/6708] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:45 2022] POST /identity/v3/users => generated 302 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:45.354639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:45.395229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62c100-a6cb-49db-b8b7-c35581d5af7b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:45.422224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:45.423022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:45.423401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:45.423770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:45.424530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:45.436628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:45.471463 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3363/6709] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:45.475272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:45.487219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c13650-40ff-45c5-b313-f19df1be2767 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:45.492657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318d5c0-6212-4b98-84c4-120c58ffbc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:45.493601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318d5c0-6212-4b98-84c4-120c58ffbc19 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:45.494027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318d5c0-6212-4b98-84c4-120c58ffbc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:45.494465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318d5c0-6212-4b98-84c4-120c58ffbc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:45.495642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318d5c0-6212-4b98-84c4-120c58ffbc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:45.497671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318d5c0-6212-4b98-84c4-120c58ffbc19 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:45.517779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:45.526605 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3347/6710] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:45.531844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0e6ec-6132-47cd-bf7f-090448bb6bb8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:45.539187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:45.540131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:45.540666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:45.541105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] PATH_INFO: `/v3/projects/05f158fe61e9475e855736af927d93fe/users/4bfc437f520043ddad68e14d108b0718/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:45.860045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f158fe61e9475e855736af927d93fe, user_id=4bfc437f520043ddad68e14d108b0718, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:45.863227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:45.890610 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3364/6711] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:45 2022] PUT /identity/v3/projects/05f158fe61e9475e855736af927d93fe/users/4bfc437f520043ddad68e14d108b0718/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:45.895555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5fad-6f15-45ec-988b-df72a3cf23b7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:45.896382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5fad-6f15-45ec-988b-df72a3cf23b7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:45.896802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5fad-6f15-45ec-988b-df72a3cf23b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:45.948838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0fdb5fad-6f15-45ec-988b-df72a3cf23b7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:45.950163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0fdb5fad-6f15-45ec-988b-df72a3cf23b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:46.083919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3348/6712] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:17:46.094060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:46.097368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:46.180591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988b640-a4ed-44c5-a0df-d9bc3b99ca7c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:46.185656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:46.187204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:46.187426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:46.187602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:46.211103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:46.212884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:46.226248 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3365/6713] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:49.591662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff0af5-b5f7-412a-a305-212ce18fa3fb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:49.592185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff0af5-b5f7-412a-a305-212ce18fa3fb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:49.592185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff0af5-b5f7-412a-a305-212ce18fa3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:49.626733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2fff0af5-b5f7-412a-a305-212ce18fa3fb None None] MFA Rules not processed for user `4bfc437f520043ddad68e14d108b0718`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:49.628198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2fff0af5-b5f7-412a-a305-212ce18fa3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:49.762558 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3349/6714] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:49.767348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:49.782668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16fa01-52dc-4fc0-80e2-1f2e80280082 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:49.789485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:49.791086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:49.791399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:49.791669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:49.792486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:49.795789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:49.837046 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3366/6715] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:49 2022] POST /identity/v3/projects => generated 401 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:49.841629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff0af5-b5f7-412a-a305-212ce18fa3fb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:49.859544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff0af5-b5f7-412a-a305-212ce18fa3fb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:49.964480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:49.998765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:49.999728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:50.000118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:50.000498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:50.001274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:50.004064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:50.065904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3350/6716] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:49 2022] POST /identity/v3/users => generated 307 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:50.069605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:50.095292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734366b-fc20-4456-bae6-e74fb572b7d0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:50.102769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:50.103034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:50.103361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:50.103548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:50.104244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:50.106074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:50.119102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3367/6717] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:17:50.124001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:50.143534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e762-c301-48b7-9267-c3b29830ff46 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:50.151868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:50.152923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:50.153396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:50.154131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] PATH_INFO: `/v3/projects/d4dbf5a79a9f4277a19a3d904f117fe5/users/5416c1a38cff45f09b872db36565946a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:50.180743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4dbf5a79a9f4277a19a3d904f117fe5, user_id=5416c1a38cff45f09b872db36565946a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:50.187560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:50.228221 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3351/6718] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:50 2022] PUT /identity/v3/projects/d4dbf5a79a9f4277a19a3d904f117fe5/users/5416c1a38cff45f09b872db36565946a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:50.230117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:50.248795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b3f7-07e3-43a5-9697-58c65b58ef7c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:50.337416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:50.373246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70795512-676e-4e6d-8f19-d7d4acc8b6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:50.374600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70795512-676e-4e6d-8f19-d7d4acc8b6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:50.374894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70795512-676e-4e6d-8f19-d7d4acc8b6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:50.375165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70795512-676e-4e6d-8f19-d7d4acc8b6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:50.376839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70795512-676e-4e6d-8f19-d7d4acc8b6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:50.377951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70795512-676e-4e6d-8f19-d7d4acc8b6b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:50.394361 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3368/6719] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:50.394866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:50.427580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205e68-6d22-4144-baf8-fe7ce31901be admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:50.434295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:50.435188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:50.435622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:50.436547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] PATH_INFO: `/v3/projects/d4dbf5a79a9f4277a19a3d904f117fe5/users/5416c1a38cff45f09b872db36565946a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:50.438812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4dbf5a79a9f4277a19a3d904f117fe5, user_id=5416c1a38cff45f09b872db36565946a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:50.442335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:50.455792 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3352/6720] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:50 2022] PUT /identity/v3/projects/d4dbf5a79a9f4277a19a3d904f117fe5/users/5416c1a38cff45f09b872db36565946a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:52.655635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:52.656140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:52.656221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:52.697830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] MFA Rules not processed for user `5416c1a38cff45f09b872db36565946a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:52.699624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:52.800215 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3369/6721] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:54.076063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:54.162999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd584a06-3191-4572-8404-d6d8f58bd0f7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:54.171281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:54.172184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.172625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.173030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] PATH_INFO: `/v3/users/4bfc437f520043ddad68e14d108b0718` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:54.174347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bfc437f520043ddad68e14d108b0718)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:54.176383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:54.234427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:54.238359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:54.239757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bfc437f520043ddad68e14d108b0718'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.258849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bfc437f520043ddad68e14d108b0718'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.268517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bfc437f520043ddad68e14d108b0718'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.270003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bfc437f520043ddad68e14d108b0718'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.270437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5ab518b2-2e1f-426d-9960-4c341d157cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bfc437f520043ddad68e14d108b0718'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.279710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710f82e-8ca0-4162-b303-aa43d0a5a0a2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:54.288318 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3353/6722] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:54 2022] DELETE /identity/v3/users/4bfc437f520043ddad68e14d108b0718 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:54.288509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:54.288509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.288509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.288509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] PATH_INFO: `/v3/users/62b8f1ec47d04fd4a0ae80c363d41e9d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:54.288880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b8f1ec47d04fd4a0ae80c363d41e9d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:54.290766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:54.352540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b8f1ec47d04fd4a0ae80c363d41e9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.353200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b8f1ec47d04fd4a0ae80c363d41e9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.364119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b8f1ec47d04fd4a0ae80c363d41e9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.364881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b8f1ec47d04fd4a0ae80c363d41e9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.378612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b8f1ec47d04fd4a0ae80c363d41e9d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.394652 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3370/6723] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:54 2022] DELETE /identity/v3/users/62b8f1ec47d04fd4a0ae80c363d41e9d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:54.398817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.400867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.400867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:54.420748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:54.422247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:54.435796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:54.479171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:54.480917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:54.509105 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3354/6724] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:54.518487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:54.519961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:54.546994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:54.584214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a554785-35b6-4f08-a6ad-9fb2251b8ff0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:54.584440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fe70d-584d-4202-a70e-281ca7c3efd6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:54.589329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:54.590086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.590477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.590577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:54.590921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:54.591130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.591461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.591738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] PATH_INFO: `/v3/projects/05f158fe61e9475e855736af927d93fe` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:54.592880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f158fe61e9475e855736af927d93fe)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:54.594900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:54.602434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:54.604066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:54.614263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3355/6725] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:17:54.654964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f158fe61e9475e855736af927d93fe', 'request_id': 'req-681a6be8-f759-466d-9b56-dbd293967529'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.661664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f158fe61e9475e855736af927d93fe', 'request_id': 'req-681a6be8-f759-466d-9b56-dbd293967529'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.674720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f158fe61e9475e855736af927d93fe', 'request_id': 'req-681a6be8-f759-466d-9b56-dbd293967529'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.693014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] The token cache is being invalidate because project 05f158fe61e9475e855736af927d93fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:54.693347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.694134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.695735 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3371/6726] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:54 2022] DELETE /identity/v3/projects/05f158fe61e9475e855736af927d93fe => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:54.699829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:54.710288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee043d4-b020-42e7-96e4-446e5b42cffb service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:54.781989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:54.782727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.783068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.783284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] PATH_INFO: `/v3/users/5416c1a38cff45f09b872db36565946a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:54.784492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5416c1a38cff45f09b872db36565946a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:54.786554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:54.809980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:54.811460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:54.837856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a6be8-f759-466d-9b56-dbd293967529 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:54.850373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5416c1a38cff45f09b872db36565946a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.852340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:54.852999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.853288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.853591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] PATH_INFO: `/v3/projects/ad306136dc654559a74a42717663c547` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:54.854814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad306136dc654559a74a42717663c547)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:54.856836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:54.865256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:54.867385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5416c1a38cff45f09b872db36565946a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.881186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5416c1a38cff45f09b872db36565946a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.882300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5416c1a38cff45f09b872db36565946a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.882967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-eb474d0d-b6c3-4250-8544-d3d1bd9094e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5416c1a38cff45f09b872db36565946a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.898795 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3356/6727] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:54 2022] DELETE /identity/v3/users/5416c1a38cff45f09b872db36565946a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:54.924259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad306136dc654559a74a42717663c547', 'request_id': 'req-cb64568d-b528-4ed6-94dd-4e09b630d63d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.932982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad306136dc654559a74a42717663c547', 'request_id': 'req-cb64568d-b528-4ed6-94dd-4e09b630d63d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.947101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad306136dc654559a74a42717663c547', 'request_id': 'req-cb64568d-b528-4ed6-94dd-4e09b630d63d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.964121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] The token cache is being invalidate because project ad306136dc654559a74a42717663c547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:54.964605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.966629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-cb64568d-b528-4ed6-94dd-4e09b630d63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:54.971875 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3372/6728] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:54 2022] DELETE /identity/v3/projects/ad306136dc654559a74a42717663c547 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:54.978952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea19f0c-b4d9-400c-ac6b-25b4085595c4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:54.979321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea19f0c-b4d9-400c-ac6b-25b4085595c4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:54.980120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea19f0c-b4d9-400c-ac6b-25b4085595c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.002386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0ea19f0c-b4d9-400c-ac6b-25b4085595c4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:55.003706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ea19f0c-b4d9-400c-ac6b-25b4085595c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:55.101513 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3357/6729] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:17:55.199956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955522-579d-4d4e-86e3-f239c6d1633e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.200347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955522-579d-4d4e-86e3-f239c6d1633e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.200705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955522-579d-4d4e-86e3-f239c6d1633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.232681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fb955522-579d-4d4e-86e3-f239c6d1633e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:55.234506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fb955522-579d-4d4e-86e3-f239c6d1633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:55.254676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3373/6730] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:55.259285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea19f0c-b4d9-400c-ac6b-25b4085595c4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.281913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea19f0c-b4d9-400c-ac6b-25b4085595c4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.294164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.295049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.295476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.295943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.296780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:17:55.297642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.299758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.314685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3358/6731] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:17:55.319085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955522-579d-4d4e-86e3-f239c6d1633e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.332550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.335391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955522-579d-4d4e-86e3-f239c6d1633e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.338526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:55.347211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.348088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.350071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.352181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.353050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.374561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.388078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c414e-bfbd-461c-9d44-19bfb72b1d68 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.407546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.407671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.408137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.408137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] PATH_INFO: `/v3/projects/d4dbf5a79a9f4277a19a3d904f117fe5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.411130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4dbf5a79a9f4277a19a3d904f117fe5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.413166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.444278 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3374/6732] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:55.450014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.522721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:55.534832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4dbf5a79a9f4277a19a3d904f117fe5', 'request_id': 'req-44f59f56-f884-43d9-9226-ca03c94e7e97'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:55.547888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4dbf5a79a9f4277a19a3d904f117fe5', 'request_id': 'req-44f59f56-f884-43d9-9226-ca03c94e7e97'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:55.557567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4dbf5a79a9f4277a19a3d904f117fe5', 'request_id': 'req-44f59f56-f884-43d9-9226-ca03c94e7e97'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:55.572389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] The token cache is being invalidate because project d4dbf5a79a9f4277a19a3d904f117fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:55.572812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:55.573600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-44f59f56-f884-43d9-9226-ca03c94e7e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:55.576774 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3359/6733] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:55 2022] DELETE /identity/v3/projects/d4dbf5a79a9f4277a19a3d904f117fe5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:55.581378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabbed-218b-43a4-bf96-f6591b0941ac None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.584227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabbed-218b-43a4-bf96-f6591b0941ac None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.584583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabbed-218b-43a4-bf96-f6591b0941ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.585667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044559fd-39cc-4eb1-954b-77128fa03948 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.610425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-28cabbed-218b-43a4-bf96-f6591b0941ac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:55.612000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28cabbed-218b-43a4-bf96-f6591b0941ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:55.684852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.685757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.686047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.693572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.694347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.699855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.702471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:55.731965 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3360/6734] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:17:55.743140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32dcd-2cea-4e54-998e-c0b2581d4e57 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.743140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32dcd-2cea-4e54-998e-c0b2581d4e57 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.743626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32dcd-2cea-4e54-998e-c0b2581d4e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.745398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3375/6735] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:55.749874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.755354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:55.781123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aff32dcd-2cea-4e54-998e-c0b2581d4e57 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:55.782479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aff32dcd-2cea-4e54-998e-c0b2581d4e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:55.784673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546789-2a7b-43ef-a69a-091b2c09ae19 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.792099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.792862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.793253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.793425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3361/6736] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:55.794500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.795478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.797991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32dcd-2cea-4e54-998e-c0b2581d4e57 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.798233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.809821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32dcd-2cea-4e54-998e-c0b2581d4e57 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.813251 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3376/6737] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:55.815384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.816993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.817608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.818402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.818682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.820877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:17:55.821338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.823803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.830208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3a5a1-6ad1-4031-b545-bb86dddbe37f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.836178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.837543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.838212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.838579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.839330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.841121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.843513 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3362/6738] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:17:55.847776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.857356 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3377/6739] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:55.861244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.862650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3ade7-fc86-4cc6-b96d-62ccea975e38 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.868971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.869624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.870267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.870666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.871311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.874657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.876802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc516502-083d-4e5e-b938-9fc609fdb90f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:55.883823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45b0b8-aa75-4e25-985c-ce50e5881c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:55.884838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45b0b8-aa75-4e25-985c-ce50e5881c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.885212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45b0b8-aa75-4e25-985c-ce50e5881c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.885619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45b0b8-aa75-4e25-985c-ce50e5881c97 admin admin] PATH_INFO: `/v3/projects/38ef85f9e05f46cbbdd7bcb54442f999/users/053b5600c4d04027ab66e6d4f27b0908/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.906887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45b0b8-aa75-4e25-985c-ce50e5881c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ef85f9e05f46cbbdd7bcb54442f999, user_id=053b5600c4d04027ab66e6d4f27b0908, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:55.909868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45b0b8-aa75-4e25-985c-ce50e5881c97 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:55.927434 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3378/6740] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:55 2022] PUT /identity/v3/projects/38ef85f9e05f46cbbdd7bcb54442f999/users/053b5600c4d04027ab66e6d4f27b0908/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:55.937305 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3363/6741] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:55.938480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:55.938764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:55.939031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:55.942862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:55.969756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] MFA Rules not processed for user `053b5600c4d04027ab66e6d4f27b0908`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:55.970973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:56.038495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bf3c2-bae8-4ea0-b032-469c5086d010 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.043910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.044528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.044869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.045073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.045879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.047800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.054268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:56.062845 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3379/6742] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:56.081514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3364/6743] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:55 2022] POST /identity/v3/users => generated 285 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:56.084597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.086184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.087716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:56.100221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2788-915c-4067-9750-962f3efdf7dd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.107888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.109184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.109847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.110272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.111584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.114077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.126702 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3365/6744] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:56.131519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.143994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2188b6e-de58-4467-ab17-f4ee4966e6af admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.149927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.150749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.151210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.151750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.152553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.154655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.168124 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3366/6745] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:56.171984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.182533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e28ec1-af14-47a5-9524-2137a807136b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.184742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c54e68-216f-48d1-81c7-4677a11a6706 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.187016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75fa77-e499-449b-bc47-dd764f4d6aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.187650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75fa77-e499-449b-bc47-dd764f4d6aff service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.187928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75fa77-e499-449b-bc47-dd764f4d6aff service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.188204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75fa77-e499-449b-bc47-dd764f4d6aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.190642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.191492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.191927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.192342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] PATH_INFO: `/v3/projects/619f5a74ff14456ca0ed6b3662f24743/users/1aaef87967bb4d57932cc8e126fae507/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.200157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75fa77-e499-449b-bc47-dd764f4d6aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.201841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75fa77-e499-449b-bc47-dd764f4d6aff service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.210416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=619f5a74ff14456ca0ed6b3662f24743, user_id=1aaef87967bb4d57932cc8e126fae507, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.212646 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3380/6746] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:56.213604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.231144 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3367/6747] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:56 2022] PUT /identity/v3/projects/619f5a74ff14456ca0ed6b3662f24743/users/1aaef87967bb4d57932cc8e126fae507/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:56.241689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6c2bc-56ac-4e74-8ba1-dbab63cdae4e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.242273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6c2bc-56ac-4e74-8ba1-dbab63cdae4e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.242273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6c2bc-56ac-4e74-8ba1-dbab63cdae4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.276091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-34f6c2bc-56ac-4e74-8ba1-dbab63cdae4e None None] MFA Rules not processed for user `1aaef87967bb4d57932cc8e126fae507`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:56.278467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-34f6c2bc-56ac-4e74-8ba1-dbab63cdae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:56.364075 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3381/6748] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:56.368716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.453287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:56.472891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341ba51-aaa1-45fa-96e4-8d25125b80f4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.492725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.493717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.494359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.494731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.496556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.499263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.552494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3368/6749] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:56 2022] POST /identity/v3/projects => generated 367 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:56.557269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6c2bc-56ac-4e74-8ba1-dbab63cdae4e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.577823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6c2bc-56ac-4e74-8ba1-dbab63cdae4e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.683971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.685103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.685660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.689851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.690749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.693984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.725333 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3382/6750] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:56 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:56.730199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.742436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d26e6-3a59-4a8a-8d0c-d2fb41cda3ae admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.749651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.750392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.750710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.751015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.751747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.753760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.766482 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3369/6751] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:17:56.771298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.784447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78b5b3-8a03-44dc-821b-ee2ac24c5157 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.790684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.791272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.791575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.792021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] PATH_INFO: `/v3/projects/a56faa6232ec48f785e648969785ecb4/users/da6b50032c30464d99cf69d113acc5c8/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.811534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56faa6232ec48f785e648969785ecb4, user_id=da6b50032c30464d99cf69d113acc5c8, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.815370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.819403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:56.834975 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3383/6752] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:56 2022] PUT /identity/v3/projects/a56faa6232ec48f785e648969785ecb4/users/da6b50032c30464d99cf69d113acc5c8/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:56.838638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.851600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4079d-d91a-47a0-9a5e-0d2b65a107c6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.932116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27162fb-41d0-4e54-a47e-4adc8f587bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.932748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27162fb-41d0-4e54-a47e-4adc8f587bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.933044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27162fb-41d0-4e54-a47e-4adc8f587bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.933297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27162fb-41d0-4e54-a47e-4adc8f587bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.933986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27162fb-41d0-4e54-a47e-4adc8f587bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.935611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27162fb-41d0-4e54-a47e-4adc8f587bd0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:56.957621 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3370/6753] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:17:56.961786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:56.975152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4417992-615e-4adc-a469-238d656afc50 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:56.991955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:56.992855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:56.993132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:56.993404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] PATH_INFO: `/v3/projects/a56faa6232ec48f785e648969785ecb4/users/da6b50032c30464d99cf69d113acc5c8/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:56.996215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56faa6232ec48f785e648969785ecb4, user_id=da6b50032c30464d99cf69d113acc5c8, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:56.999111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:57.018281 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3384/6754] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:56 2022] PUT /identity/v3/projects/a56faa6232ec48f785e648969785ecb4/users/da6b50032c30464d99cf69d113acc5c8/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:57.029934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f58ff4-03ac-44b7-9fca-9eea8080b1a4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:57.030292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f58ff4-03ac-44b7-9fca-9eea8080b1a4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:57.030482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f58ff4-03ac-44b7-9fca-9eea8080b1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:57.064446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-87f58ff4-03ac-44b7-9fca-9eea8080b1a4 None None] MFA Rules not processed for user `da6b50032c30464d99cf69d113acc5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:57.067031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87f58ff4-03ac-44b7-9fca-9eea8080b1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:57.158146 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3371/6755] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:57.169762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:57.171831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:57.239936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b647d2f-9eda-40d9-af23-4d90d0569258 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:57.244165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933242d7-16e3-48ca-b206-ef94651c3e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:57.246073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933242d7-16e3-48ca-b206-ef94651c3e2d service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:57.246514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933242d7-16e3-48ca-b206-ef94651c3e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:57.246931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933242d7-16e3-48ca-b206-ef94651c3e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:57.258170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933242d7-16e3-48ca-b206-ef94651c3e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:57.259926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933242d7-16e3-48ca-b206-ef94651c3e2d service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:57.283597 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3385/6756] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:58.893307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f58ff4-03ac-44b7-9fca-9eea8080b1a4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:58.960377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:58.976189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f58ff4-03ac-44b7-9fca-9eea8080b1a4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:58.982118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:58.983500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:58.983998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:58.984505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] PATH_INFO: `/v3/users/1aaef87967bb4d57932cc8e126fae507` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:58.986281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aaef87967bb4d57932cc8e126fae507)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:58.989091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:59.051156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aaef87967bb4d57932cc8e126fae507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:59.064525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aaef87967bb4d57932cc8e126fae507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:59.074428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aaef87967bb4d57932cc8e126fae507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:59.075410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aaef87967bb4d57932cc8e126fae507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:59.076039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aaef87967bb4d57932cc8e126fae507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:59.093065 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3372/6757] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:58 2022] DELETE /identity/v3/users/1aaef87967bb4d57932cc8e126fae507 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:59.094916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:59.095407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:59.095532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:59.116709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:59.118255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:59.187959 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3386/6758] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:17:59.203302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:59.204732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:59.289690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45201c73-e9d8-4a1f-adef-b4b0ecb8c70d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:59.293710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:59.294642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:59.294760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:59.295090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:59.307696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:59.309404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:59.320587 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3373/6759] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:00.082935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:00.084541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:00.098288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badb792-54c7-4789-9cc0-b03e0618d874 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:00.103996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:00.104785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:00.105122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:00.105662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] PATH_INFO: `/v3/projects/619f5a74ff14456ca0ed6b3662f24743` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:00.106982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=619f5a74ff14456ca0ed6b3662f24743)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:00.109208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:00.168909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '619f5a74ff14456ca0ed6b3662f24743', 'request_id': 'req-544c734c-290f-43da-aeed-a788a25837bc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.175689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '619f5a74ff14456ca0ed6b3662f24743', 'request_id': 'req-544c734c-290f-43da-aeed-a788a25837bc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.185537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '619f5a74ff14456ca0ed6b3662f24743', 'request_id': 'req-544c734c-290f-43da-aeed-a788a25837bc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.201644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] The token cache is being invalidate because project 619f5a74ff14456ca0ed6b3662f24743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:00.201863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.202839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.204999 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3387/6760] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:00 2022] DELETE /identity/v3/projects/619f5a74ff14456ca0ed6b3662f24743 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:00.208706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:00.220016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af054b32-572b-4fd5-a333-cab5c0ac987b service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:00.337764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:00.339912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:00.340402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:00.341189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] PATH_INFO: `/v3/users/da6b50032c30464d99cf69d113acc5c8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:00.343352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6b50032c30464d99cf69d113acc5c8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:00.346611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:00.367624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:00.425431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6b50032c30464d99cf69d113acc5c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.448869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6b50032c30464d99cf69d113acc5c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.461489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6b50032c30464d99cf69d113acc5c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.462113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6b50032c30464d99cf69d113acc5c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.462703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-62664dc9-0f3f-4072-99e3-1d40005b0b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6b50032c30464d99cf69d113acc5c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:00.475098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3374/6761] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:00 2022] DELETE /identity/v3/users/da6b50032c30464d99cf69d113acc5c8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:00.930615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:00.932076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:00.950295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c734c-290f-43da-aeed-a788a25837bc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:00.955285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:01.011894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.012544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.012837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.013125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] PATH_INFO: `/v3/projects/a56faa6232ec48f785e648969785ecb4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.014341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56faa6232ec48f785e648969785ecb4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.016310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.072076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56faa6232ec48f785e648969785ecb4', 'request_id': 'req-b4a39865-aa35-47ee-98cb-c67158faf964'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:01.079059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56faa6232ec48f785e648969785ecb4', 'request_id': 'req-b4a39865-aa35-47ee-98cb-c67158faf964'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:01.087087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56faa6232ec48f785e648969785ecb4', 'request_id': 'req-b4a39865-aa35-47ee-98cb-c67158faf964'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:01.102180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] The token cache is being invalidate because project a56faa6232ec48f785e648969785ecb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:01.103288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:01.104006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b4a39865-aa35-47ee-98cb-c67158faf964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:01.105938 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3388/6762] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:00 2022] DELETE /identity/v3/projects/a56faa6232ec48f785e648969785ecb4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:01.111182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c441-4d16-485b-b593-e831b3b95d81 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.111634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c441-4d16-485b-b593-e831b3b95d81 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.112070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c441-4d16-485b-b593-e831b3b95d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.141964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-28a5c441-4d16-485b-b593-e831b3b95d81 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:01.143281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-28a5c441-4d16-485b-b593-e831b3b95d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:01.218502 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3375/6763] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:01.227707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b918-1cec-478b-a2ff-6116279fbcb5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.228300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b918-1cec-478b-a2ff-6116279fbcb5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.228300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b918-1cec-478b-a2ff-6116279fbcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.252510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9919b918-1cec-478b-a2ff-6116279fbcb5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:01.254107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9919b918-1cec-478b-a2ff-6116279fbcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:01.265501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3389/6764] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:01.268805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c441-4d16-485b-b593-e831b3b95d81 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.282182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c441-4d16-485b-b593-e831b3b95d81 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.288324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.289172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.289478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.289755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.290415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:01.290753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.292573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.307808 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3376/6765] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:01.312040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919b918-1cec-478b-a2ff-6116279fbcb5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.323242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919b918-1cec-478b-a2ff-6116279fbcb5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.330498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.331127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.331648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.331648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.332261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.334457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.341076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.343031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:01.370976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3390/6766] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:01 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:01.376739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.448628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f18c4c-afc8-4a88-99ea-5289df516f22 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.484238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6baba-86eb-4c63-801e-dee540acd791 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.491190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.491873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.492149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.492421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.493112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.495713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.520293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:01.527807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:01.533836 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3391/6767] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:01 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:01.538407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.556947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b4eed-79fd-4819-b1c2-c6d8a135a800 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.559394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.560403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.560894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.561657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.565495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:01.572874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.573549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.573803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.574071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.574726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.576412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.580848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.583767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.593387 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3392/6768] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:01.597390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.605042 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3377/6769] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:01.610121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2192-f531-4964-9c1d-7dc2f2327f55 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.615747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4cd17-07d0-4d42-b3da-807e644d3c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.616479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4cd17-07d0-4d42-b3da-807e644d3c0f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.616963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4cd17-07d0-4d42-b3da-807e644d3c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.617353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4cd17-07d0-4d42-b3da-807e644d3c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.618187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4cd17-07d0-4d42-b3da-807e644d3c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.620230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4cd17-07d0-4d42-b3da-807e644d3c0f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.631676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3393/6770] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:01.636167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.649253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f0710-bd2c-40ce-80ef-26fe202a9d00 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.654881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.654881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.655130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.655362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] PATH_INFO: `/v3/projects/c81aa9c09de54cbb9103d6b8bc6442ad/users/28cd2509195a4093b08e9b4dbdd8e931/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.672729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81aa9c09de54cbb9103d6b8bc6442ad, user_id=28cd2509195a4093b08e9b4dbdd8e931, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.675663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:01.694930 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3378/6771] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:01 2022] PUT /identity/v3/projects/c81aa9c09de54cbb9103d6b8bc6442ad/users/28cd2509195a4093b08e9b4dbdd8e931/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:01.705950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb19396-0038-42c1-953b-f3361956bc00 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.706286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb19396-0038-42c1-953b-f3361956bc00 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.706593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb19396-0038-42c1-953b-f3361956bc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.741327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bfb19396-0038-42c1-953b-f3361956bc00 None None] MFA Rules not processed for user `28cd2509195a4093b08e9b4dbdd8e931`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:01.753485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bfb19396-0038-42c1-953b-f3361956bc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:01.870089 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3394/6772] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:01.874721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:01.955958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b6af-e06d-44ae-95c6-237e4090d30b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:01.961073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:01.961724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:01.962003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:01.962298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:01.963069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:01.967356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.008924 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3379/6773] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:01 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:02.013147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb19396-0038-42c1-953b-f3361956bc00 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.025440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb19396-0038-42c1-953b-f3361956bc00 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.086761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.087553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.087894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.088170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.088801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.091015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.124527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3395/6774] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:02 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:18:02.128765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.141499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173201f-2eb9-41b5-9dc3-e0f8ab4f4cc3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.146756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.147558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.147907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.148771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.149032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.150932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.164091 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3380/6775] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:02.168375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.185891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233029-a60e-44c9-a0b0-53705bdd45b1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.200482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698e6ae-0a76-48a2-b868-95281eb78d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.201177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698e6ae-0a76-48a2-b868-95281eb78d9e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.202433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698e6ae-0a76-48a2-b868-95281eb78d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.202801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698e6ae-0a76-48a2-b868-95281eb78d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.203682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698e6ae-0a76-48a2-b868-95281eb78d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.205602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698e6ae-0a76-48a2-b868-95281eb78d9e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.216854 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3396/6776] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:02.221201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.232221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293239b0-a72d-4ab4-a255-949caf16f483 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.237953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0895c-b316-4437-8a49-6da0c2d92e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.238539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0895c-b316-4437-8a49-6da0c2d92e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.238973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0895c-b316-4437-8a49-6da0c2d92e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.239255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0895c-b316-4437-8a49-6da0c2d92e0d admin admin] PATH_INFO: `/v3/projects/9471aaa9e2fe4ec083e8fb3b4655ddf8/users/a9e718ab890446479270e32876c97d62/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.256006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0895c-b316-4437-8a49-6da0c2d92e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9471aaa9e2fe4ec083e8fb3b4655ddf8, user_id=a9e718ab890446479270e32876c97d62, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.258863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0895c-b316-4437-8a49-6da0c2d92e0d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.285482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa40ac-00d5-4f51-8d87-ad14d1e3f4b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.286325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa40ac-00d5-4f51-8d87-ad14d1e3f4b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.286948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa40ac-00d5-4f51-8d87-ad14d1e3f4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.296918 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3381/6777] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:02 2022] PUT /identity/v3/projects/9471aaa9e2fe4ec083e8fb3b4655ddf8/users/a9e718ab890446479270e32876c97d62/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:02.316625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7afa40ac-00d5-4f51-8d87-ad14d1e3f4b1 None None] MFA Rules not processed for user `47f724dca5d5409d9a3a02289c286d89`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:02.318237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7afa40ac-00d5-4f51-8d87-ad14d1e3f4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:02.323067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6d6e-9f6e-4fc9-b937-51fe64e1d650 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.323418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6d6e-9f6e-4fc9-b937-51fe64e1d650 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.323758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6d6e-9f6e-4fc9-b937-51fe64e1d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.357005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ba9e6d6e-9f6e-4fc9-b937-51fe64e1d650 None None] MFA Rules not processed for user `a9e718ab890446479270e32876c97d62`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:02.358719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ba9e6d6e-9f6e-4fc9-b937-51fe64e1d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:02.373138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:02.392218 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3397/6778] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:02.414508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:02.434252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3382/6779] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:02.442367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa40ac-00d5-4f51-8d87-ad14d1e3f4b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.523398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa40ac-00d5-4f51-8d87-ad14d1e3f4b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.530205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.531108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.531547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.531975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.532772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.535020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.577099 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3398/6780] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:02.577962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e6d6e-9f6e-4fc9-b937-51fe64e1d650 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.591852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e6d6e-9f6e-4fc9-b937-51fe64e1d650 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.669990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.670726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.671102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.671569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.672275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.674656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.713866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.726644 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3383/6781] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:02 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:02.730155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfcae6-6df4-424f-9212-6c5ebee520a7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.735811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.736591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.737860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.738305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.739070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.740974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.762465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.775716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3399/6782] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:02.778311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e48f3e-2841-49f8-802f-701a7e5eb9c2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.784102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.785424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.785831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.786201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] PATH_INFO: `/v3/projects/ee1bf0dd63be4bb9a9d0083d0b1fdeac/users/23b099b3ad3649e1bfd04a5e2a787c0e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.814985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1bf0dd63be4bb9a9d0083d0b1fdeac, user_id=23b099b3ad3649e1bfd04a5e2a787c0e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.818209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.843633 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3384/6783] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:02 2022] PUT /identity/v3/projects/ee1bf0dd63be4bb9a9d0083d0b1fdeac/users/23b099b3ad3649e1bfd04a5e2a787c0e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:18:02.848709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.865914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3333ad-eed8-462b-afc2-aca75a0022ae admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.942244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62879a8-3fe1-4f53-a8e3-ba32a2cbaec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.942856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62879a8-3fe1-4f53-a8e3-ba32a2cbaec3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.943152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62879a8-3fe1-4f53-a8e3-ba32a2cbaec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.943420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62879a8-3fe1-4f53-a8e3-ba32a2cbaec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.944057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62879a8-3fe1-4f53-a8e3-ba32a2cbaec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.945969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62879a8-3fe1-4f53-a8e3-ba32a2cbaec3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.960231 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3400/6784] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:02.962011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:02.973072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10f28-b5b1-4565-b98f-2b3e149af1ea admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:02.978031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:02.978685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:02.978965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:02.979283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] PATH_INFO: `/v3/projects/ee1bf0dd63be4bb9a9d0083d0b1fdeac/users/23b099b3ad3649e1bfd04a5e2a787c0e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:02.981344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1bf0dd63be4bb9a9d0083d0b1fdeac, user_id=23b099b3ad3649e1bfd04a5e2a787c0e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:02.984061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:02.998324 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3385/6785] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:02 2022] PUT /identity/v3/projects/ee1bf0dd63be4bb9a9d0083d0b1fdeac/users/23b099b3ad3649e1bfd04a5e2a787c0e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:03.008685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:03.008971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:03.009244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:03.042459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] MFA Rules not processed for user `23b099b3ad3649e1bfd04a5e2a787c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:03.043886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:03.095830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:03.123965 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3401/6786] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:03.138550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:03.140974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:03.220699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcedc1e-93d0-40bc-8c49-f7b692d8db4a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:03.225572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:03.226684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:03.227055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:03.227331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:03.238941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:03.241307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:03.264873 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3386/6787] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:03.886789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:03.890769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:03.905181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c9702-f583-4b37-815c-5df47cb436b4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:03.909703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:03.910865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:03.911664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:03.911664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:03.978172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:03.979997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:03.991323 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3402/6788] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:08.963758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:08.990062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a308045-e4ac-4a5a-adaf-22bb2fb72abf service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:08.998318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e5e0a-d03b-4c14-be8b-df8fb7e05915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:08.999948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e5e0a-d03b-4c14-be8b-df8fb7e05915 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:09.000346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e5e0a-d03b-4c14-be8b-df8fb7e05915 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:09.000598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e5e0a-d03b-4c14-be8b-df8fb7e05915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:09.064570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:09.089627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e5e0a-d03b-4c14-be8b-df8fb7e05915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:09.091219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e5e0a-d03b-4c14-be8b-df8fb7e05915 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:09.101425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3387/6789] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:11.476603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:11.578508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e4f93-9b72-449f-bf4b-b366ea2d3391 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:11.583469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:11.584112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:11.584390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:11.585205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] PATH_INFO: `/v3/users/28cd2509195a4093b08e9b4dbdd8e931` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:11.586533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28cd2509195a4093b08e9b4dbdd8e931)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:11.589377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:11.650405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28cd2509195a4093b08e9b4dbdd8e931'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.651123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28cd2509195a4093b08e9b4dbdd8e931'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.659921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28cd2509195a4093b08e9b4dbdd8e931'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.660714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28cd2509195a4093b08e9b4dbdd8e931'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.673939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28cd2509195a4093b08e9b4dbdd8e931'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.692474 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3403/6790] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:11 2022] DELETE /identity/v3/users/28cd2509195a4093b08e9b4dbdd8e931 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:11.692676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:11.692676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:11.692676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:11.712339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:11.713589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:11.769123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:11.771028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:11.780629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:11.797370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3388/6791] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:11.810665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:11.812396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:11.872517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d4c7-d7e6-48a7-bb87-2f44ea466a85 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:11.878221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:11.879610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:11.879610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:11.879610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] PATH_INFO: `/v3/users/053b5600c4d04027ab66e6d4f27b0908` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:11.880562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=053b5600c4d04027ab66e6d4f27b0908)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:11.883017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:11.906899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6227f-894d-49a2-b1ef-57c12ff15bc8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:11.912644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d0b83-1a8b-46fe-8bd1-8ea635d189e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:11.913371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0b83-1a8b-46fe-8bd1-8ea635d189e0 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:11.913688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0b83-1a8b-46fe-8bd1-8ea635d189e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:11.913964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0b83-1a8b-46fe-8bd1-8ea635d189e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:11.924496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d0b83-1a8b-46fe-8bd1-8ea635d189e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:11.926118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d0b83-1a8b-46fe-8bd1-8ea635d189e0 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:11.937487 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3389/6792] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:11.943738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053b5600c4d04027ab66e6d4f27b0908'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.944669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053b5600c4d04027ab66e6d4f27b0908'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.954505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053b5600c4d04027ab66e6d4f27b0908'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.955543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053b5600c4d04027ab66e6d4f27b0908'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.970367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053b5600c4d04027ab66e6d4f27b0908'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:11.995182 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3404/6793] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:11 2022] DELETE /identity/v3/users/053b5600c4d04027ab66e6d4f27b0908 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:11.999749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.000280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.000706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.033766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:12.035296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:12.121884 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3390/6794] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:12.141193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:12.142878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:12.224831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60640-e843-46f3-ab3c-131aa5f39991 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:12.228967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:12.229645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.229905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.230168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.241021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:12.242859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:12.254404 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3405/6795] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:12.360883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:12.362570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:12.375985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a73366-e280-4794-a82e-3c991c0ddbe5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:12.381284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:12.382643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.383052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.383424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] PATH_INFO: `/v3/projects/c81aa9c09de54cbb9103d6b8bc6442ad` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.384666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81aa9c09de54cbb9103d6b8bc6442ad)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:12.386745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:12.417659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:12.449135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81aa9c09de54cbb9103d6b8bc6442ad', 'request_id': 'req-01e8ab04-f741-4f7d-abec-57e7e0593495'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.458846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81aa9c09de54cbb9103d6b8bc6442ad', 'request_id': 'req-01e8ab04-f741-4f7d-abec-57e7e0593495'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.465502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81aa9c09de54cbb9103d6b8bc6442ad', 'request_id': 'req-01e8ab04-f741-4f7d-abec-57e7e0593495'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.490836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] The token cache is being invalidate because project c81aa9c09de54cbb9103d6b8bc6442ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:12.491572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.492326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.496555 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3391/6796] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:12 2022] DELETE /identity/v3/projects/c81aa9c09de54cbb9103d6b8bc6442ad => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:12.501298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:12.510338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:12.512663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:12.514094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28033b6b-e754-48ec-9ab5-fe245ced6db4 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:12.560815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:12.601048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:12.601750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.602021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.602292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] PATH_INFO: `/v3/users/a9e718ab890446479270e32876c97d62` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.603577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e718ab890446479270e32876c97d62)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:12.605954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:12.610802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8ab04-f741-4f7d-abec-57e7e0593495 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:12.616244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:12.616984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.617306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.617673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] PATH_INFO: `/v3/projects/38ef85f9e05f46cbbdd7bcb54442f999` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.618942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ef85f9e05f46cbbdd7bcb54442f999)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:12.621224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:12.664518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e718ab890446479270e32876c97d62'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.665253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e718ab890446479270e32876c97d62'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.683506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e718ab890446479270e32876c97d62'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.684289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e718ab890446479270e32876c97d62'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.696658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ef85f9e05f46cbbdd7bcb54442f999', 'request_id': 'req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.705590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ef85f9e05f46cbbdd7bcb54442f999', 'request_id': 'req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.708158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-63ad8202-07dd-4b25-9df9-6649a5b20ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e718ab890446479270e32876c97d62'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.721652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ef85f9e05f46cbbdd7bcb54442f999', 'request_id': 'req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.732967 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3406/6797] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:12 2022] DELETE /identity/v3/users/a9e718ab890446479270e32876c97d62 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:12.738573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] The token cache is being invalidate because project 38ef85f9e05f46cbbdd7bcb54442f999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:12.739109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.739942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fb71ea95-a151-4d5e-8eef-789ca1d0cf17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:12.742581 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3392/6798] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:12 2022] DELETE /identity/v3/projects/38ef85f9e05f46cbbdd7bcb54442f999 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:12.748969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed420-2bae-46ab-b638-056e894eb663 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.749499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed420-2bae-46ab-b638-056e894eb663 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.750062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ed420-2bae-46ab-b638-056e894eb663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.776747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-131ed420-2bae-46ab-b638-056e894eb663 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:12.778276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-131ed420-2bae-46ab-b638-056e894eb663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:12.856827 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3407/6799] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:12.864875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437d007-ff8c-4061-bb0c-d1eb1f3e5d86 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.865155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437d007-ff8c-4061-bb0c-d1eb1f3e5d86 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.865391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437d007-ff8c-4061-bb0c-d1eb1f3e5d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.888737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b437d007-ff8c-4061-bb0c-d1eb1f3e5d86 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:12.890237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b437d007-ff8c-4061-bb0c-d1eb1f3e5d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:12.903977 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3393/6800] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:12.904959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ed420-2bae-46ab-b638-056e894eb663 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:12.926632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ed420-2bae-46ab-b638-056e894eb663 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:12.931792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:12.932411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.932688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.932953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.933648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:12.933947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:12.935565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:12.952948 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3408/6801] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:18:12.954230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437d007-ff8c-4061-bb0c-d1eb1f3e5d86 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:12.964824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437d007-ff8c-4061-bb0c-d1eb1f3e5d86 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:12.970047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:12.970671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:12.970948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:12.971211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:12.972016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:12.974154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.006986 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3394/6802] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:12 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:13.010959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.024413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e1a66-d893-4b86-89a1-66e913126ea2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.066088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.071067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:13.077325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:13.087940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:13.110620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.111237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.111516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.111913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.112579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.114685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.139907 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3409/6803] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:13 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:13.146082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.151201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7b783-1090-4589-b51f-f773b4ff5c43 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.156008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.156650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.156840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1fc1-9e99-4a1e-9fad-13b6cd37ed28 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.156948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.157182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] PATH_INFO: `/v3/projects/9471aaa9e2fe4ec083e8fb3b4655ddf8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.158452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] RBAC: Authorizing `identity:delete_project(project_id=9471aaa9e2fe4ec083e8fb3b4655ddf8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.160566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.161608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.162215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.162538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.162814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.163522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.165105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.175723 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3410/6804] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:13.179894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.190086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12af62d-ffdd-4e60-829c-7cd544284478 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.194790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.195385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.195660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.195933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.196602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.198245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.208655 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3411/6805] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:13.210297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9471aaa9e2fe4ec083e8fb3b4655ddf8', 'request_id': 'req-76436ce5-172f-4f4b-bac3-617bec0c80da'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.212973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.218945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9471aaa9e2fe4ec083e8fb3b4655ddf8', 'request_id': 'req-76436ce5-172f-4f4b-bac3-617bec0c80da'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.223632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd776ad-5749-4410-bd96-ebc4b6478a18 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.225983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9471aaa9e2fe4ec083e8fb3b4655ddf8', 'request_id': 'req-76436ce5-172f-4f4b-bac3-617bec0c80da'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.238307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] The token cache is being invalidate because project 9471aaa9e2fe4ec083e8fb3b4655ddf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:13.238737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.239522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.241217 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3395/6806] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:13 2022] DELETE /identity/v3/projects/9471aaa9e2fe4ec083e8fb3b4655ddf8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:13.245447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.246850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:13.284397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e23f7-230e-452f-a26a-a946389e136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.285067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e23f7-230e-452f-a26a-a946389e136b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.285338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e23f7-230e-452f-a26a-a946389e136b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.285698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e23f7-230e-452f-a26a-a946389e136b admin admin] PATH_INFO: `/v3/projects/f5457492c1db4888802770dcf7055c1d/users/df2f60fa0bde4397aaabf6caa8e4efe2/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.301935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e23f7-230e-452f-a26a-a946389e136b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5457492c1db4888802770dcf7055c1d, user_id=df2f60fa0bde4397aaabf6caa8e4efe2, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.304650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e23f7-230e-452f-a26a-a946389e136b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.316278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436ce5-172f-4f4b-bac3-617bec0c80da admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.329122 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3412/6807] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:13 2022] PUT /identity/v3/projects/f5457492c1db4888802770dcf7055c1d/users/df2f60fa0bde4397aaabf6caa8e4efe2/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:18:13.331337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.331618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.332052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.367892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] MFA Rules not processed for user `df2f60fa0bde4397aaabf6caa8e4efe2`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:13.369243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:13.377145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:13.392239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.392885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.393388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.393783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] PATH_INFO: `/v3/users/23b099b3ad3649e1bfd04a5e2a787c0e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.395716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b099b3ad3649e1bfd04a5e2a787c0e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.398361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.442540 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3413/6808] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:13.448389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.450765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:13.454883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b099b3ad3649e1bfd04a5e2a787c0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.465582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0e599-7007-43b6-8e5d-bff47803cfa5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.468225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b099b3ad3649e1bfd04a5e2a787c0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.471519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.472178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.472562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.472842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.473847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.475829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.480014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b099b3ad3649e1bfd04a5e2a787c0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.481093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b099b3ad3649e1bfd04a5e2a787c0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.482757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b099b3ad3649e1bfd04a5e2a787c0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:13.495856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:13.503178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3396/6809] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:13 2022] DELETE /identity/v3/users/23b099b3ad3649e1bfd04a5e2a787c0e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:18:13.529567 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3414/6810] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:13 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:13.533295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.536304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:13.620699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f32ad-ad67-4c6c-8fee-05065683b4fe admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.684023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.684360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.684648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.684921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.685643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.687623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.714656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3397/6811] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:13 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:13.718938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.731902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f206fd-c926-490b-86aa-ebe2785f3743 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.738380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.739218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.739610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.739827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.740665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.742756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.755723 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3415/6812] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:13.760394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.771532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59571193-8693-4b4b-b550-00cf66555164 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.777055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.777867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.778211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.778925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] PATH_INFO: `/v3/projects/cd108287b7204dc59995bd50ce0e5428/users/13e4c127ef5d4ba4aeafd228de0d9254/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.781193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:13.797079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd108287b7204dc59995bd50ce0e5428, user_id=13e4c127ef5d4ba4aeafd228de0d9254, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.799812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.814967 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3398/6813] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:13 2022] PUT /identity/v3/projects/cd108287b7204dc59995bd50ce0e5428/users/13e4c127ef5d4ba4aeafd228de0d9254/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:13.819422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.848025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff721b-57b2-4d5c-bafd-32250c1aba15 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.849887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.851715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:13.947763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.948720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.949005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.949279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.950590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.952538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.966069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9101c20-e339-4cb0-911e-2e74460d8059 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:13.967243 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3416/6814] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:13.971469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:13.980602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:13.981503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:13.981883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:13.982159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] PATH_INFO: `/v3/projects/ee1bf0dd63be4bb9a9d0083d0b1fdeac` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:13.984480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1bf0dd63be4bb9a9d0083d0b1fdeac)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:13.986366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:13.991392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96070d74-6305-4a6d-bf4c-551a23da4ebd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.000543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10130e-8083-423c-afc5-791c0e5f1810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.001567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10130e-8083-423c-afc5-791c0e5f1810 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.002674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10130e-8083-423c-afc5-791c0e5f1810 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.003089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10130e-8083-423c-afc5-791c0e5f1810 admin admin] PATH_INFO: `/v3/projects/cd108287b7204dc59995bd50ce0e5428/users/13e4c127ef5d4ba4aeafd228de0d9254/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.005935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10130e-8083-423c-afc5-791c0e5f1810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd108287b7204dc59995bd50ce0e5428, user_id=13e4c127ef5d4ba4aeafd228de0d9254, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.009525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10130e-8083-423c-afc5-791c0e5f1810 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.025555 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3417/6815] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:13 2022] PUT /identity/v3/projects/cd108287b7204dc59995bd50ce0e5428/users/13e4c127ef5d4ba4aeafd228de0d9254/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:14.036497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.036790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.037067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.057230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1bf0dd63be4bb9a9d0083d0b1fdeac', 'request_id': 'req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:14.065959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1bf0dd63be4bb9a9d0083d0b1fdeac', 'request_id': 'req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:14.068370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] MFA Rules not processed for user `13e4c127ef5d4ba4aeafd228de0d9254`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:14.069888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:14.074482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1bf0dd63be4bb9a9d0083d0b1fdeac', 'request_id': 'req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:14.102237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] The token cache is being invalidate because project ee1bf0dd63be4bb9a9d0083d0b1fdeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:14.102743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:14.104032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7845d8fe-41ed-47b7-85e2-f50fc16f63c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:14.110806 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3399/6816] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:13 2022] DELETE /identity/v3/projects/ee1bf0dd63be4bb9a9d0083d0b1fdeac => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:14.112645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460140a-2e0e-4245-abcc-fda1e56941f9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.113636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460140a-2e0e-4245-abcc-fda1e56941f9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.114075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460140a-2e0e-4245-abcc-fda1e56941f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.137173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1460140a-2e0e-4245-abcc-fda1e56941f9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:14.138809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1460140a-2e0e-4245-abcc-fda1e56941f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:14.156339 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3418/6817] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:14.171485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.173155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:14.210497 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3400/6818] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:14.220192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832d956-9cb0-499b-977b-ed20afd932ee None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.220476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832d956-9cb0-499b-977b-ed20afd932ee None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.220750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832d956-9cb0-499b-977b-ed20afd932ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.238736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:14.241956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c832d956-9cb0-499b-977b-ed20afd932ee None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:14.243358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c832d956-9cb0-499b-977b-ed20afd932ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:14.243892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b322cc8-0b8e-408a-8c20-bfdb957226cd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.249375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.251562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.252173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.255962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.256131 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3401/6819] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:14.257061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832d956-9cb0-499b-977b-ed20afd932ee None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.264993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.266707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.267065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832d956-9cb0-499b-977b-ed20afd932ee None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.272666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.273481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.274018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.274350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.275678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:14.275822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.278502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.278633 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3419/6820] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:14.280466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:14.299378 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3402/6821] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:14.303587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.314379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b47c4-af88-4d7f-a097-27d21e9a87cc service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.319159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.319711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.319989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.320261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.320888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.323485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.354553 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3420/6822] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:18:14.358977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.370058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392ac27-bf63-4b5e-8157-42bccb2bafbb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.372498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.428922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.429446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.429800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.430117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.430226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c31eac-5bb3-454f-9cae-4f710371ba16 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.430746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.432721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.433904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.434507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.434781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.435048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.436827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:14.460869 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3403/6823] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:14.464914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.475305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cfc6d-6306-4f1a-bfd6-a77857e8a0af admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.479856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.480445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.480717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.480985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.481656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.483300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.494989 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3404/6824] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:14.498958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.502593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.504023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.508724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec14cb7-c93b-40b4-9865-e7b3ab647fac admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.513061 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3421/6825] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:14.513262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110295a-4579-462d-9d44-5220087d647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.513848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110295a-4579-462d-9d44-5220087d647f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.514125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110295a-4579-462d-9d44-5220087d647f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.514397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110295a-4579-462d-9d44-5220087d647f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.515031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110295a-4579-462d-9d44-5220087d647f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.516649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110295a-4579-462d-9d44-5220087d647f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.529384 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3405/6826] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:14.531229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.542243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c4a1a-e78f-4de0-9817-997ad7a01ada service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.547086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.547744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.548017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.548306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] PATH_INFO: `/v3/projects/111884222b25460a9613135036eb9c9f/users/7fad04528a2d48698acb72682d1a2b86/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.563443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111884222b25460a9613135036eb9c9f, user_id=7fad04528a2d48698acb72682d1a2b86, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.566141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.580037 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3422/6827] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:14 2022] PUT /identity/v3/projects/111884222b25460a9613135036eb9c9f/users/7fad04528a2d48698acb72682d1a2b86/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:14.592130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be0b2-28ea-47a9-8e8d-311551b40acb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.592494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be0b2-28ea-47a9-8e8d-311551b40acb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.592843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be0b2-28ea-47a9-8e8d-311551b40acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.621845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aa1be0b2-28ea-47a9-8e8d-311551b40acb None None] MFA Rules not processed for user `7fad04528a2d48698acb72682d1a2b86`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:14.623059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa1be0b2-28ea-47a9-8e8d-311551b40acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:14.738300 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3406/6828] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:14.744007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.806957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d16e03-dcfb-46f3-8ef5-b692f4b522e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.811604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.812248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.812520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.812790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.813432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.815388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.819915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:14.846196 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3423/6829] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:18:14.850430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1be0b2-28ea-47a9-8e8d-311551b40acb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.861967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1be0b2-28ea-47a9-8e8d-311551b40acb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.911988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:14.924071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.924718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.924992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.925264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.925930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.927925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.950287 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3407/6830] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:14 2022] POST /identity/v3/users => generated 307 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:14.954231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:14.965040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd12022-cb46-4cfa-8e52-5f3e36364a2b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:14.970161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:14.970756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:14.971029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:14.971296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:14.971945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:14.973606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:14.983509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3424/6831] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:14.989339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:15.000868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8dbde-968f-43d7-9fc0-1168014337cf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:15.005907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:15.006630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:15.006854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:15.007127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] PATH_INFO: `/v3/projects/eb4f2de770454f698e1cd045bf4d5647/users/bc47164c83c14338a6fd4befd5a0facf/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:15.022466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4f2de770454f698e1cd045bf4d5647, user_id=bc47164c83c14338a6fd4befd5a0facf, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:15.025178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:15.040952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3408/6832] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:14 2022] PUT /identity/v3/projects/eb4f2de770454f698e1cd045bf4d5647/users/bc47164c83c14338a6fd4befd5a0facf/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:15.045269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:15.056868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610b4d5-611b-4669-8a44-2be4ea4dd873 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:15.116524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26ea5c-297d-4df5-bf03-2c35248237d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:15.117143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26ea5c-297d-4df5-bf03-2c35248237d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:15.117454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26ea5c-297d-4df5-bf03-2c35248237d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:15.117763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26ea5c-297d-4df5-bf03-2c35248237d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:15.118408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26ea5c-297d-4df5-bf03-2c35248237d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:15.120370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26ea5c-297d-4df5-bf03-2c35248237d3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:15.130553 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3425/6833] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:15.134943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:15.145691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e593da-4f99-49fa-a48b-669fa1c031f5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:15.150655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:15.151307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:15.151584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:15.151845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] PATH_INFO: `/v3/projects/eb4f2de770454f698e1cd045bf4d5647/users/bc47164c83c14338a6fd4befd5a0facf/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:15.153858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4f2de770454f698e1cd045bf4d5647, user_id=bc47164c83c14338a6fd4befd5a0facf, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:15.156501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:15.168641 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3409/6834] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:15 2022] PUT /identity/v3/projects/eb4f2de770454f698e1cd045bf4d5647/users/bc47164c83c14338a6fd4befd5a0facf/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:15.179205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdec0b-a599-434e-8ccf-64c6c9fc8318 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:15.179542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdec0b-a599-434e-8ccf-64c6c9fc8318 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:15.180088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdec0b-a599-434e-8ccf-64c6c9fc8318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:15.211096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3dcdec0b-a599-434e-8ccf-64c6c9fc8318 None None] MFA Rules not processed for user `bc47164c83c14338a6fd4befd5a0facf`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:15.212307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3dcdec0b-a599-434e-8ccf-64c6c9fc8318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:15.281391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3426/6835] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:15.294657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:15.359155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d22b6e-571c-416f-b0bc-f510cf7fdc1e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:15.362870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:15.363537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:15.363747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:15.364015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:15.374192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:15.375639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:15.385630 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3410/6836] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:15 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:15.473375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdec0b-a599-434e-8ccf-64c6c9fc8318 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:15.484978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdec0b-a599-434e-8ccf-64c6c9fc8318 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:15.489239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576be53d-b7b8-4096-8239-57f56db3b825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:15.490059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be53d-b7b8-4096-8239-57f56db3b825 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:15.490516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be53d-b7b8-4096-8239-57f56db3b825 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:15.490788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be53d-b7b8-4096-8239-57f56db3b825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:15.549253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:15.552226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576be53d-b7b8-4096-8239-57f56db3b825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:15.554128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576be53d-b7b8-4096-8239-57f56db3b825 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:15.565685 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3427/6837] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:19.806047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:19.885918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ed9ff-6abf-4aed-b3a1-7048c4921862 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:19.890795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:19.891244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:19.891618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:19.891979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] PATH_INFO: `/v3/users/7fad04528a2d48698acb72682d1a2b86` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:19.893225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fad04528a2d48698acb72682d1a2b86)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:19.895541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:19.952245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fad04528a2d48698acb72682d1a2b86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:19.954166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:19.971142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fad04528a2d48698acb72682d1a2b86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:19.982032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fad04528a2d48698acb72682d1a2b86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:19.982524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fad04528a2d48698acb72682d1a2b86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:19.983473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fad04528a2d48698acb72682d1a2b86'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:19.998786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3411/6838] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:19 2022] DELETE /identity/v3/users/7fad04528a2d48698acb72682d1a2b86 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:20.003159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.003675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.003899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.024567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:20.025988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.168176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3428/6839] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:20.179663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.181270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.351191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ab9d0-7673-4b40-824a-3f52c52fb5d8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.358295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.359011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.359459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.359710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.371774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.373564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.386772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3412/6840] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:20.855102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.856554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.879617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216e8b0-d8b7-4c7b-a8c1-da980f0e9c79 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.888010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.888954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.889380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.889887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] PATH_INFO: `/v3/projects/111884222b25460a9613135036eb9c9f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.893093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=111884222b25460a9613135036eb9c9f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.895917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.962393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111884222b25460a9613135036eb9c9f', 'request_id': 'req-2bbe4819-0dd1-45dd-8bfe-a153b380f131'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.970142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111884222b25460a9613135036eb9c9f', 'request_id': 'req-2bbe4819-0dd1-45dd-8bfe-a153b380f131'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.982716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111884222b25460a9613135036eb9c9f', 'request_id': 'req-2bbe4819-0dd1-45dd-8bfe-a153b380f131'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.000237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] The token cache is being invalidate because project 111884222b25460a9613135036eb9c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:21.000534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.001382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.003326 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3429/6841] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:20 2022] DELETE /identity/v3/projects/111884222b25460a9613135036eb9c9f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:21.010195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:21.026081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7c4f-3d71-46d9-b886-237703aef0ec service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:21.088366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:21.089051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:21.089338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:21.089652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] PATH_INFO: `/v3/users/bc47164c83c14338a6fd4befd5a0facf` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:21.090787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc47164c83c14338a6fd4befd5a0facf)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:21.092742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:21.158448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc47164c83c14338a6fd4befd5a0facf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.189261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc47164c83c14338a6fd4befd5a0facf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.196007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:21.208107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc47164c83c14338a6fd4befd5a0facf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.210518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc47164c83c14338a6fd4befd5a0facf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.214018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1986fca7-aea8-4ca5-b6cf-0aa9922feb3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc47164c83c14338a6fd4befd5a0facf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.232785 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3413/6842] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:21 2022] DELETE /identity/v3/users/bc47164c83c14338a6fd4befd5a0facf => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:21.560336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:21.561885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:21.578615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe4819-0dd1-45dd-8bfe-a153b380f131 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:21.639787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:21.640330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:21.640601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:21.640841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] PATH_INFO: `/v3/projects/eb4f2de770454f698e1cd045bf4d5647` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:21.642072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4f2de770454f698e1cd045bf4d5647)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:21.644120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:21.666715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:21.693964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4f2de770454f698e1cd045bf4d5647', 'request_id': 'req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.700373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4f2de770454f698e1cd045bf4d5647', 'request_id': 'req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.709372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4f2de770454f698e1cd045bf4d5647', 'request_id': 'req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.721168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] The token cache is being invalidate because project eb4f2de770454f698e1cd045bf4d5647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:21.721513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.722140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ceb2431c-f083-4b95-9cdb-dc3c1e42e8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:21.723720 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3430/6843] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:21 2022] DELETE /identity/v3/projects/eb4f2de770454f698e1cd045bf4d5647 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:21.728733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef2f3a-c8bb-4436-af13-af2fd303df7b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:21.729018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef2f3a-c8bb-4436-af13-af2fd303df7b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:21.729289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef2f3a-c8bb-4436-af13-af2fd303df7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:21.751370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-05ef2f3a-c8bb-4436-af13-af2fd303df7b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:21.752574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-05ef2f3a-c8bb-4436-af13-af2fd303df7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:21.814677 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3414/6844] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:21.822924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0058a09-7148-499c-ad13-0c95be5745ca None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:21.823183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0058a09-7148-499c-ad13-0c95be5745ca None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:21.823474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0058a09-7148-499c-ad13-0c95be5745ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:21.846978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c0058a09-7148-499c-ad13-0c95be5745ca None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:21.848341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c0058a09-7148-499c-ad13-0c95be5745ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:21.857280 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3431/6845] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:21.861679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef2f3a-c8bb-4436-af13-af2fd303df7b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:21.872171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef2f3a-c8bb-4436-af13-af2fd303df7b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:21.876928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:21.877631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:21.877970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:21.878301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:21.879005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:21.879385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:21.881266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:21.899259 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3415/6846] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:21.900745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0058a09-7148-499c-ad13-0c95be5745ca None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:21.910783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0058a09-7148-499c-ad13-0c95be5745ca None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:21.915552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:21.916155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:21.916427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:21.916701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:21.917351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:21.919369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:21.957858 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3432/6847] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:21 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:21.962149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:21.982962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601c3b9-dd62-46df-9193-c172ddf5d43b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.072138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.073322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.073921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.075224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.076063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.078347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.121544 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3416/6848] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:21 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:22.124455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.135372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea12063-7b65-457c-b432-7d3c81897a6e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.139954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.140644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.140981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.141311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.142046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.144190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.154564 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3433/6849] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:22.158627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.169824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fed36-38c8-48a8-8726-d93655b397b3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.175535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b401da7-ff7a-41bb-baac-c803fa93b496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.176258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b401da7-ff7a-41bb-baac-c803fa93b496 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.176610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b401da7-ff7a-41bb-baac-c803fa93b496 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.176953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b401da7-ff7a-41bb-baac-c803fa93b496 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.177715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b401da7-ff7a-41bb-baac-c803fa93b496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.179602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b401da7-ff7a-41bb-baac-c803fa93b496 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.191319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3417/6850] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:22.195709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.206212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686e48-3c91-415e-91c0-0031902dcd6c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.211860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.212519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.212799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.213167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] PATH_INFO: `/v3/projects/47e50b5e859d456c8dc651b28a13882a/users/87b9497fea8a4a9680e02065b15862b8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.228175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e50b5e859d456c8dc651b28a13882a, user_id=87b9497fea8a4a9680e02065b15862b8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.230935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.248039 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3434/6851] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:22 2022] PUT /identity/v3/projects/47e50b5e859d456c8dc651b28a13882a/users/87b9497fea8a4a9680e02065b15862b8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:22.258430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227a461-c634-4a23-9fa3-cf542625e92a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.258728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227a461-c634-4a23-9fa3-cf542625e92a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.258979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227a461-c634-4a23-9fa3-cf542625e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.294084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6227a461-c634-4a23-9fa3-cf542625e92a None None] MFA Rules not processed for user `87b9497fea8a4a9680e02065b15862b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:22.295925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6227a461-c634-4a23-9fa3-cf542625e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:22.348966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:22.387016 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3418/6852] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:22.391960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.473184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd1e98-3804-47b5-ab23-9ba3a78f5c7e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.477974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.478619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.478898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.479173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.479804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.481741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.513784 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3435/6853] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:22 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:22.514965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227a461-c634-4a23-9fa3-cf542625e92a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.525991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227a461-c634-4a23-9fa3-cf542625e92a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.586267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.586885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.587158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.587453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.588097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.590702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.622700 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3419/6854] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:22 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:18:22.623683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.634619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0516f-bf08-4d93-9273-d0c8e6c8aaba admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.639847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.640541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.640820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.641090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.641782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.643611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.654996 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3436/6855] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:22.659316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.670473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5f52b-d024-4acf-b429-496aef61d5ed admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.677489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.678204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.678483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.678756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] PATH_INFO: `/v3/projects/0088903f21314553a328ff37b691b018/users/4342ac9734a94b0787a0f8a074ae05f8/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.695832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0088903f21314553a328ff37b691b018, user_id=4342ac9734a94b0787a0f8a074ae05f8, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.698625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.714841 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3420/6856] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:22 2022] PUT /identity/v3/projects/0088903f21314553a328ff37b691b018/users/4342ac9734a94b0787a0f8a074ae05f8/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:22.718815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.731163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0465f56-998b-4226-847b-82c0f5d3be83 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.745524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:22.794134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a579e2-8d01-4bd3-adff-c0b08692fe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.794759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a579e2-8d01-4bd3-adff-c0b08692fe2f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.795045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a579e2-8d01-4bd3-adff-c0b08692fe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.795316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a579e2-8d01-4bd3-adff-c0b08692fe2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.795959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a579e2-8d01-4bd3-adff-c0b08692fe2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.797608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a579e2-8d01-4bd3-adff-c0b08692fe2f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.807973 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3437/6857] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:22.812265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.823788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c762-d84c-475f-8af2-547db9fe4704 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:22.829242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:22.829979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.830263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.830657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] PATH_INFO: `/v3/projects/0088903f21314553a328ff37b691b018/users/4342ac9734a94b0787a0f8a074ae05f8/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.832673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0088903f21314553a328ff37b691b018, user_id=4342ac9734a94b0787a0f8a074ae05f8, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:22.835532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:22.846515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3421/6858] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:22 2022] PUT /identity/v3/projects/0088903f21314553a328ff37b691b018/users/4342ac9734a94b0787a0f8a074ae05f8/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:22.857955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce83a6-1c52-4caa-94d5-b16015f22119 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:22.858250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce83a6-1c52-4caa-94d5-b16015f22119 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:22.858523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce83a6-1c52-4caa-94d5-b16015f22119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:22.886893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-60ce83a6-1c52-4caa-94d5-b16015f22119 None None] MFA Rules not processed for user `4342ac9734a94b0787a0f8a074ae05f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:22.888761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-60ce83a6-1c52-4caa-94d5-b16015f22119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:22.970990 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3438/6859] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:22.981790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:22.983493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:23.045832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dcca0-e1da-47d0-855d-1f70d6c1989c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:23.049764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ede9c-f0c9-4d0c-badf-1c5fe7c8ad65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:23.050402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ede9c-f0c9-4d0c-badf-1c5fe7c8ad65 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:23.050677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ede9c-f0c9-4d0c-badf-1c5fe7c8ad65 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:23.050943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ede9c-f0c9-4d0c-badf-1c5fe7c8ad65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:23.060565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ede9c-f0c9-4d0c-badf-1c5fe7c8ad65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:23.062061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ede9c-f0c9-4d0c-badf-1c5fe7c8ad65 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:23.071624 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3422/6860] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:23.250196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce83a6-1c52-4caa-94d5-b16015f22119 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:23.359871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce83a6-1c52-4caa-94d5-b16015f22119 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:23.364691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:23.365562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:23.365992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:23.366881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] PATH_INFO: `/v3/users/87b9497fea8a4a9680e02065b15862b8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:23.368833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b9497fea8a4a9680e02065b15862b8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:23.371673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:23.424744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:23.431896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b9497fea8a4a9680e02065b15862b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:23.432671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b9497fea8a4a9680e02065b15862b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:23.441872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b9497fea8a4a9680e02065b15862b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:23.442498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b9497fea8a4a9680e02065b15862b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:23.458395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b9497fea8a4a9680e02065b15862b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:23.473060 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3439/6861] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:23 2022] DELETE /identity/v3/users/87b9497fea8a4a9680e02065b15862b8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:23.478002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:23.478532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:23.478532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:23.507224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:23.509496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:23.552096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:23.585883 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3423/6862] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:23.597167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:23.598755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:23.674478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a678d-b1ef-4ffa-a930-b23e891bf131 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:23.680195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:23.681083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:23.681620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:23.681902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:23.694486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:23.696124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:23.707205 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3440/6863] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:24.540178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:24.541891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:24.556195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fc8a2-b54c-49d8-b010-9c3576620cde None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:24.566914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:24.567576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:24.567847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:24.568331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] PATH_INFO: `/v3/projects/47e50b5e859d456c8dc651b28a13882a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:24.570779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e50b5e859d456c8dc651b28a13882a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:24.573171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:24.650351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e50b5e859d456c8dc651b28a13882a', 'request_id': 'req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.668887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e50b5e859d456c8dc651b28a13882a', 'request_id': 'req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.675603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e50b5e859d456c8dc651b28a13882a', 'request_id': 'req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.698529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] The token cache is being invalidate because project 47e50b5e859d456c8dc651b28a13882a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:24.698529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.699619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.701429 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3424/6864] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:24 2022] DELETE /identity/v3/projects/47e50b5e859d456c8dc651b28a13882a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:24.705584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:24.717179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d04c8-9066-4be9-b08f-e4e9797e6770 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:24.801121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:24.801806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:24.802082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:24.802395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] PATH_INFO: `/v3/users/4342ac9734a94b0787a0f8a074ae05f8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:24.803454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4342ac9734a94b0787a0f8a074ae05f8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:24.805509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:24.873802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4342ac9734a94b0787a0f8a074ae05f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.874623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4342ac9734a94b0787a0f8a074ae05f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.886589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4342ac9734a94b0787a0f8a074ae05f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.887210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4342ac9734a94b0787a0f8a074ae05f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.909947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:24.932069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d051b198-d4ab-4841-afa3-a8fc17744068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4342ac9734a94b0787a0f8a074ae05f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:24.947110 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3441/6865] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:24 2022] DELETE /identity/v3/users/4342ac9734a94b0787a0f8a074ae05f8 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:25.281720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:25.283182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:25.297451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae85415-0cd5-40b8-a458-9f6c4bb38f10 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:25.418628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:25.420240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:25.421384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:25.421384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] PATH_INFO: `/v3/projects/0088903f21314553a328ff37b691b018` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:25.423058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0088903f21314553a328ff37b691b018)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:25.425804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:25.506998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0088903f21314553a328ff37b691b018', 'request_id': 'req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:25.517543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0088903f21314553a328ff37b691b018', 'request_id': 'req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:25.523858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:25.525442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0088903f21314553a328ff37b691b018', 'request_id': 'req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:25.541455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] The token cache is being invalidate because project 0088903f21314553a328ff37b691b018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:25.542115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:25.542781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6c4eefc3-6fda-4fb8-9d6c-73768be16d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:25.544890 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3425/6866] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:25 2022] DELETE /identity/v3/projects/0088903f21314553a328ff37b691b018 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:25.554543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed4f80-1d92-435b-86f6-dabe544d2447 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:25.555024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed4f80-1d92-435b-86f6-dabe544d2447 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:25.555299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed4f80-1d92-435b-86f6-dabe544d2447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:25.604878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b4ed4f80-1d92-435b-86f6-dabe544d2447 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:25.606142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4ed4f80-1d92-435b-86f6-dabe544d2447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:25.761132 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3442/6867] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:25.763800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70739087-a31e-48da-a77a-afca6f12f423 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:25.764085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70739087-a31e-48da-a77a-afca6f12f423 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:25.764359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70739087-a31e-48da-a77a-afca6f12f423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:25.801501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-70739087-a31e-48da-a77a-afca6f12f423 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:25.802999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-70739087-a31e-48da-a77a-afca6f12f423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:25.834007 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3426/6868] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:25.838056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed4f80-1d92-435b-86f6-dabe544d2447 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:25.849781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed4f80-1d92-435b-86f6-dabe544d2447 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:25.858670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:25.859616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:25.860678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:25.861103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:25.862098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:25.862593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:25.864395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:25.878678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3443/6869] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:18:25.883742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70739087-a31e-48da-a77a-afca6f12f423 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:25.900642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70739087-a31e-48da-a77a-afca6f12f423 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:25.907702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:25.908348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:25.908810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:25.909035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:25.909708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:25.911671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:25.965164 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3427/6870] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:25 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:25.969337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.000649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508459a-8b02-4c8f-888e-0ab865332058 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.092441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.093797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.094104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.094355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.094990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.096929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.124791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3444/6871] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:25 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:26.125740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.136764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de87f8-0859-423d-ae46-a848703b2a93 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.142772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.143385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.143665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.143936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.144585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.146263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.170365 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3428/6872] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:26.172817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.186435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b777e-8312-45c8-a5ab-6c1684be58aa admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.192300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.192913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.193187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.193453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.194124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.196219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.212236 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3445/6873] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:26.218647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.227621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.229373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:26.246207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969b916-2a83-47e9-8756-ac212e2f81ca admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.247000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da1c53-8e8c-4274-b8e1-ba6d53393f31 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.253095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.253820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.254134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030d69b-7fcc-4203-80a4-ac511067c618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.254372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.254477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] PATH_INFO: `/v3/users/47f724dca5d5409d9a3a02289c286d89` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.255581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f724dca5d5409d9a3a02289c286d89)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.255830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030d69b-7fcc-4203-80a4-ac511067c618 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.256177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030d69b-7fcc-4203-80a4-ac511067c618 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.256629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030d69b-7fcc-4203-80a4-ac511067c618 admin admin] PATH_INFO: `/v3/projects/6aabbad62d754b98945a20cc5470e27b/users/b408eef04f5649ac9f47e89d245433fc/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.257772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.274139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030d69b-7fcc-4203-80a4-ac511067c618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aabbad62d754b98945a20cc5470e27b, user_id=b408eef04f5649ac9f47e89d245433fc, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.276989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030d69b-7fcc-4203-80a4-ac511067c618 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.297854 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3429/6874] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:26 2022] PUT /identity/v3/projects/6aabbad62d754b98945a20cc5470e27b/users/b408eef04f5649ac9f47e89d245433fc/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:26.312340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.312625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.312901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.327032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:26.341305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f724dca5d5409d9a3a02289c286d89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.346746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f724dca5d5409d9a3a02289c286d89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.364746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f724dca5d5409d9a3a02289c286d89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.365285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f724dca5d5409d9a3a02289c286d89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.367748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] MFA Rules not processed for user `b408eef04f5649ac9f47e89d245433fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:26.369293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:26.387519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f724dca5d5409d9a3a02289c286d89'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.405379 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3446/6875] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:26 2022] DELETE /identity/v3/users/47f724dca5d5409d9a3a02289c286d89 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:18:26.497558 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3430/6876] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:26.514749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.516527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:26.603207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33335d8d-432e-446a-90b8-4a5465b69aee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.607286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.609116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.609591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.609876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.665288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.668098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:26.693201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.695031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.706022 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3447/6877] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:26.732921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:26.734916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:26.789528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460391-8f18-4a4d-98f8-8bdaa5f7521c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.803901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00098e-9e0c-430b-9612-056fd2198cd7 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:26.808627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.809257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.809565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.809790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:26.809927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] PATH_INFO: `/v3/users/b408eef04f5649ac9f47e89d245433fc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.810964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b408eef04f5649ac9f47e89d245433fc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.812888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.829721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:26.847610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:26.848251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.848528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.848797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] PATH_INFO: `/v3/projects/4c0548e01f2249a98e28b618ca3ab8cc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.850002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c0548e01f2249a98e28b618ca3ab8cc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:26.852145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:26.867624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b408eef04f5649ac9f47e89d245433fc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.868305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b408eef04f5649ac9f47e89d245433fc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.877894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b408eef04f5649ac9f47e89d245433fc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.878622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b408eef04f5649ac9f47e89d245433fc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.891725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2b1b5f3c-3566-4d75-9a1d-c9a3fa3c2151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b408eef04f5649ac9f47e89d245433fc'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.900676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c0548e01f2249a98e28b618ca3ab8cc', 'request_id': 'req-46b240c6-f725-413e-9dd9-8ebdc5a033fd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.904046 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3448/6878] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:26 2022] DELETE /identity/v3/users/b408eef04f5649ac9f47e89d245433fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:26.908489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.908755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.909029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.909800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c0548e01f2249a98e28b618ca3ab8cc', 'request_id': 'req-46b240c6-f725-413e-9dd9-8ebdc5a033fd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.916223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c0548e01f2249a98e28b618ca3ab8cc', 'request_id': 'req-46b240c6-f725-413e-9dd9-8ebdc5a033fd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.927581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] The token cache is being invalidate because project 4c0548e01f2249a98e28b618ca3ab8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:26.927871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.928244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:26.928535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-46b240c6-f725-413e-9dd9-8ebdc5a033fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:26.929426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:26.930194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3431/6879] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:26 2022] DELETE /identity/v3/projects/4c0548e01f2249a98e28b618ca3ab8cc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:26.934975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf430-b458-4a83-8d77-12cc532af4c4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:26.935253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf430-b458-4a83-8d77-12cc532af4c4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:26.935526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf430-b458-4a83-8d77-12cc532af4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:26.955222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-43bbf430-b458-4a83-8d77-12cc532af4c4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:26.956574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-43bbf430-b458-4a83-8d77-12cc532af4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:26.995160 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3449/6880] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:27.002711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.004255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:27.016079 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3432/6881] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:27.024101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc53a-0748-4be6-9875-3c05f27795ae None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.024387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc53a-0748-4be6-9875-3c05f27795ae None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.024657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc53a-0748-4be6-9875-3c05f27795ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.044688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8a3dc53a-0748-4be6-9875-3c05f27795ae None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:27.045949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a3dc53a-0748-4be6-9875-3c05f27795ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:27.054106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3433/6882] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:27.058226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dc53a-0748-4be6-9875-3c05f27795ae None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.067397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407932e-6676-4b96-b0d0-1c105a8c461a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.067553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dc53a-0748-4be6-9875-3c05f27795ae None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.071094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.071699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.071980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.072241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.072417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.073016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.073293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.073579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.074213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:27.074512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.076185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.081695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.083181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.088100 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3434/6883] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:18:27.092346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.092870 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3450/6884] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:27.102617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f67a16-d39f-4cc3-a749-72f426ed847c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.107259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.107861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.108138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.108436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.109065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.111057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.141453 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3435/6885] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:27 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:27.145750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.155920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed5566-f5b8-4dc0-9f2c-0540d0c845e1 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.188687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:27.215236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.215848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.216131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.216397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.217026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.219024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.246108 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3451/6886] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:27 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:27.247537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.258963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270e58b-b4dc-428e-92a5-d25644a25235 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.263897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.264518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.264795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.265064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.265742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.267376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.279107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3436/6887] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:27.283244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.294362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4042b80-a8a6-4657-b4cf-89416c10db2b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.299689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63394d94-8679-4f27-9b86-364bcb950c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.300322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63394d94-8679-4f27-9b86-364bcb950c7c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.300649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63394d94-8679-4f27-9b86-364bcb950c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.300908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63394d94-8679-4f27-9b86-364bcb950c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.301609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63394d94-8679-4f27-9b86-364bcb950c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.303240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63394d94-8679-4f27-9b86-364bcb950c7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.314484 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3452/6888] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:27.319125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.324807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:27.332911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3880d-4c9c-4bc4-92f4-e6e340445380 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.340123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.340903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.341254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.341629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] PATH_INFO: `/v3/projects/a37593cb29164fa2bbb0a54a758fa3bd/users/f08edf449e334897a129a1d26aedb507/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.358577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37593cb29164fa2bbb0a54a758fa3bd, user_id=f08edf449e334897a129a1d26aedb507, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.361389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.379098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3437/6889] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:27 2022] PUT /identity/v3/projects/a37593cb29164fa2bbb0a54a758fa3bd/users/f08edf449e334897a129a1d26aedb507/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:27.385051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.385330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.385628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.406702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:27.408044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:27.473205 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3453/6890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:27.483450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.485324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:27.557362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad45541-fb3c-4711-83db-323d28a38674 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.561358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3af5c-3d35-4dc0-a64c-edc38273c43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.561898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3af5c-3d35-4dc0-a64c-edc38273c43e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.562185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3af5c-3d35-4dc0-a64c-edc38273c43e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.562481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3af5c-3d35-4dc0-a64c-edc38273c43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.572659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3af5c-3d35-4dc0-a64c-edc38273c43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.574490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3af5c-3d35-4dc0-a64c-edc38273c43e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.585633 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3438/6891] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:27.772555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:27.774253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:27.787485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fec28-a46b-41bf-a308-c8fc638b4b3f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:27.817051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:27.817770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.818082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.818362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] PATH_INFO: `/v3/projects/6aabbad62d754b98945a20cc5470e27b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:27.821902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aabbad62d754b98945a20cc5470e27b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:27.826221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:27.931362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aabbad62d754b98945a20cc5470e27b', 'request_id': 'req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:27.938522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aabbad62d754b98945a20cc5470e27b', 'request_id': 'req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:27.954789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aabbad62d754b98945a20cc5470e27b', 'request_id': 'req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:27.969981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] The token cache is being invalidate because project 6aabbad62d754b98945a20cc5470e27b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:27.970491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:27.971772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5a0ba13f-89df-40b7-bbdb-dc04ac6450cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:27.976599 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3454/6892] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:27 2022] DELETE /identity/v3/projects/6aabbad62d754b98945a20cc5470e27b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:27.978493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490f263-b94c-46b2-9b2a-dc8bb949ef88 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:27.978988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490f263-b94c-46b2-9b2a-dc8bb949ef88 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:27.979224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490f263-b94c-46b2-9b2a-dc8bb949ef88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.005113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2490f263-b94c-46b2-9b2a-dc8bb949ef88 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:28.006759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2490f263-b94c-46b2-9b2a-dc8bb949ef88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:28.085247 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3439/6893] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:28.094147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1785935-6a99-4283-b953-1f0d3e62779d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.094895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1785935-6a99-4283-b953-1f0d3e62779d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.095268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1785935-6a99-4283-b953-1f0d3e62779d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.119949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a1785935-6a99-4283-b953-1f0d3e62779d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:28.121322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a1785935-6a99-4283-b953-1f0d3e62779d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:28.130631 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3455/6894] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:28.135942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490f263-b94c-46b2-9b2a-dc8bb949ef88 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:28.147229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490f263-b94c-46b2-9b2a-dc8bb949ef88 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:28.152979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:28.153665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.153946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.154214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.154874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:28.155175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:28.156807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:28.173025 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3440/6895] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:28.174362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1785935-6a99-4283-b953-1f0d3e62779d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:28.185784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1785935-6a99-4283-b953-1f0d3e62779d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:28.191675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:28.192140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.192451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.192802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.193625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:28.195778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:28.226683 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3456/6896] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:28 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:28.230832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:28.241363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7771-89e2-4328-a04e-ab9d97c63181 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:28.304740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:28.305501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.305842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.306177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.306904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:28.309173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:28.328213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:28.340185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3441/6897] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:28 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:18:28.341157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:28.353302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b03dc-4702-40fa-9326-724802c6d2e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:28.359546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:28.360765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.361131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.361686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.362502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:28.364576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:28.367714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:28.386007 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3457/6898] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:28.387419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:28.401096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441df7a-cdca-4d85-8d5e-0e2985aa7b2a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:28.406325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e409ec-aa21-4b5e-a31e-bd85e57b4347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:28.406932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e409ec-aa21-4b5e-a31e-bd85e57b4347 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.407207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e409ec-aa21-4b5e-a31e-bd85e57b4347 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.407481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e409ec-aa21-4b5e-a31e-bd85e57b4347 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.408112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e409ec-aa21-4b5e-a31e-bd85e57b4347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:28.409906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e409ec-aa21-4b5e-a31e-bd85e57b4347 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:28.422294 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3442/6899] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:28.426758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:28.436940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793a506-f51c-4471-a774-3ce0c891a8bc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:28.441584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:28.442237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.442515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.442787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] PATH_INFO: `/v3/projects/e23900d23028416480c628993067f208/users/cc1fd61c78d04d65b4cf36052ae85970/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.457963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23900d23028416480c628993067f208, user_id=cc1fd61c78d04d65b4cf36052ae85970, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:28.460695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:28.474448 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3458/6900] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:28 2022] PUT /identity/v3/projects/e23900d23028416480c628993067f208/users/cc1fd61c78d04d65b4cf36052ae85970/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:28.485486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b565a1-ddf6-4f3f-bc58-2d043d460322 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.485812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b565a1-ddf6-4f3f-bc58-2d043d460322 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.486056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b565a1-ddf6-4f3f-bc58-2d043d460322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.515195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-76b565a1-ddf6-4f3f-bc58-2d043d460322 None None] MFA Rules not processed for user `cc1fd61c78d04d65b4cf36052ae85970`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:28.516409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76b565a1-ddf6-4f3f-bc58-2d043d460322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:28.591844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3443/6901] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:28.603846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:28.605678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:28.693184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8de54d-4f9e-4135-a664-4f84713d69c5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:28.697387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e7b01-532a-4e3a-b5ce-e50043025e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:28.698075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7b01-532a-4e3a-b5ce-e50043025e96 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:28.698354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7b01-532a-4e3a-b5ce-e50043025e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:28.698627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7b01-532a-4e3a-b5ce-e50043025e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:28.713042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e7b01-532a-4e3a-b5ce-e50043025e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:28.714778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e7b01-532a-4e3a-b5ce-e50043025e96 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:28.725596 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3459/6902] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:32.337499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b565a1-ddf6-4f3f-bc58-2d043d460322 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:32.481078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b565a1-ddf6-4f3f-bc58-2d043d460322 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:32.484368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de502e46-edd0-4bda-bcee-cc5c78057e97 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:32.484840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de502e46-edd0-4bda-bcee-cc5c78057e97 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:32.484919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de502e46-edd0-4bda-bcee-cc5c78057e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:32.486571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:32.487821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:32.488175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:32.488615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] PATH_INFO: `/v3/users/cc1fd61c78d04d65b4cf36052ae85970` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:32.489985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1fd61c78d04d65b4cf36052ae85970)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:32.492419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:32.567756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-de502e46-edd0-4bda-bcee-cc5c78057e97 None None] MFA Rules not processed for user `f08edf449e334897a129a1d26aedb507`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:32.569067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-de502e46-edd0-4bda-bcee-cc5c78057e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:32.577767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1fd61c78d04d65b4cf36052ae85970'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:32.601824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1fd61c78d04d65b4cf36052ae85970'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:32.616713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1fd61c78d04d65b4cf36052ae85970'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:32.617775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1fd61c78d04d65b4cf36052ae85970'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:32.618648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79d650ab-19b5-44f5-a6f4-57b30ad08a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1fd61c78d04d65b4cf36052ae85970'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:32.626422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:32.640565 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3444/6903] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:32 2022] DELETE /identity/v3/users/cc1fd61c78d04d65b4cf36052ae85970 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:32.645437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:32.645887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:32.646436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:32.679709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:32.681685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:32.703988 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3460/6904] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:32.714372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:32.715032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:32.715744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:32.778292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] MFA Rules not processed for user `f08edf449e334897a129a1d26aedb507`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:32.780057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:32.838418 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3445/6905] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:32.850161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:32.852090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:32.863052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3461/6906] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:32.881244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:32.887873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:32.954297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:33.018060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790e121-6114-42cc-beb0-1be1e2389578 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:33.030364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:33.030364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:33.030364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:33.030364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:33.059307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:33.060795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:33.072470 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3446/6907] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:33.116230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1421ac-768f-49bb-9e3b-1ff3b0bc9e31 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:33.123495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdefed8-0be1-4a93-8246-2328d02cb112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:33.124461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdefed8-0be1-4a93-8246-2328d02cb112 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:33.124911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdefed8-0be1-4a93-8246-2328d02cb112 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:33.125344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdefed8-0be1-4a93-8246-2328d02cb112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:33.136179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdefed8-0be1-4a93-8246-2328d02cb112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:33.138957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdefed8-0be1-4a93-8246-2328d02cb112 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:33.157797 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3462/6908] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:33.679249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:33.680859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:33.694916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6538728-a728-4a31-9ea0-a7f4eaaf6b73 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:33.700769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:33.702284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:33.702284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:33.702284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] PATH_INFO: `/v3/projects/e23900d23028416480c628993067f208` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:33.703690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23900d23028416480c628993067f208)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:33.705766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:33.762781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23900d23028416480c628993067f208', 'request_id': 'req-3dc7596f-d40f-4809-9016-2325c9ce099d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:33.775138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23900d23028416480c628993067f208', 'request_id': 'req-3dc7596f-d40f-4809-9016-2325c9ce099d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:33.784414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23900d23028416480c628993067f208', 'request_id': 'req-3dc7596f-d40f-4809-9016-2325c9ce099d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:33.798129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] The token cache is being invalidate because project e23900d23028416480c628993067f208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:33.799141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:33.800230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3dc7596f-d40f-4809-9016-2325c9ce099d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:33.802319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3447/6909] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:33 2022] DELETE /identity/v3/projects/e23900d23028416480c628993067f208 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:33.808297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd4d7-1aba-4a11-b6df-74b0aab5b3f8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:33.808859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd4d7-1aba-4a11-b6df-74b0aab5b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:33.808859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd4d7-1aba-4a11-b6df-74b0aab5b3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:33.832096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-614dd4d7-1aba-4a11-b6df-74b0aab5b3f8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:33.833312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-614dd4d7-1aba-4a11-b6df-74b0aab5b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:33.933583 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3463/6910] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:33.937193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33a4d5-a55b-4a8c-bb88-6a07738e2fff None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:33.937691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33a4d5-a55b-4a8c-bb88-6a07738e2fff None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:33.938038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33a4d5-a55b-4a8c-bb88-6a07738e2fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:33.969773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4f33a4d5-a55b-4a8c-bb88-6a07738e2fff None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:33.971087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4f33a4d5-a55b-4a8c-bb88-6a07738e2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:33.985732 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3448/6911] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:33.989964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd4d7-1aba-4a11-b6df-74b0aab5b3f8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:34.001482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd4d7-1aba-4a11-b6df-74b0aab5b3f8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:34.007094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:34.007959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.008330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.008721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.009794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:34.009936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:34.011827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:34.032582 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3464/6912] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:34.037367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33a4d5-a55b-4a8c-bb88-6a07738e2fff None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:34.048297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33a4d5-a55b-4a8c-bb88-6a07738e2fff None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:34.062119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:34.063110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.063711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.064122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.064907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:34.068096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:34.121601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:34.145796 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3449/6913] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:34 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:34.154319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:34.171789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993188-1dee-4146-a2f7-f1a8abcc3269 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:34.283368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:34.284006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.284277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.284542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.286362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:34.287133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:34.325896 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3465/6914] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:34 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:34.330727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:34.344104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc95dc-e020-4f7f-8055-175e401eda98 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:34.361518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:34.361518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.361518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.361518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.361934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:34.362599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:34.393434 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3450/6915] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:34.398119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:34.410747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f022c6-6d9b-4f9f-9152-bfc58240a266 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:34.425407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487cfd3-4337-4bcb-8b76-294b0fb4d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:34.426193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cfd3-4337-4bcb-8b76-294b0fb4d78f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.426486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cfd3-4337-4bcb-8b76-294b0fb4d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.426758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cfd3-4337-4bcb-8b76-294b0fb4d78f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.427507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487cfd3-4337-4bcb-8b76-294b0fb4d78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:34.429277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487cfd3-4337-4bcb-8b76-294b0fb4d78f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:34.452954 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3466/6916] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:34.454552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:34.466788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d35902-7104-40a3-b6c3-43eb5bfbbcd3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:34.481861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:34.482548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.482833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.483105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] PATH_INFO: `/v3/projects/335955c0be3743eda2541b7d04ed2e06/users/fd15b9d354eb44b0bfa83bcdcf5c1384/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.524477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=335955c0be3743eda2541b7d04ed2e06, user_id=fd15b9d354eb44b0bfa83bcdcf5c1384, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:34.529920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:34.562803 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3451/6917] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:34 2022] PUT /identity/v3/projects/335955c0be3743eda2541b7d04ed2e06/users/fd15b9d354eb44b0bfa83bcdcf5c1384/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:34.568620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f0dc6-18b3-4ab2-927e-a2669f4138b0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.568905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f0dc6-18b3-4ab2-927e-a2669f4138b0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.569356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f0dc6-18b3-4ab2-927e-a2669f4138b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.580657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:34.611460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6a3f0dc6-18b3-4ab2-927e-a2669f4138b0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:34.613020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6a3f0dc6-18b3-4ab2-927e-a2669f4138b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:34.745993 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3467/6918] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:34.755042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:34.758432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:34.843456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18846fb9-b26f-400c-8d64-cf9faea9bbbd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:34.847994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:34.848870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:34.849252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:34.849717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:34.861361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:34.863008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:34.876321 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3452/6919] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:35.835410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:35.835945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:35.836187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:35.871617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] MFA Rules not processed for user `fd15b9d354eb44b0bfa83bcdcf5c1384`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:35.873215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:35.982652 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3468/6920] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:35.993056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:35.995225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:36.067081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e952ae-fabd-4517-b4c3-58a7104b3e17 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:36.071079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:36.071750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:36.071980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:36.072254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:36.087759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:36.089616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:36.104122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3453/6921] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:41.126670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:41.134109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:41.290132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebcfdd-ec04-48f4-91c7-cc49f34f0670 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:41.298358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed542f5-8196-4280-8e7f-28ee98815e18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:41.299806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed542f5-8196-4280-8e7f-28ee98815e18 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:41.300060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed542f5-8196-4280-8e7f-28ee98815e18 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:41.300402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed542f5-8196-4280-8e7f-28ee98815e18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:41.303540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fed542f5-8196-4280-8e7f-28ee98815e18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:41.321519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed542f5-8196-4280-8e7f-28ee98815e18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:41.322874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed542f5-8196-4280-8e7f-28ee98815e18 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:41.338847 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3469/6922] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:44.662011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:44.683888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a6f5-d587-486e-8976-8b45dedc31eb service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:44.692132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:44.692904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:44.693292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:44.693849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:44.698904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:44.728006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:44.729687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:44.751997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3454/6923] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:48.710590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:48.711155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:48.711155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:48.749153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:48.800939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:48.800939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:48.945717 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3470/6924] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:49.048593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:49.050112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:49.191011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:49.232628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979152a-32de-4ea1-b54d-e238b2f87474 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:49.245484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000ab70-3009-42a2-8e46-0a675772c3bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:49.246675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ab70-3009-42a2-8e46-0a675772c3bb service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:49.246675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ab70-3009-42a2-8e46-0a675772c3bb service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:49.248980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ab70-3009-42a2-8e46-0a675772c3bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:49.261182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000ab70-3009-42a2-8e46-0a675772c3bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:49.268986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000ab70-3009-42a2-8e46-0a675772c3bb service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:49.290176 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3455/6925] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:18:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:18:50.861140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:50.862707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:50.895464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e09a64-7441-42e0-819b-763698f4c5b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:50.900584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:50.901772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:50.902199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:50.902605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] PATH_INFO: `/v3/users/df2f60fa0bde4397aaabf6caa8e4efe2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:50.904205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] RBAC: Authorizing `identity:delete_user(user_id=df2f60fa0bde4397aaabf6caa8e4efe2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:50.906941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:50.992010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df2f60fa0bde4397aaabf6caa8e4efe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:50.992695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df2f60fa0bde4397aaabf6caa8e4efe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:51.006009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df2f60fa0bde4397aaabf6caa8e4efe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:51.007472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df2f60fa0bde4397aaabf6caa8e4efe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:51.043355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df2f60fa0bde4397aaabf6caa8e4efe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:51.075988 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3471/6926] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:50 2022] DELETE /identity/v3/users/df2f60fa0bde4397aaabf6caa8e4efe2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:51.083359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:51.084503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:51.084786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:51.119850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:51.122381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:51.226407 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3456/6927] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:51.239707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:51.241641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:51.364053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d21f9c-3d4e-4dc4-9fc2-5bf4eb60fabd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:51.369600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:51.370541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:51.373678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:51.373678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:51.395299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:51.397121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:51.422536 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3472/6928] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:52.377311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:52.378808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:52.391232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a076f-c119-457d-90b2-dc6d8ba5360a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:52.396885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:52.397943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:52.398508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:52.398694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] PATH_INFO: `/v3/projects/f5457492c1db4888802770dcf7055c1d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:52.399936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5457492c1db4888802770dcf7055c1d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:52.401935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:52.480175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5457492c1db4888802770dcf7055c1d', 'request_id': 'req-72eab9ec-6068-41e9-a369-f5a78ce136c6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.502846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5457492c1db4888802770dcf7055c1d', 'request_id': 'req-72eab9ec-6068-41e9-a369-f5a78ce136c6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.516308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5457492c1db4888802770dcf7055c1d', 'request_id': 'req-72eab9ec-6068-41e9-a369-f5a78ce136c6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.537902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] The token cache is being invalidate because project f5457492c1db4888802770dcf7055c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:52.538263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.538986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-72eab9ec-6068-41e9-a369-f5a78ce136c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.544113 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3457/6929] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:52 2022] DELETE /identity/v3/projects/f5457492c1db4888802770dcf7055c1d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:52.545089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:52.560320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a68ae-b597-48ca-ac6a-99f1579a37d9 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:52.594895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:52.665573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:52.666345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:52.666721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:52.667030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] PATH_INFO: `/v3/users/13e4c127ef5d4ba4aeafd228de0d9254` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:52.675512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e4c127ef5d4ba4aeafd228de0d9254)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:52.677596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:52.762329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e4c127ef5d4ba4aeafd228de0d9254'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.762906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e4c127ef5d4ba4aeafd228de0d9254'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.772482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e4c127ef5d4ba4aeafd228de0d9254'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.773499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e4c127ef5d4ba4aeafd228de0d9254'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.792425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e4c127ef5d4ba4aeafd228de0d9254'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:52.821225 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3473/6930] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:52 2022] DELETE /identity/v3/users/13e4c127ef5d4ba4aeafd228de0d9254 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:52.947906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20b9e-ff33-44db-bed8-399ba834098c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:52.948201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20b9e-ff33-44db-bed8-399ba834098c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:52.948481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20b9e-ff33-44db-bed8-399ba834098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:53.006854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-84f20b9e-ff33-44db-bed8-399ba834098c None None] MFA Rules not processed for user `fd15b9d354eb44b0bfa83bcdcf5c1384`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:53.008178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-84f20b9e-ff33-44db-bed8-399ba834098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:53.082125 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3458/6931] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:53.297212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:53.298748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:53.407657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71435b-7c50-4116-9c19-03134c73b190 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:53.416684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:53.417572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:53.418024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:53.418446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] PATH_INFO: `/v3/projects/cd108287b7204dc59995bd50ce0e5428` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:53.426896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd108287b7204dc59995bd50ce0e5428)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:53.434324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:53.508421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:53.513640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd108287b7204dc59995bd50ce0e5428', 'request_id': 'req-b983af73-1bb6-4731-b064-78c57233184d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:53.521324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd108287b7204dc59995bd50ce0e5428', 'request_id': 'req-b983af73-1bb6-4731-b064-78c57233184d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:53.541226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd108287b7204dc59995bd50ce0e5428', 'request_id': 'req-b983af73-1bb6-4731-b064-78c57233184d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:53.557404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] The token cache is being invalidate because project cd108287b7204dc59995bd50ce0e5428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:53.558696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:53.559780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:53.562969 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3474/6932] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:53 2022] DELETE /identity/v3/projects/cd108287b7204dc59995bd50ce0e5428 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:53.568197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163b44f-4463-4cd6-a515-a42ff99c2f78 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:53.568782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163b44f-4463-4cd6-a515-a42ff99c2f78 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:53.568782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163b44f-4463-4cd6-a515-a42ff99c2f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:53.589623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:53.591071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:53.609273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0163b44f-4463-4cd6-a515-a42ff99c2f78 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:53.611082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0163b44f-4463-4cd6-a515-a42ff99c2f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:53.625109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:53.696725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983af73-1bb6-4731-b064-78c57233184d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:53.714207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:53.715383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:53.715665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:53.715935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:53.727120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:53.740774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:53.744465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:53.756352 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3459/6933] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:53.762533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967399c-9dc0-458d-93b7-e3d4d29bdaed None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:53.762820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967399c-9dc0-458d-93b7-e3d4d29bdaed None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:53.763112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967399c-9dc0-458d-93b7-e3d4d29bdaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:53.775770 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3475/6934] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:53.800152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5967399c-9dc0-458d-93b7-e3d4d29bdaed None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:53.805906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5967399c-9dc0-458d-93b7-e3d4d29bdaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:53.823998 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3460/6935] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:53.825657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:53.845727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eeb19-6835-42e3-8e20-91d4b18f333f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:53.855614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:53.856250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:53.856526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:53.856939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:53.857635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:53.858346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:53.865742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:53.893656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3476/6936] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:53.895305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967399c-9dc0-458d-93b7-e3d4d29bdaed None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:53.907713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967399c-9dc0-458d-93b7-e3d4d29bdaed None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:53.919673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:53.920593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:53.921039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:53.921316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:53.922829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:53.927761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:54.002952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3461/6937] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:53 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:54.007838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:54.024090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86407720-7243-4008-a0e4-9d05dda3cb39 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:54.171541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:54.172349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:54.172787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:54.173201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:54.174255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:54.176353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:54.232151 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3477/6938] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:54 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:18:54.234343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:54.257799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da2cd1-2b77-472e-bf19-91883fb6bbfc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:54.276512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:54.277146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:54.277430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:54.277737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:54.278392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:54.280024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:54.307172 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3462/6939] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:54.311115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:54.326254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28ee16-019d-4862-8402-9ca0c14b1bbc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:54.341166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e77495-8f66-4a73-9191-d474ad91a9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:54.342940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e77495-8f66-4a73-9191-d474ad91a9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:54.344902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e77495-8f66-4a73-9191-d474ad91a9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:54.346362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e77495-8f66-4a73-9191-d474ad91a9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:54.349918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e77495-8f66-4a73-9191-d474ad91a9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:54.353381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e77495-8f66-4a73-9191-d474ad91a9d3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:54.388731 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3478/6940] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:54.390048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:54.401842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d378bd-d676-436e-b364-f11b1a56cfdc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:54.414220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:54.414999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:54.415372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:54.415740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] PATH_INFO: `/v3/projects/0bb75145fbc84eac924ff1ca0838225f/users/508fb2130c0448f681915f6c10e7a19a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:54.442020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb75145fbc84eac924ff1ca0838225f, user_id=508fb2130c0448f681915f6c10e7a19a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:54.446579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:54.488551 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3463/6941] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:54 2022] PUT /identity/v3/projects/0bb75145fbc84eac924ff1ca0838225f/users/508fb2130c0448f681915f6c10e7a19a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:54.513727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650961d1-bd2a-41f7-a1a1-6f17beeea83f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:54.514869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650961d1-bd2a-41f7-a1a1-6f17beeea83f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:54.515673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650961d1-bd2a-41f7-a1a1-6f17beeea83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:54.566261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-650961d1-bd2a-41f7-a1a1-6f17beeea83f None None] MFA Rules not processed for user `508fb2130c0448f681915f6c10e7a19a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:54.567908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-650961d1-bd2a-41f7-a1a1-6f17beeea83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:54.637417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:54.781531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3479/6942] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:54.786185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:54.958687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c013e0f-16d9-4b77-841b-7c27256b2208 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:54.964423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:54.965095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:54.965371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:54.965734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:54.966481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:54.970863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:55.047651 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3464/6943] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:54 2022] POST /identity/v3/projects => generated 373 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:55.049501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650961d1-bd2a-41f7-a1a1-6f17beeea83f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:55.068957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650961d1-bd2a-41f7-a1a1-6f17beeea83f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:55.195646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:55.196341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:55.196625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:55.196942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:55.197626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:55.201857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:55.255475 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3480/6944] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:55 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:18:55.259597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:55.282112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c953d6-12c1-481c-bc6d-c3635500122c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:55.295482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:55.298517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:55.300277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:55.301639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:55.303944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:55.307763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:55.330629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:55.338617 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3465/6945] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:55.344372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b337d1-043f-4970-af1c-cbedd58930a4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:55.350017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:55.350725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:55.351010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:55.351463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] PATH_INFO: `/v3/projects/c4493c0bea6d4ed2ae8dbd1ec846463f/users/bf76aaf6dac3407092808722a6270a57/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:55.373548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4493c0bea6d4ed2ae8dbd1ec846463f, user_id=bf76aaf6dac3407092808722a6270a57, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:55.381418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:55.436477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:55.436934 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3481/6946] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:55 2022] PUT /identity/v3/projects/c4493c0bea6d4ed2ae8dbd1ec846463f/users/bf76aaf6dac3407092808722a6270a57/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:55.447886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c03f96-a821-4a71-a0b2-fbc8199d517a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:55.468292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:55.560630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa958871-d33c-45c5-8b48-44dfba426299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:55.561236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa958871-d33c-45c5-8b48-44dfba426299 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:55.561899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa958871-d33c-45c5-8b48-44dfba426299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:55.562175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa958871-d33c-45c5-8b48-44dfba426299 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:55.563219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa958871-d33c-45c5-8b48-44dfba426299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:55.568156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa958871-d33c-45c5-8b48-44dfba426299 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:55.596587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:55.597859 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3466/6947] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:55.609547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f7762-fe07-48c2-bb89-3477ef3ffd07 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:55.623728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:55.624472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:55.624752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:55.625013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] PATH_INFO: `/v3/projects/c4493c0bea6d4ed2ae8dbd1ec846463f/users/bf76aaf6dac3407092808722a6270a57/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:55.629382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4493c0bea6d4ed2ae8dbd1ec846463f, user_id=bf76aaf6dac3407092808722a6270a57, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:55.633635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:55.660623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:55.660910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:55.661181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:55.678452 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3482/6948] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:55 2022] PUT /identity/v3/projects/c4493c0bea6d4ed2ae8dbd1ec846463f/users/bf76aaf6dac3407092808722a6270a57/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:55.741740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] MFA Rules not processed for user `bf76aaf6dac3407092808722a6270a57`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:55.743813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:55.921900 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3467/6949] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:55.935342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:55.937675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:56.059521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b119da7-b82f-4c08-a996-3a91c7a7860d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:56.064149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:56.064781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:56.065055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:56.065329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:56.077962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:56.079470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:56.118345 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3483/6950] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:56.589331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:56.589331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:56.785300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:56.828394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941ee0c-0007-43f4-8bdd-6dba3c8c9944 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:56.834289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:56.836985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:56.837674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:56.837954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:56.838227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] PATH_INFO: `/v3/users/f08edf449e334897a129a1d26aedb507` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:56.841425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08edf449e334897a129a1d26aedb507)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:56.843484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:56.900021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dca37-8571-4558-b806-54a0f28add41 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:56.906926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:56.908393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:56.909792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:56.910369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] PATH_INFO: `/v3/users/508fb2130c0448f681915f6c10e7a19a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:56.912005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508fb2130c0448f681915f6c10e7a19a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:56.914454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:56.927322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08edf449e334897a129a1d26aedb507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:56.934353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:56.947242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08edf449e334897a129a1d26aedb507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:56.962538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08edf449e334897a129a1d26aedb507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:56.964774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08edf449e334897a129a1d26aedb507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:56.970144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cb2774b1-aee6-4f3b-8c45-33c2edf0be8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08edf449e334897a129a1d26aedb507'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:56.987196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3468/6951] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:56 2022] DELETE /identity/v3/users/f08edf449e334897a129a1d26aedb507 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:57.006913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508fb2130c0448f681915f6c10e7a19a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.011838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508fb2130c0448f681915f6c10e7a19a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.023260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508fb2130c0448f681915f6c10e7a19a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.024186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508fb2130c0448f681915f6c10e7a19a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.051467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508fb2130c0448f681915f6c10e7a19a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.069637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3484/6952] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:56 2022] DELETE /identity/v3/users/508fb2130c0448f681915f6c10e7a19a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:18:57.075434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:57.076088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:57.076739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:57.142178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:57.145351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:57.301269 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3469/6953] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:57.307706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:57.309175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:57.317351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:57.318944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:57.329497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7c62f-0905-4d2b-b466-f7d9d7bee435 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:57.337042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:57.338866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:57.339697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:57.340033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] PATH_INFO: `/v3/projects/a37593cb29164fa2bbb0a54a758fa3bd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:57.347889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37593cb29164fa2bbb0a54a758fa3bd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:57.351462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:57.425096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3e00-ace8-4239-ba02-4d7f69a04e24 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:57.430331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da98d20-ae6d-4f6e-86cd-bdcbb42c5a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:57.430718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da98d20-ae6d-4f6e-86cd-bdcbb42c5a42 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:57.431273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da98d20-ae6d-4f6e-86cd-bdcbb42c5a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:57.431605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da98d20-ae6d-4f6e-86cd-bdcbb42c5a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:57.456241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da98d20-ae6d-4f6e-86cd-bdcbb42c5a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:57.458695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da98d20-ae6d-4f6e-86cd-bdcbb42c5a42 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:57.482900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37593cb29164fa2bbb0a54a758fa3bd', 'request_id': 'req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.489684 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3470/6954] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:57.493359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37593cb29164fa2bbb0a54a758fa3bd', 'request_id': 'req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.514947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37593cb29164fa2bbb0a54a758fa3bd', 'request_id': 'req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.558559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] The token cache is being invalidate because project a37593cb29164fa2bbb0a54a758fa3bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:57.559138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.560935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6b3e97c-8e4d-433c-ae22-ffbd61f36b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:57.574667 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3485/6955] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:57 2022] DELETE /identity/v3/projects/a37593cb29164fa2bbb0a54a758fa3bd => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:57.576744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5692d1c-ad35-45f9-af7a-5ad55d233783 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:57.577095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5692d1c-ad35-45f9-af7a-5ad55d233783 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:57.578264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5692d1c-ad35-45f9-af7a-5ad55d233783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:57.641980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a5692d1c-ad35-45f9-af7a-5ad55d233783 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:57.643390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a5692d1c-ad35-45f9-af7a-5ad55d233783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:57.773330 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3471/6956] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:18:57.782566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a9d6-313d-4e5b-9845-7e695b78b78a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:57.782923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a9d6-313d-4e5b-9845-7e695b78b78a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:57.783274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a9d6-313d-4e5b-9845-7e695b78b78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:57.807706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1551a9d6-313d-4e5b-9845-7e695b78b78a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:57.809035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1551a9d6-313d-4e5b-9845-7e695b78b78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:57.823512 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3486/6957] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:57.824276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5692d1c-ad35-45f9-af7a-5ad55d233783 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:57.836098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:57.854845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5692d1c-ad35-45f9-af7a-5ad55d233783 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:57.862810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:57.863561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:57.864059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:57.864425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:57.865377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:57.865757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:57.867819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:57.888914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a9d6-313d-4e5b-9845-7e695b78b78a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:57.895480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3472/6958] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:18:57.901648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a9d6-313d-4e5b-9845-7e695b78b78a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:57.908771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:57.910073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:57.910861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:57.911225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:57.911979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:57.914317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:57.959731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:57.981244 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3487/6959] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:57 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:57.982961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.004570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e2dd2-1b41-4977-acc9-97a22085a39d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.023736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.025601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:58.150596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:58.151472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.151844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.152244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.153054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:58.155433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:58.177671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecec0d-5cc9-4d12-b371-bbf1f54e421b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.192544 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3473/6960] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:57 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:58.195969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:58.196792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.197163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.197632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.197852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] PATH_INFO: `/v3/projects/0bb75145fbc84eac924ff1ca0838225f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.199855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb75145fbc84eac924ff1ca0838225f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:58.203146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:58.212392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7871c1-557b-45a3-9420-7e965a2d7671 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.218816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:58.219764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.221432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.222007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.223916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:58.227782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:58.246832 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3474/6961] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:58.250741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.270903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f9df1-0e27-461b-b319-bbb74746fe27 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.300175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb75145fbc84eac924ff1ca0838225f', 'request_id': 'req-888004ba-8817-44fd-b9ca-05d25a525ba0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.308701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb75145fbc84eac924ff1ca0838225f', 'request_id': 'req-888004ba-8817-44fd-b9ca-05d25a525ba0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.337332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb75145fbc84eac924ff1ca0838225f', 'request_id': 'req-888004ba-8817-44fd-b9ca-05d25a525ba0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.377199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:58.378414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.378800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.379165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.380068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:58.382276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:58.396944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] The token cache is being invalidate because project 0bb75145fbc84eac924ff1ca0838225f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:58.398996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.400729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.406381 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3488/6962] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:58 2022] DELETE /identity/v3/projects/0bb75145fbc84eac924ff1ca0838225f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:18:58.407491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.411024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:58.420561 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3475/6963] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:18:58.427973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.432547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:58.433590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888004ba-8817-44fd-b9ca-05d25a525ba0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.442451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:58.443566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.443852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.444299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] PATH_INFO: `/v3/users/bf76aaf6dac3407092808722a6270a57` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.445728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf76aaf6dac3407092808722a6270a57)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:58.447815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:58.451468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06f7d3-e555-4c36-be40-cd1967dd8224 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.461519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfab50-1325-4c53-9564-3fca1c240e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:58.463340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfab50-1325-4c53-9564-3fca1c240e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.463823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfab50-1325-4c53-9564-3fca1c240e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.464243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfab50-1325-4c53-9564-3fca1c240e0d admin admin] PATH_INFO: `/v3/projects/05516767841641de8a3c48312f31fa3f/users/18fa7550fc73419dbc0a56127289462d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.488086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfab50-1325-4c53-9564-3fca1c240e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05516767841641de8a3c48312f31fa3f, user_id=18fa7550fc73419dbc0a56127289462d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:58.491359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfab50-1325-4c53-9564-3fca1c240e0d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:58.521505 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3476/6964] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:58 2022] PUT /identity/v3/projects/05516767841641de8a3c48312f31fa3f/users/18fa7550fc73419dbc0a56127289462d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:58.531563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.531889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.532144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.540064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf76aaf6dac3407092808722a6270a57'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.541966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf76aaf6dac3407092808722a6270a57'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.581259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] MFA Rules not processed for user `18fa7550fc73419dbc0a56127289462d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:58.582826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:58.599259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf76aaf6dac3407092808722a6270a57'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.600502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf76aaf6dac3407092808722a6270a57'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.626843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:58.632209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf76aaf6dac3407092808722a6270a57'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:58.662467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:58.669940 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3489/6965] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:58 2022] DELETE /identity/v3/users/bf76aaf6dac3407092808722a6270a57 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:58.696310 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3477/6966] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:58.709507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.712305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:58.782497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:58.784069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:58.844655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b131e-f1c6-4c72-b240-6032a70c3c93 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.850183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:58.851123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:58.851468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:58.851795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:58.938266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:58.943560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:58.959254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3629d-ad41-401e-8faf-4c8ce1045310 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:58.976123 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3490/6967] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:59.063560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac8a73-653f-431e-8717-4632d75d8576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:59.065790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8a73-653f-431e-8717-4632d75d8576 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:59.065790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8a73-653f-431e-8717-4632d75d8576 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:59.065790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8a73-653f-431e-8717-4632d75d8576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:59.069666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-66ac8a73-653f-431e-8717-4632d75d8576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:59.098791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8a73-653f-431e-8717-4632d75d8576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:59.101194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8a73-653f-431e-8717-4632d75d8576 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:59.121304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:59.127143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:59.129361 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3478/6968] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:18:59.342737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9d8d6-4c04-4850-98c6-b7d517b59645 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:59.347837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:59.354736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:59.355111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:59.355413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] PATH_INFO: `/v3/projects/c4493c0bea6d4ed2ae8dbd1ec846463f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:59.356768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4493c0bea6d4ed2ae8dbd1ec846463f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:59.358811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:59.483346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4493c0bea6d4ed2ae8dbd1ec846463f', 'request_id': 'req-b621463f-eed8-421c-9b69-9c7990cfe9ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:59.492318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4493c0bea6d4ed2ae8dbd1ec846463f', 'request_id': 'req-b621463f-eed8-421c-9b69-9c7990cfe9ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:59.496282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:59.505254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4493c0bea6d4ed2ae8dbd1ec846463f', 'request_id': 'req-b621463f-eed8-421c-9b69-9c7990cfe9ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:59.529119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] The token cache is being invalidate because project c4493c0bea6d4ed2ae8dbd1ec846463f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:59.529881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:59.534171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b621463f-eed8-421c-9b69-9c7990cfe9ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:59.541350 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3491/6969] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:59 2022] DELETE /identity/v3/projects/c4493c0bea6d4ed2ae8dbd1ec846463f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:18:59.547458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dec07-62c4-49be-a47b-7d260ceeb652 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:59.548234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dec07-62c4-49be-a47b-7d260ceeb652 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:59.548740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dec07-62c4-49be-a47b-7d260ceeb652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:59.595128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-767dec07-62c4-49be-a47b-7d260ceeb652 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:59.595128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-767dec07-62c4-49be-a47b-7d260ceeb652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:59.715861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:59.745249 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3479/6970] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:59.753686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9a612-9e83-45c1-b1d6-9fbed02b1f6f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:59.754178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9a612-9e83-45c1-b1d6-9fbed02b1f6f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:59.754660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9a612-9e83-45c1-b1d6-9fbed02b1f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:59.814141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d8d9a612-9e83-45c1-b1d6-9fbed02b1f6f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:59.815354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d8d9a612-9e83-45c1-b1d6-9fbed02b1f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:59.832254 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3492/6971] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:59.834674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dec07-62c4-49be-a47b-7d260ceeb652 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:59.848046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dec07-62c4-49be-a47b-7d260ceeb652 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:59.853371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:59.854833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:59.855115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:59.855383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:59.856036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:59.856331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:59.858153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:59.875430 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3480/6972] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:59.880388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9a612-9e83-45c1-b1d6-9fbed02b1f6f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:59.896136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9a612-9e83-45c1-b1d6-9fbed02b1f6f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:59.907934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:59.908590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:59.908829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:59.909096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:59.909858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:59.912701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:59.970474 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3493/6973] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:59 2022] POST /identity/v3/projects => generated 363 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:59.975758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:59.988025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc202-5ea9-4ce9-8ec7-37466b44b44b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.072138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.073685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.074303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.074582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.075358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.077299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.114495 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3481/6974] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:59 2022] POST /identity/v3/users => generated 282 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:00.118506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.132872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab799db5-5ade-4b87-ac4b-6c951a16a225 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.138024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.138646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.138923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.139190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.139845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.141484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.152490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3494/6975] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:00.156991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.169268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8e97b-86a1-44bd-b26c-0795da61a073 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.176583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a35a0e-8458-43dc-9549-285269dbc778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.177397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35a0e-8458-43dc-9549-285269dbc778 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.178180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35a0e-8458-43dc-9549-285269dbc778 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.181801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35a0e-8458-43dc-9549-285269dbc778 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.182506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a35a0e-8458-43dc-9549-285269dbc778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.184211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a35a0e-8458-43dc-9549-285269dbc778 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.198681 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3482/6976] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:00.203740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.216520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9abf21-db5b-473a-afd4-fe0968479c95 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.223185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.224780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.225749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.226258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] PATH_INFO: `/v3/projects/e1c7cf7ba72a4c8295cb0abe6cbabbc8/users/873d452fecac4d5db86e3dc072270252/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.245058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c7cf7ba72a4c8295cb0abe6cbabbc8, user_id=873d452fecac4d5db86e3dc072270252, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.247847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.267931 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3495/6977] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:00 2022] PUT /identity/v3/projects/e1c7cf7ba72a4c8295cb0abe6cbabbc8/users/873d452fecac4d5db86e3dc072270252/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:00.277813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa48a-2e19-4f5e-a513-9573eb681b8c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.278109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa48a-2e19-4f5e-a513-9573eb681b8c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.278391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa48a-2e19-4f5e-a513-9573eb681b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.315213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-67ffa48a-2e19-4f5e-a513-9573eb681b8c None None] MFA Rules not processed for user `873d452fecac4d5db86e3dc072270252`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:00.316764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-67ffa48a-2e19-4f5e-a513-9573eb681b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:00.421393 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3483/6978] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:00.426536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.509816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec9eb2-cd97-4217-a679-63441cf2c9ec admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.515351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.516046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.516387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.516715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.517494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.519503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.570314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3496/6979] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:00 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:00.571765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa48a-2e19-4f5e-a513-9573eb681b8c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.591573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa48a-2e19-4f5e-a513-9573eb681b8c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.674697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:00.708172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.708915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.709214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.709513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.710327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.712824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.761516 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3484/6980] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:00 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:00.766317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.778230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52808537-5a82-40a3-bf85-4b4c2b554401 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.785023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.786133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.786656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.787234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.788214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.790196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.829890 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3497/6981] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:00.834399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.849413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f22ae-997a-4382-9347-752147418a04 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:00.866031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:00.866734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:00.867014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:00.867289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] PATH_INFO: `/v3/projects/90c8fc4e85664e7690066c0acae0de58/users/3d1418970f2e4192a7e8508960f828b9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:00.885902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c8fc4e85664e7690066c0acae0de58, user_id=3d1418970f2e4192a7e8508960f828b9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:00.889923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:00.913105 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3485/6982] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:00 2022] PUT /identity/v3/projects/90c8fc4e85664e7690066c0acae0de58/users/3d1418970f2e4192a7e8508960f828b9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:00.917488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:00.940435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970df0f-e330-44e7-9352-cdffc634655e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:01.028278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:01.040738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640ebeb-51a2-46a8-b1b8-8e5940bae6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:01.041832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ebeb-51a2-46a8-b1b8-8e5940bae6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:01.042317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ebeb-51a2-46a8-b1b8-8e5940bae6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:01.042790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ebeb-51a2-46a8-b1b8-8e5940bae6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:01.043666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640ebeb-51a2-46a8-b1b8-8e5940bae6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:01.045784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640ebeb-51a2-46a8-b1b8-8e5940bae6a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:01.096513 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3498/6983] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:01.097870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:01.175221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c026b-3ac9-4daa-a615-77cb975ec5fc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:01.206820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:01.207823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:01.207968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:01.208308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] PATH_INFO: `/v3/projects/90c8fc4e85664e7690066c0acae0de58/users/3d1418970f2e4192a7e8508960f828b9/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:01.211824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c8fc4e85664e7690066c0acae0de58, user_id=3d1418970f2e4192a7e8508960f828b9, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:01.215190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:01.250922 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3486/6984] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:01 2022] PUT /identity/v3/projects/90c8fc4e85664e7690066c0acae0de58/users/3d1418970f2e4192a7e8508960f828b9/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:01.263170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748d0d4-e674-4b9e-9e5f-573a55fe55d5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:01.263536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748d0d4-e674-4b9e-9e5f-573a55fe55d5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:01.263949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748d0d4-e674-4b9e-9e5f-573a55fe55d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:01.303611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0748d0d4-e674-4b9e-9e5f-573a55fe55d5 None None] MFA Rules not processed for user `3d1418970f2e4192a7e8508960f828b9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:01.307360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0748d0d4-e674-4b9e-9e5f-573a55fe55d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:01.396299 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3499/6985] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:01.408125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:01.410529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:01.505107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f4077-ccfd-4dd9-a4b8-0b1bfce95f40 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:01.509798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dfcfe-b7d8-4c32-9f31-8c67672abf35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:01.510437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dfcfe-b7d8-4c32-9f31-8c67672abf35 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:01.510713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dfcfe-b7d8-4c32-9f31-8c67672abf35 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:01.511007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dfcfe-b7d8-4c32-9f31-8c67672abf35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:01.522143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dfcfe-b7d8-4c32-9f31-8c67672abf35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:01.524435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dfcfe-b7d8-4c32-9f31-8c67672abf35 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:01.539444 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3487/6986] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:02.224209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748d0d4-e674-4b9e-9e5f-573a55fe55d5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:02.342799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748d0d4-e674-4b9e-9e5f-573a55fe55d5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:02.356469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:02.357145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:02.357427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:02.358110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] PATH_INFO: `/v3/users/873d452fecac4d5db86e3dc072270252` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:02.358913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] RBAC: Authorizing `identity:delete_user(user_id=873d452fecac4d5db86e3dc072270252)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:02.360984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:02.435636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873d452fecac4d5db86e3dc072270252'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:02.437054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873d452fecac4d5db86e3dc072270252'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:02.453187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873d452fecac4d5db86e3dc072270252'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:02.456876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873d452fecac4d5db86e3dc072270252'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:02.480008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873d452fecac4d5db86e3dc072270252'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:02.486594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:02.498831 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3500/6987] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:02 2022] DELETE /identity/v3/users/873d452fecac4d5db86e3dc072270252 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:02.503584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:02.504046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:02.504046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:02.533236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:02.534499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:02.632273 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3488/6988] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:02.642200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:02.643835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:02.787629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fb75a-7097-4321-ae64-4284bf7d031e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:02.794094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:02.794946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:02.795289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:02.796082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:02.814459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:02.828203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:02.854001 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3501/6989] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:03.549375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:03.560344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:03.575223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4cab-a500-4ada-900f-d2045b0eacb4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:03.590216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:03.591271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:03.591753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:03.592222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] PATH_INFO: `/v3/projects/e1c7cf7ba72a4c8295cb0abe6cbabbc8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:03.594064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c7cf7ba72a4c8295cb0abe6cbabbc8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:03.596742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:03.718937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:03.759950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c7cf7ba72a4c8295cb0abe6cbabbc8', 'request_id': 'req-cab5fcaa-3104-411e-8467-531b15e16e96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:03.793251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c7cf7ba72a4c8295cb0abe6cbabbc8', 'request_id': 'req-cab5fcaa-3104-411e-8467-531b15e16e96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:03.813395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c7cf7ba72a4c8295cb0abe6cbabbc8', 'request_id': 'req-cab5fcaa-3104-411e-8467-531b15e16e96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:03.856667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] The token cache is being invalidate because project e1c7cf7ba72a4c8295cb0abe6cbabbc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:03.862979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:03.866243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:03.871696 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3489/6990] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:03 2022] DELETE /identity/v3/projects/e1c7cf7ba72a4c8295cb0abe6cbabbc8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:03.875886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:03.894635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4954403-2db6-4664-bea6-04630926eb74 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:04.068566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:04.069280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:04.069582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:04.069850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] PATH_INFO: `/v3/users/3d1418970f2e4192a7e8508960f828b9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:04.072352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d1418970f2e4192a7e8508960f828b9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:04.074365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:04.199318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d1418970f2e4192a7e8508960f828b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:04.199662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d1418970f2e4192a7e8508960f828b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:04.223673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d1418970f2e4192a7e8508960f828b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:04.224499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d1418970f2e4192a7e8508960f828b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:04.246616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-85e4f462-eaab-4318-8e4d-aacd6d052f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d1418970f2e4192a7e8508960f828b9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:04.274829 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3502/6991] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:03 2022] DELETE /identity/v3/users/3d1418970f2e4192a7e8508960f828b9 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:05.020341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:05.023548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:05.057253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5fcaa-3104-411e-8467-531b15e16e96 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:05.165105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:05.166284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:05.166765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:05.167264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] PATH_INFO: `/v3/projects/90c8fc4e85664e7690066c0acae0de58` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:05.169809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c8fc4e85664e7690066c0acae0de58)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:05.176962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:05.319435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c8fc4e85664e7690066c0acae0de58', 'request_id': 'req-9083f5d6-1a39-493a-930f-2a6be203779d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:05.348553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c8fc4e85664e7690066c0acae0de58', 'request_id': 'req-9083f5d6-1a39-493a-930f-2a6be203779d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:05.372285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c8fc4e85664e7690066c0acae0de58', 'request_id': 'req-9083f5d6-1a39-493a-930f-2a6be203779d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:05.404269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] The token cache is being invalidate because project 90c8fc4e85664e7690066c0acae0de58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:05.404616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:05.405360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9083f5d6-1a39-493a-930f-2a6be203779d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:05.407771 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3490/6992] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:05 2022] DELETE /identity/v3/projects/90c8fc4e85664e7690066c0acae0de58 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:05.415595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584d128-2017-4447-99b4-0259951a80d3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:05.416827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584d128-2017-4447-99b4-0259951a80d3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:05.417255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584d128-2017-4447-99b4-0259951a80d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:05.455819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a584d128-2017-4447-99b4-0259951a80d3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:05.455819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a584d128-2017-4447-99b4-0259951a80d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:05.468281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:05.597152 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3503/6993] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:05.600508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca117-13a9-4151-975b-f6033245b23e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:05.601346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca117-13a9-4151-975b-f6033245b23e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:05.601654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca117-13a9-4151-975b-f6033245b23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:05.635424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8e6ca117-13a9-4151-975b-f6033245b23e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:05.637685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8e6ca117-13a9-4151-975b-f6033245b23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:05.649657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3491/6994] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:05.654120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584d128-2017-4447-99b4-0259951a80d3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:05.666501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584d128-2017-4447-99b4-0259951a80d3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:05.681152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:05.681576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:05.681922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:05.682285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:05.683604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:05.683982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:05.685838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:05.711597 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3504/6995] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:05.716083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ca117-13a9-4151-975b-f6033245b23e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:05.726079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ca117-13a9-4151-975b-f6033245b23e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:05.731228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:05.731842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:05.732809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:05.732809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:05.733007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:05.735022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:05.772920 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3492/6996] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:05 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:05.776443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:05.794246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742b7ff-b396-443e-adf8-f9ec6d1c47dd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:05.886932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:05.888248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:05.888248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:05.888448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:05.889397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:05.892110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:05.922923 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3505/6997] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:05 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:05.927305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:05.943248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cf64-456c-4c2c-bdfb-858f38048da6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:05.955965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:05.956609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:05.957003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:05.957744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:05.958592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:05.961145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:05.981872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:05.983095 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3493/6998] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:05.995055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3f484-e49d-4476-8e8f-06a6e0740e3d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.012287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80894670-d766-4e38-8258-b57b92bd2f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.013735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80894670-d766-4e38-8258-b57b92bd2f09 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.013849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80894670-d766-4e38-8258-b57b92bd2f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.014182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80894670-d766-4e38-8258-b57b92bd2f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.015325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80894670-d766-4e38-8258-b57b92bd2f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.017738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80894670-d766-4e38-8258-b57b92bd2f09 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.037545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3506/6999] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:06.042700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:06.056016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acb4e9-f52b-473b-afc4-659a84b2dc0e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.062196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.062918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.063268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.063549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] PATH_INFO: `/v3/projects/0dae56f733674335a6e579ed7da013e2/users/fc139e5ee8054df988ea2be82067172b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.079619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:06.087250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dae56f733674335a6e579ed7da013e2, user_id=fc139e5ee8054df988ea2be82067172b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.092432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.142272 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3494/7000] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:06 2022] PUT /identity/v3/projects/0dae56f733674335a6e579ed7da013e2/users/fc139e5ee8054df988ea2be82067172b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:06.144080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f2b6-d22d-44cd-aae3-54ee27f21f51 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.144593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f2b6-d22d-44cd-aae3-54ee27f21f51 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.145064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f2b6-d22d-44cd-aae3-54ee27f21f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.187743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c5d4f2b6-d22d-44cd-aae3-54ee27f21f51 None None] MFA Rules not processed for user `fc139e5ee8054df988ea2be82067172b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:06.188681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c5d4f2b6-d22d-44cd-aae3-54ee27f21f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:06.291630 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3507/7001] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:06.295778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:06.418529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ab8dc-150a-43f4-b341-9065c76093ef admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.425493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.426313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.426972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.427253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.428317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.431040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.482680 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3495/7002] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:06 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:06.486779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4f2b6-d22d-44cd-aae3-54ee27f21f51 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:06.506864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4f2b6-d22d-44cd-aae3-54ee27f21f51 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.526901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:06.633996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.635152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.635475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.635780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.636778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.643050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.704139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:06.706192 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3508/7003] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:06 2022] POST /identity/v3/users => generated 298 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:06.718317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74797661-53ea-45a1-8c2f-7a1bb3bf8742 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.726562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.727407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.727745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.728170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.728865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.730862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.746944 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3496/7004] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:06.752697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:06.765509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1cd0-95aa-4fb6-bc04-5855d1c206b8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.770303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.771046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.771392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.771731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] PATH_INFO: `/v3/projects/b4878231cfd54dccb0673184d0130553/users/a537532b84c64753b6420c978c95e041/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.789757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4878231cfd54dccb0673184d0130553, user_id=a537532b84c64753b6420c978c95e041, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.792441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.810473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3509/7005] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:06 2022] PUT /identity/v3/projects/b4878231cfd54dccb0673184d0130553/users/a537532b84c64753b6420c978c95e041/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:06.811901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:06.825832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8e4b9-75c2-40d0-884c-74f27ae28882 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.896896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ef385-7ad4-4027-bfac-080d0cabebf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.898143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef385-7ad4-4027-bfac-080d0cabebf7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.898380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef385-7ad4-4027-bfac-080d0cabebf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.898858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef385-7ad4-4027-bfac-080d0cabebf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.899570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ef385-7ad4-4027-bfac-080d0cabebf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.901406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ef385-7ad4-4027-bfac-080d0cabebf7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.918257 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3497/7006] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:06.920220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:06.942520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf276a4-eaf7-4352-af80-eed6159f2d98 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:06.955268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:06.955954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:06.956228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:06.956502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] PATH_INFO: `/v3/projects/b4878231cfd54dccb0673184d0130553/users/a537532b84c64753b6420c978c95e041/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:06.959563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4878231cfd54dccb0673184d0130553, user_id=a537532b84c64753b6420c978c95e041, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:06.962492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:06.997728 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3510/7007] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:06 2022] PUT /identity/v3/projects/b4878231cfd54dccb0673184d0130553/users/a537532b84c64753b6420c978c95e041/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:07.012049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:07.012334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:07.012612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:07.075756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] MFA Rules not processed for user `a537532b84c64753b6420c978c95e041`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:07.077381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:07.157781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:07.190259 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3498/7008] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:07.203481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:07.205294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:07.337132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832a8f-4038-48f3-aaad-04c95e807c89 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:07.343405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:07.344293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:07.344623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:07.345209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:07.358262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:07.360748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:07.382014 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3511/7009] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:07.386748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:07.397975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:07.521066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:07.530387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e979a-a55a-4c21-a085-43774846b216 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:07.536598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19578be7-f4fc-4797-9c6e-861dfdb97cd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:07.537180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5dae-c28c-4e24-a9dc-8d5af46c00b1 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:07.537312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19578be7-f4fc-4797-9c6e-861dfdb97cd5 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:07.537725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19578be7-f4fc-4797-9c6e-861dfdb97cd5 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:07.538045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19578be7-f4fc-4797-9c6e-861dfdb97cd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:07.540024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-19578be7-f4fc-4797-9c6e-861dfdb97cd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:07.546746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:07.547512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:07.547795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:07.548069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:07.626043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19578be7-f4fc-4797-9c6e-861dfdb97cd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:07.627957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19578be7-f4fc-4797-9c6e-861dfdb97cd5 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:07.645974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:07.647823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:07.648575 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3499/7010] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:07.661515 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3512/7011] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:08.997825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d846e7-6111-4ca3-88bc-b2d9d79ec48e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:08.998648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d846e7-6111-4ca3-88bc-b2d9d79ec48e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:08.999013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d846e7-6111-4ca3-88bc-b2d9d79ec48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:09.022046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d8d846e7-6111-4ca3-88bc-b2d9d79ec48e None None] MFA Rules not processed for user `7ff5b886208a44c0891c2d1ba65dc083`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:09.023354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d8d846e7-6111-4ca3-88bc-b2d9d79ec48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:09.094631 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3500/7012] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:09.225559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:09.228174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:09.247898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:09.382556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f4d94-4072-48ea-b2f9-331d8fbc537c service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:09.387513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:09.388260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:09.388583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:09.388947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] PATH_INFO: `/v3/users/18fa7550fc73419dbc0a56127289462d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:09.390409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18fa7550fc73419dbc0a56127289462d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:09.392398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:09.484610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18fa7550fc73419dbc0a56127289462d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:09.485398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18fa7550fc73419dbc0a56127289462d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:09.497762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18fa7550fc73419dbc0a56127289462d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:09.499000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18fa7550fc73419dbc0a56127289462d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:09.518305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18fa7550fc73419dbc0a56127289462d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:09.532187 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3513/7013] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:09 2022] DELETE /identity/v3/users/18fa7550fc73419dbc0a56127289462d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:09.538926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:09.539208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:09.539557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:09.566327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:09.567657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:09.616975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:09.674923 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3501/7014] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:09.688070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:09.692018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:09.781052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938264c4-6579-4543-bbde-c0f7c7817576 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:09.791482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d03da2-f33f-4750-9371-7a15d3ab6d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:09.792193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d03da2-f33f-4750-9371-7a15d3ab6d8d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:09.792469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d03da2-f33f-4750-9371-7a15d3ab6d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:09.792809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d03da2-f33f-4750-9371-7a15d3ab6d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:09.811089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d03da2-f33f-4750-9371-7a15d3ab6d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:09.812651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d03da2-f33f-4750-9371-7a15d3ab6d8d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:09.841786 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3514/7015] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:10.377656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:10.379936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:10.403877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ddb08-910a-442c-90a0-98d6e9b04df6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:10.418193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:10.420204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:10.420485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:10.421715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] PATH_INFO: `/v3/projects/05516767841641de8a3c48312f31fa3f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:10.425592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05516767841641de8a3c48312f31fa3f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:10.430360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:10.498357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05516767841641de8a3c48312f31fa3f', 'request_id': 'req-c06ee8c7-bf35-4769-88d7-31a948e79018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.508438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05516767841641de8a3c48312f31fa3f', 'request_id': 'req-c06ee8c7-bf35-4769-88d7-31a948e79018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.517088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05516767841641de8a3c48312f31fa3f', 'request_id': 'req-c06ee8c7-bf35-4769-88d7-31a948e79018'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.546638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] The token cache is being invalidate because project 05516767841641de8a3c48312f31fa3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:10.546802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.547837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.549690 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3502/7016] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:10 2022] DELETE /identity/v3/projects/05516767841641de8a3c48312f31fa3f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:10.555637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5d63f-4b0e-4245-9fd0-cc68b80ea5cf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:10.556165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5d63f-4b0e-4245-9fd0-cc68b80ea5cf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:10.556598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5d63f-4b0e-4245-9fd0-cc68b80ea5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:10.594276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:10.596391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:10.596790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-00f5d63f-4b0e-4245-9fd0-cc68b80ea5cf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:10.599724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-00f5d63f-4b0e-4245-9fd0-cc68b80ea5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:10.660633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:10.760514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ee8c7-bf35-4769-88d7-31a948e79018 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:10.760917 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3515/7017] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:10.770365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebbb7a-1ac4-4e5f-a8af-365366c10f33 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:10.771470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebbb7a-1ac4-4e5f-a8af-365366c10f33 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:10.772010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebbb7a-1ac4-4e5f-a8af-365366c10f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:10.780054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:10.780758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:10.781042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:10.782070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] PATH_INFO: `/v3/users/fc139e5ee8054df988ea2be82067172b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:10.783373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc139e5ee8054df988ea2be82067172b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:10.785810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:10.800666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-86ebbb7a-1ac4-4e5f-a8af-365366c10f33 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:10.802253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-86ebbb7a-1ac4-4e5f-a8af-365366c10f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:10.812950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3516/7018] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:10.816097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebbb7a-1ac4-4e5f-a8af-365366c10f33 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:10.829357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebbb7a-1ac4-4e5f-a8af-365366c10f33 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:10.835620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:10.836917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:10.837010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:10.837527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:10.838681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:10.839056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:10.840780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:10.867464 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3517/7019] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:10.872262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:10.885182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185dd4ca-010e-4eb2-9ee9-3b88dd4827fe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:10.892966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:10.893668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:10.893959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:10.894408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:10.895032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:10.897350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:10.903471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc139e5ee8054df988ea2be82067172b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.911433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:10.925417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc139e5ee8054df988ea2be82067172b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.938648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc139e5ee8054df988ea2be82067172b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.939701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc139e5ee8054df988ea2be82067172b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.940942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-994efc1d-567b-47cb-8dee-c2f68114ec7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc139e5ee8054df988ea2be82067172b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:10.947976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3518/7020] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:10 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:10.953590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:10.956255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:10.979720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3503/7021] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:10 2022] DELETE /identity/v3/users/fc139e5ee8054df988ea2be82067172b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:10.985638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb049132-eced-43f7-9272-169932431f90 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:10.985930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb049132-eced-43f7-9272-169932431f90 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:10.986210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb049132-eced-43f7-9272-169932431f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.028688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cb049132-eced-43f7-9272-169932431f90 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:11.031189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cb049132-eced-43f7-9272-169932431f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:11.084523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303511-b7f1-43b4-a450-4a7f0d03e666 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:11.170347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:11.171550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.172080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.172452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.172821 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3504/7022] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:11.174218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:11.177213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:11.183782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb049132-eced-43f7-9272-169932431f90 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:11.185238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cb049132-eced-43f7-9272-169932431f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:11.224297 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3519/7023] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:10 2022] POST /identity/v3/users => generated 284 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:11.225624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:11.242611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90862b0a-777d-4839-a416-09da486bacd0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:11.249892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:11.250938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.251398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.251933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.252778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:11.254716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:11.267930 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3520/7024] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:11.270295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb049132-eced-43f7-9272-169932431f90 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:11.271940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:11.280313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:11.281034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.281520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.281791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.295515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3265d8-2929-43d1-8abd-2efe853a8c47 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:11.300482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:11.302733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:11.313566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797578ff-bc5b-4ee4-8642-489349f3628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:11.314804 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3505/7025] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:11.315050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797578ff-bc5b-4ee4-8642-489349f3628e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.315526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797578ff-bc5b-4ee4-8642-489349f3628e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.315803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797578ff-bc5b-4ee4-8642-489349f3628e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.316513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797578ff-bc5b-4ee4-8642-489349f3628e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:11.318174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797578ff-bc5b-4ee4-8642-489349f3628e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:11.336254 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3521/7026] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:11.340299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:11.359149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e61f4-b4ac-47e8-a67f-4fd0b4042f14 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:11.367050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:11.367050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.367393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.367609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] PATH_INFO: `/v3/projects/8f3662c7980740c9a1095c77901be860/users/e1994544002e4642a84e430aea8e92e0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.405563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3662c7980740c9a1095c77901be860, user_id=e1994544002e4642a84e430aea8e92e0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:11.409040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:11.441567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3506/7027] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:11 2022] PUT /identity/v3/projects/8f3662c7980740c9a1095c77901be860/users/e1994544002e4642a84e430aea8e92e0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:11.456704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b384-ec82-4749-9c64-41f4ef99aec6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.457270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b384-ec82-4749-9c64-41f4ef99aec6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.458170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b384-ec82-4749-9c64-41f4ef99aec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.466520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:11.512415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f881b384-ec82-4749-9c64-41f4ef99aec6 None None] MFA Rules not processed for user `e1994544002e4642a84e430aea8e92e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:11.516953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f881b384-ec82-4749-9c64-41f4ef99aec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:11.672622 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3522/7028] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:11.677715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:11.765262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:11.804605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62712d-b0aa-4aa4-8356-031b809ce666 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:11.814743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:11.816160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.816653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.817145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.818450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:11.820922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:11.870373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881b384-ec82-4749-9c64-41f4ef99aec6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:11.873181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3507/7029] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:11 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:11.887989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881b384-ec82-4749-9c64-41f4ef99aec6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:11.892541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:11.896693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:11.963417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:11.964031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:11.964306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:11.964575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:11.965246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:11.967470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:11.995834 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3523/7030] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:11 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:12.000059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.006622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8edd1-2495-4dbd-a3de-4d089dec5b54 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.012196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.012904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.013137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.013429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] PATH_INFO: `/v3/projects/0dae56f733674335a6e579ed7da013e2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.014031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b4507-cc99-4c40-bf1a-75afc88a566e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.014699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dae56f733674335a6e579ed7da013e2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.016630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.020553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.021420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.022518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.022724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.023448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.025071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.040272 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3524/7031] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:12.041233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.052453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b920c-4ea1-470a-affb-58d51d7a5765 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.057663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.058591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.059138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.059634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] PATH_INFO: `/v3/projects/594e226345d149fc8265b0658218fc9c/users/5b5bd95a2abe4454ae4554525df6ed20/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.078985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594e226345d149fc8265b0658218fc9c, user_id=5b5bd95a2abe4454ae4554525df6ed20, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.081919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.094601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dae56f733674335a6e579ed7da013e2', 'request_id': 'req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.104045 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3525/7032] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:12 2022] PUT /identity/v3/projects/594e226345d149fc8265b0658218fc9c/users/5b5bd95a2abe4454ae4554525df6ed20/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:12.105202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.105778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dae56f733674335a6e579ed7da013e2', 'request_id': 'req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.116875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dae56f733674335a6e579ed7da013e2', 'request_id': 'req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.139479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] The token cache is being invalidate because project 0dae56f733674335a6e579ed7da013e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:12.140139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.141429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.143475 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3508/7033] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:11 2022] DELETE /identity/v3/projects/0dae56f733674335a6e579ed7da013e2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:12.147471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.149493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:12.165953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:12.191453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1051e-5f49-4758-a5ec-eb8747754683 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.198422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58329727-ec26-40e7-a248-759177489056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.199791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58329727-ec26-40e7-a248-759177489056 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.201816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58329727-ec26-40e7-a248-759177489056 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.201816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58329727-ec26-40e7-a248-759177489056 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.202803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58329727-ec26-40e7-a248-759177489056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.205652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58329727-ec26-40e7-a248-759177489056 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.222775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3526/7034] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:12.227174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58329727-ec26-40e7-a248-759177489056 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.229027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-58329727-ec26-40e7-a248-759177489056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:12.240429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cdc20-6ad2-4483-b0d0-34a5f4d523d6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.245055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58329727-ec26-40e7-a248-759177489056 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.246470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.247511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.247882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.248256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] PATH_INFO: `/v3/users/a537532b84c64753b6420c978c95e041` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.249973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a537532b84c64753b6420c978c95e041)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.253041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461a34e-1eb0-4703-96cb-f2e2e1b7aad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.253999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461a34e-1eb0-4703-96cb-f2e2e1b7aad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.254420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461a34e-1eb0-4703-96cb-f2e2e1b7aad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.254841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461a34e-1eb0-4703-96cb-f2e2e1b7aad6 admin admin] PATH_INFO: `/v3/projects/594e226345d149fc8265b0658218fc9c/users/5b5bd95a2abe4454ae4554525df6ed20/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.257065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461a34e-1eb0-4703-96cb-f2e2e1b7aad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594e226345d149fc8265b0658218fc9c, user_id=5b5bd95a2abe4454ae4554525df6ed20, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.260737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461a34e-1eb0-4703-96cb-f2e2e1b7aad6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.265711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.272435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:12.283965 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3527/7035] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:12 2022] PUT /identity/v3/projects/594e226345d149fc8265b0658218fc9c/users/5b5bd95a2abe4454ae4554525df6ed20/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:12.295851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105263-d373-4ec3-bc41-055619d7b5ea None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.296129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105263-d373-4ec3-bc41-055619d7b5ea None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.296455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105263-d373-4ec3-bc41-055619d7b5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.353793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ed105263-d373-4ec3-bc41-055619d7b5ea None None] MFA Rules not processed for user `5b5bd95a2abe4454ae4554525df6ed20`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:12.355696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ed105263-d373-4ec3-bc41-055619d7b5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:12.360918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a537532b84c64753b6420c978c95e041'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.384346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a537532b84c64753b6420c978c95e041'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.398733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a537532b84c64753b6420c978c95e041'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.399475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a537532b84c64753b6420c978c95e041'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.400511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a537532b84c64753b6420c978c95e041'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:12.426056 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3509/7036] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:12 2022] DELETE /identity/v3/users/a537532b84c64753b6420c978c95e041 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:12.452613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3528/7037] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:12.457020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.534379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de23b41-947d-4b0c-8799-8928ecc742e0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.541769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.542639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.542859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.543212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.543978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.545703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.597745 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3510/7038] 10.222.0.26 () {60 vars in 1150 bytes} [Sat Sep 17 21:19:12 2022] POST /identity/v3/domains => generated 275 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:12.602315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105263-d373-4ec3-bc41-055619d7b5ea None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.614036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105263-d373-4ec3-bc41-055619d7b5ea None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.708790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.709448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.710108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.710392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.711129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.712829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.736379 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3529/7039] 10.222.0.26 () {60 vars in 1150 bytes} [Sat Sep 17 21:19:12 2022] POST /identity/v3/domains => generated 275 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:12.740523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.752886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f5c64-6c62-4864-8717-20ed6de4a572 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.835303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.835943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.836321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.836563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.837984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.839781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:12.871318 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3511/7040] 10.222.0.26 () {60 vars in 1150 bytes} [Sat Sep 17 21:19:12 2022] POST /identity/v3/domains => generated 278 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:12.875543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:12.888799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b683ba-e84b-4b44-992a-752b425b7fdb tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:12.970852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:12.989146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:12.989904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:12.990207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:12.990760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:12.991643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:12.994317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:13.036398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3530/7041] 10.222.0.26 () {60 vars in 1150 bytes} [Sat Sep 17 21:19:12 2022] POST /identity/v3/domains => generated 278 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:13.038000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:13.052071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adfd57-de1b-40c4-a3f6-0aebff0be2d0 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:13.070912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:13.163821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:13.164819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:13.165328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:13.165710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/167e9ca071094169a1a694a3284fe644` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:13.181301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=167e9ca071094169a1a694a3284fe644)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:13.183809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:13.216699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '167e9ca071094169a1a694a3284fe644'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.228189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '167e9ca071094169a1a694a3284fe644'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.285195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '167e9ca071094169a1a694a3284fe644', 'request_id': 'req-84c6fb18-b19b-4774-b769-64737520203c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.296869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '167e9ca071094169a1a694a3284fe644', 'request_id': 'req-84c6fb18-b19b-4774-b769-64737520203c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.305493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '167e9ca071094169a1a694a3284fe644', 'request_id': 'req-84c6fb18-b19b-4774-b769-64737520203c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.323512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] The token cache is being invalidate because project 167e9ca071094169a1a694a3284fe644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:13.324723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.326796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.337562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '167e9ca071094169a1a694a3284fe644', 'request_id': 'req-84c6fb18-b19b-4774-b769-64737520203c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.342363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '167e9ca071094169a1a694a3284fe644', 'request_id': 'req-84c6fb18-b19b-4774-b769-64737520203c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.343174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '167e9ca071094169a1a694a3284fe644', 'request_id': 'req-84c6fb18-b19b-4774-b769-64737520203c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.360452 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3512/7042] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:13 2022] DELETE /identity/v3/domains/167e9ca071094169a1a694a3284fe644 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:13.366874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:13.370090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:13.388684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a041b-41c7-4b15-8054-1583bd300b52 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:13.400158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:13.401629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:13.497647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:13.519447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:13.520110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:13.520426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:13.520696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:13.521867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:13.525129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:13.558239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6fb18-b19b-4774-b769-64737520203c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:13.563732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:13.564936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:13.565224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:13.565531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] PATH_INFO: `/v3/projects/b4878231cfd54dccb0673184d0130553` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:13.568248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4878231cfd54dccb0673184d0130553)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:13.571760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:13.572391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3531/7043] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:19:13 2022] POST /identity/v3/domains => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:13.577828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:13.672113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4878231cfd54dccb0673184d0130553', 'request_id': 'req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.682728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4878231cfd54dccb0673184d0130553', 'request_id': 'req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.704337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4878231cfd54dccb0673184d0130553', 'request_id': 'req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.724353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] The token cache is being invalidate because project b4878231cfd54dccb0673184d0130553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:13.724710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.725441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8373bc5a-4aa2-4bb3-83e7-f208f8bf9bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.727462 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3513/7044] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:13 2022] DELETE /identity/v3/projects/b4878231cfd54dccb0673184d0130553 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:13.728749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a96b5-0581-4495-916f-488a67165382 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:13.729120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a96b5-0581-4495-916f-488a67165382 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:13.729449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a96b5-0581-4495-916f-488a67165382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:13.741239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c099c-ff55-4026-beae-3b61950d1e90 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:13.757953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-331a96b5-0581-4495-916f-488a67165382 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:13.759061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-331a96b5-0581-4495-916f-488a67165382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:13.844115 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3514/7045] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:13.845667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:13.845948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:13.846383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:13.871525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:13.873287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:13.880665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:13.893425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:13.894333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:13.894522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/749caaed2238452a8efba41687dc03f8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:13.922366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=749caaed2238452a8efba41687dc03f8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:13.924194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:13.960660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:13.963449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '749caaed2238452a8efba41687dc03f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:13.967121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '749caaed2238452a8efba41687dc03f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.002259 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3515/7046] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:14.003381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.004886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.028764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '749caaed2238452a8efba41687dc03f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.030461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dcdaee0d-dfa4-420b-9b58-ec62e5497316 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '749caaed2238452a8efba41687dc03f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.035805 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3532/7047] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:13 2022] PATCH /identity/v3/domains/749caaed2238452a8efba41687dc03f8 => generated 249 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:14.041346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.041555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:14.041910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.042199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.082424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:14.083899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.103047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faa210-b0aa-44cb-b24e-0ef5e9a14938 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.177519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.178541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.179119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.179736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.182625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.197807 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3533/7048] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:14.202692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.204051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.255291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.257128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.266227 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3516/7049] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:19:14.267148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.277881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd6315-303d-40c0-a651-5bfdfbc64743 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.282763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.283505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.283709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.284035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.284694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:14.284996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.286732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.290237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba50cd-d667-4edf-b3e7-738f7372edd9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.295202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.296067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.296512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.296923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/749caaed2238452a8efba41687dc03f8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.303947 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3517/7050] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:14.308383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.315916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=749caaed2238452a8efba41687dc03f8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.322366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a7faf-0794-400f-bd8e-285601471d2f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.326563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.327428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.328371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.328703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.328980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.330571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.332643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.364268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '749caaed2238452a8efba41687dc03f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.367949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '749caaed2238452a8efba41687dc03f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.386079 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3518/7051] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:14 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:14.389863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.462311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749caaed2238452a8efba41687dc03f8', 'request_id': 'req-3aa0bcfb-37a9-4791-b76d-024a8200e257'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.463652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:14.478418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749caaed2238452a8efba41687dc03f8', 'request_id': 'req-3aa0bcfb-37a9-4791-b76d-024a8200e257'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.479091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:14.497310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749caaed2238452a8efba41687dc03f8', 'request_id': 'req-3aa0bcfb-37a9-4791-b76d-024a8200e257'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.513850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] The token cache is being invalidate because project 749caaed2238452a8efba41687dc03f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:14.514160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.514811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.522477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '749caaed2238452a8efba41687dc03f8', 'request_id': 'req-3aa0bcfb-37a9-4791-b76d-024a8200e257'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.523174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2630-7374-40c5-9446-7d3e5dc97a2c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.525798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '749caaed2238452a8efba41687dc03f8', 'request_id': 'req-3aa0bcfb-37a9-4791-b76d-024a8200e257'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.526627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '749caaed2238452a8efba41687dc03f8', 'request_id': 'req-3aa0bcfb-37a9-4791-b76d-024a8200e257'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.532771 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3534/7052] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:14 2022] DELETE /identity/v3/domains/749caaed2238452a8efba41687dc03f8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:14.549805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.553319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.612147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.613225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.613918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.614101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.614746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.617623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.642954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3519/7053] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:14 2022] POST /identity/v3/users => generated 284 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:14.643420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0bcfb-37a9-4791-b76d-024a8200e257 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.649953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.650573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.650869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.651132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.651797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.652400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.653402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.654086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.669192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e14a3-07b1-4f60-9cf3-c1f0d3bcce63 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.674231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.677932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.678239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.678585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.679185 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3535/7054] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:19:14 2022] POST /identity/v3/domains => generated 279 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:14.679327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.680931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.683446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.694058 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3520/7055] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:14.705940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.726782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a14e7-8c43-4e41-89e3-6bf720303da6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.769658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432b82-2486-436c-9232-34c3ea6f111e tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.774893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.775655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.775999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.776333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/6d3c70a79ec64838b36741c241a817bf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.787801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6d3c70a79ec64838b36741c241a817bf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.790198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.796416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.797017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.797341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.797641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.798275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.799999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.813802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3521/7056] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:14.814868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.816164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6d3c70a79ec64838b36741c241a817bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.816868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6d3c70a79ec64838b36741c241a817bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.824669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bab10-acfe-4262-9d72-26ee1e7f48f2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.843893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6d3c70a79ec64838b36741c241a817bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.844512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6d3c70a79ec64838b36741c241a817bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:14.847194 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3536/7057] 10.222.0.26 () {60 vars in 1250 bytes} [Sat Sep 17 21:19:14 2022] PATCH /identity/v3/domains/6d3c70a79ec64838b36741c241a817bf => generated 277 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:14.851278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:14.852576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.863583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=7,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:14.879431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:14.887752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ad9c9-5ed6-41c9-8861-f0f93afeb61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.888441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ad9c9-5ed6-41c9-8861-f0f93afeb61c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.888715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ad9c9-5ed6-41c9-8861-f0f93afeb61c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.889007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ad9c9-5ed6-41c9-8861-f0f93afeb61c admin admin] PATH_INFO: `/v3/projects/d4a92ab682ff4f62bf589544eee3e0f4/users/608aaeabfdb24796832640f803c01abc/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.905731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ad9c9-5ed6-41c9-8861-f0f93afeb61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a92ab682ff4f62bf589544eee3e0f4, user_id=608aaeabfdb24796832640f803c01abc, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:14.908465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ad9c9-5ed6-41c9-8861-f0f93afeb61c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:14.921430 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3522/7058] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:14 2022] PUT /identity/v3/projects/d4a92ab682ff4f62bf589544eee3e0f4/users/608aaeabfdb24796832640f803c01abc/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:14.931528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29375a1-7b8a-4118-93b4-0867b4e90ab4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:14.934270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.934270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.934270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:14.966286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] MFA Rules not processed for user `608aaeabfdb24796832640f803c01abc`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:14.966713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:14.988267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:14.988914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:14.989200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:14.989560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/6d3c70a79ec64838b36741c241a817bf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.001354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=6d3c70a79ec64838b36741c241a817bf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.003523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.005200 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3537/7059] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:19:14 2022] GET /identity/v3/domains/6d3c70a79ec64838b36741c241a817bf => generated 277 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:15.009225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.022269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87333063-c5f4-4efa-8d41-c8be62df11bc tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.027861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.028145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.028435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.028711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/6d3c70a79ec64838b36741c241a817bf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.029927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6d3c70a79ec64838b36741c241a817bf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.031601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.036496 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3523/7060] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:15.041959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.044113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:15.067226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6d3c70a79ec64838b36741c241a817bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.071559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6d3c70a79ec64838b36741c241a817bf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.132603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3c70a79ec64838b36741c241a817bf', 'request_id': 'req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.135615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067ff3a-0be3-48db-8ab3-eaeefb66de0c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.142337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3c70a79ec64838b36741c241a817bf', 'request_id': 'req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.154976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3c70a79ec64838b36741c241a817bf', 'request_id': 'req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.174936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] The token cache is being invalidate because project 6d3c70a79ec64838b36741c241a817bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:15.175578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.177390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.185412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:15.189420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6d3c70a79ec64838b36741c241a817bf', 'request_id': 'req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.195283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6d3c70a79ec64838b36741c241a817bf', 'request_id': 'req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.196184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6d3c70a79ec64838b36741c241a817bf', 'request_id': 'req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:15.207590 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3538/7061] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:15 2022] DELETE /identity/v3/domains/6d3c70a79ec64838b36741c241a817bf => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:15.208404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.210042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:15.211704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.212328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.212658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.212934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.213629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.215572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.240536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:15.251934 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3524/7062] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:15 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:15.256708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.260763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:15.376908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b09cb-54e8-4024-9315-d685b1eee1aa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.390328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c321bf-3f1d-41bb-a97e-3b6b41b9e582 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.391213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.391861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.392241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.392612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.393259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.399625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.475263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3525/7063] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:15 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:15.481400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.496215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb766b-ebbf-43da-8226-731c39ee2f09 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.502906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.503545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.503827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.504103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.504939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.507449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.514324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.515071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.515428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.515755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.516470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.519050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.536603 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3526/7064] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:15.537804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.548439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947d81b-53b2-4608-9b5e-ca256f8d6532 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.553528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.554322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.554740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.555019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] PATH_INFO: `/v3/projects/8f73dea8dbac459ca86e0258382f7630/users/88bffac8cf38485c8fa7cc203ff80bf0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.570987 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3539/7065] 10.222.0.26 () {58 vars in 1128 bytes} [Sat Sep 17 21:19:15 2022] GET /identity/v3/domains => generated 1346 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:15.570987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.575978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f73dea8dbac459ca86e0258382f7630, user_id=88bffac8cf38485c8fa7cc203ff80bf0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.578831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.583754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b0108-e847-410a-974d-4a0fefe6fbae tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.589876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.590659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.590945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.591297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/6d3c70a79ec64838b36741c241a817bf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.598105 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3527/7066] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:15 2022] PUT /identity/v3/projects/8f73dea8dbac459ca86e0258382f7630/users/88bffac8cf38485c8fa7cc203ff80bf0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:15.602179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.608516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find domain: 6d3c70a79ec64838b36741c241a817bf.: keystone.exception.DomainNotFound: Could not find domain: 6d3c70a79ec64838b36741c241a817bf. Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6d3c70a79ec64838b36741c241a817bf. Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:15.613919 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:15.614959 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 6d3c70a79ec64838b36741c241a817bf. Sep 17 21:19:15.616046 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:15.617113 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3540/7067] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:15 2022] PATCH /identity/v3/domains/6d3c70a79ec64838b36741c241a817bf => generated 112 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:15.620844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.633888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260ac21-9e03-4d0c-b477-b0562e32576b tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.707284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388baec-3313-41bc-b94e-fbae5bb14ea2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.717537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.718176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.718450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.718722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.719948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.721681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.734260 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3528/7068] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:15.738725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.742047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.742768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.743118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.743465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.744568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.746334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.756803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34e9aa-a31e-41e7-9b24-9d410ede9d89 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.761895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61fbd9-1b78-42b9-bfa0-2e1675f4584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:15.762878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61fbd9-1b78-42b9-bfa0-2e1675f4584b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.763216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61fbd9-1b78-42b9-bfa0-2e1675f4584b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.763860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61fbd9-1b78-42b9-bfa0-2e1675f4584b admin admin] PATH_INFO: `/v3/projects/8f73dea8dbac459ca86e0258382f7630/users/88bffac8cf38485c8fa7cc203ff80bf0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.765852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61fbd9-1b78-42b9-bfa0-2e1675f4584b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f73dea8dbac459ca86e0258382f7630, user_id=88bffac8cf38485c8fa7cc203ff80bf0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:15.768930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61fbd9-1b78-42b9-bfa0-2e1675f4584b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:15.776859 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3541/7069] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:19:15 2022] POST /identity/v3/domains => generated 276 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:15.779442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:15.794404 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3529/7070] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:15 2022] PUT /identity/v3/projects/8f73dea8dbac459ca86e0258382f7630/users/88bffac8cf38485c8fa7cc203ff80bf0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:15.795976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:15.796262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:15.796536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:15.815214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:15.830120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] MFA Rules not processed for user `88bffac8cf38485c8fa7cc203ff80bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:15.831519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:15.900977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff8ff6-fdbe-48be-89ae-f810e62d8cbf tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:15.908868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:16.000459 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3530/7071] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:16.001886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:16.002592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:16.002940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:16.003279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:16.004209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:16.006776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:16.011171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:16.012367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:16.097153 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3542/7072] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:15 2022] POST /identity/v3/users => generated 348 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:16.101995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:16.129407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82182c6-cfeb-4563-bf95-3758322e8541 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:16.140696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:16.141360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:16.141785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:16.142072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/groups` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:16.142717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:16.145047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:16.173789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65876133-3a7d-4ad7-ab7a-fec123b2f250 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:16.184779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:16.186344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:16.186683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:16.187072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:16.221349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:16.223025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:16.223470 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3543/7073] 10.222.0.26 () {60 vars in 1147 bytes} [Sat Sep 17 21:19:16 2022] POST /identity/v3/groups => generated 322 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:16.228095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:16.241885 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3531/7074] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:16.257169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fa18d-61d2-4e1c-8f38-671f4d2e99f8 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:16.277003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:16.277694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:16.278100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:16.278631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:16.279897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=53fdb3de6f9b48c6b51c862627e71e3c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:16.281740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:16.330239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.341918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.374890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:16.391655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.392374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.395583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b69f0e-d5b7-4496-8d4b-3fdf0f0720dd service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:16.402752 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3544/7075] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:16 2022] PATCH /identity/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c => generated 277 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:16.406610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:16.408541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:16.505630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:16.534491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:16.536313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:16.536598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:16.536870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:16.542040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:16.546103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2616f7-2ba0-433c-b355-c00765583127 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:16.557943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:16.558609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:16.558890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:16.559166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:16.587309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=53fdb3de6f9b48c6b51c862627e71e3c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:16.588967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:16.636313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.642287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.699730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:16.701242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:16.724262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3532/7076] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:16.882454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1d65de6c2944a3b708f0acffd4491f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.883277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1d65de6c2944a3b708f0acffd4491f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.897194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1d65de6c2944a3b708f0acffd4491f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.898159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1d65de6c2944a3b708f0acffd4491f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:16.916699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1d65de6c2944a3b708f0acffd4491f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.016183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:17.028469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c', 'request_id': 'req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.037178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c', 'request_id': 'req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.048940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c', 'request_id': 'req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.084127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] The token cache is being invalidate because project 53fdb3de6f9b48c6b51c862627e71e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:17.085164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.088909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.108934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c', 'request_id': 'req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.115042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c', 'request_id': 'req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.118453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '53fdb3de6f9b48c6b51c862627e71e3c', 'request_id': 'req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:17.134060 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3545/7077] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:16 2022] DELETE /identity/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:17.135077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:17.149314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eeda83-6a2f-4387-ab98-187a3d100a51 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:17.213712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:17.393736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:17.393736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:17.393736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:17.394191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:17.414505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=53fdb3de6f9b48c6b51c862627e71e3c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:17.417177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c.: keystone.exception.DomainNotFound: Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:17.460819 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:17.461988 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:17.463098 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:19:17.477983 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3533/7078] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:19:17 2022] GET /identity/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c => generated 112 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:19:17.482477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:17.484007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:17.508399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c06eb-e267-4bbe-9ee6-99f6a5b1cfe5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:17.518462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:17.519084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:17.519367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:17.519644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/users/ab1d65de6c2944a3b708f0acffd4491f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:17.534691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:get_user(user_id=ab1d65de6c2944a3b708f0acffd4491f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:17.538568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find user: ab1d65de6c2944a3b708f0acffd4491f.: keystone.exception.UserNotFound: Could not find user: ab1d65de6c2944a3b708f0acffd4491f. Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 21:19:17.573218 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:17.574526 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: ab1d65de6c2944a3b708f0acffd4491f. Sep 17 21:19:17.575975 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:17.586159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:17.595799 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3546/7079] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:19:17 2022] GET /identity/v3/users/ab1d65de6c2944a3b708f0acffd4491f => generated 110 bytes in 102 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:17.616531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c45944-2b90-4658-a0bd-4e7eb8b0cfd5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:17.636714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:17.637579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:17.638416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:17.638416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/groups/3882562c098b4c2c8f35096e36d32bb5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:17.655981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:get_group(group_id=3882562c098b4c2c8f35096e36d32bb5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:17.658518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find group: 3882562c098b4c2c8f35096e36d32bb5.: keystone.exception.GroupNotFound: Could not find group: 3882562c098b4c2c8f35096e36d32bb5. Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_group(group_id) Sep 17 21:19:17.726586 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:17.727821 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 3882562c098b4c2c8f35096e36d32bb5. Sep 17 21:19:17.729053 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:19:17.732733 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3534/7080] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:17 2022] GET /identity/v3/groups/3882562c098b4c2c8f35096e36d32bb5 => generated 111 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:17.745916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:17.776183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319f4bc-25d8-4bf4-8eda-0b97f151d36f tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:17.829041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:17.829984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:17.830096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:17.830311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/groups/3882562c098b4c2c8f35096e36d32bb5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:17.839454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find group: 3882562c098b4c2c8f35096e36d32bb5.: keystone.exception.GroupNotFound: Could not find group: 3882562c098b4c2c8f35096e36d32bb5. Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Sep 17 21:19:17.847375 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:17.848496 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 3882562c098b4c2c8f35096e36d32bb5. Sep 17 21:19:17.849753 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:17.855250 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3547/7081] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:19:17 2022] DELETE /identity/v3/groups/3882562c098b4c2c8f35096e36d32bb5 => generated 111 bytes in 110 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:17.865608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:17.890898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b7e1e-878f-4fc6-b4a5-fa1c77a8a2a5 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:17.903884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:17.904642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:17.905005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:17.905785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/users/ab1d65de6c2944a3b708f0acffd4491f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:17.942270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ab1d65de6c2944a3b708f0acffd4491f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:17.944980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find user: ab1d65de6c2944a3b708f0acffd4491f.: keystone.exception.UserNotFound: Could not find user: ab1d65de6c2944a3b708f0acffd4491f. Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Sep 17 21:19:17.983156 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:17.984604 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: ab1d65de6c2944a3b708f0acffd4491f. Sep 17 21:19:17.986124 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:19:17.987351 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3535/7082] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:17 2022] DELETE /identity/v3/users/ab1d65de6c2944a3b708f0acffd4491f => generated 110 bytes in 129 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:17.989555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:18.002550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8aadbb-a042-4307-b570-527292a2becd tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:18.025660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:18.026832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:18.027530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:18.028266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:18.101722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c.: keystone.exception.DomainNotFound: Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:18.110517 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:18.111877 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:18.113084 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:18.115167 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3548/7083] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:17 2022] PATCH /identity/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c => generated 112 bytes in 126 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:18.121908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:18.136130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8aeb2a-cbcf-4adc-b5f9-10ab6af3a4ba tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:18.145905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:18.150619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:18.151017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:18.151382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:18.152490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:18.155506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:18.233380 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3536/7084] 10.222.0.26 () {58 vars in 1128 bytes} [Sat Sep 17 21:19:18 2022] GET /identity/v3/domains => generated 1346 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:18.237615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:18.255109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0b98-dfe3-4274-b5fb-ece2099d837c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:18.268721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:18.269369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:18.269671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:18.269948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:18.270616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:18.270919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:18.282269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:18.309038 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3549/7085] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:18 2022] GET /identity/v3/domains?enabled=True => generated 1092 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:18.314415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:18.332163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bab42-fef4-458e-8a16-3c0b3a4ccfd9 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:18.346889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:18.347947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:18.350082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:18.350656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:18.352113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-780826304) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:18.354310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:18.357901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:18.374950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3537/7086] 10.222.0.26 () {58 vars in 1197 bytes} [Sat Sep 17 21:19:18 2022] GET /identity/v3/domains?name=tempest-test_domain-780826304 => generated 406 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:18.379665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:18.393999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672bb5-97c7-443d-99ae-935e2ee84457 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:18.401023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:18.402195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:18.402676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:18.403035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:18.414986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c.: keystone.exception.DomainNotFound: Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:18.421481 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:18.422774 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 53fdb3de6f9b48c6b51c862627e71e3c. Sep 17 21:19:18.423861 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:18.425099 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3550/7087] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:18 2022] PATCH /identity/v3/domains/53fdb3de6f9b48c6b51c862627e71e3c => generated 112 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:18.427496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:18.446153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090c05-a3ce-453d-8414-bc7d277f5ee1 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:18.460629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:18.463314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:18.464568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:18.464955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/644b7532bbf34a1cb20f69953c0d5c3e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:18.501124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=644b7532bbf34a1cb20f69953c0d5c3e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:18.505242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:18.634330 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3538/7088] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:18 2022] PATCH /identity/v3/domains/644b7532bbf34a1cb20f69953c0d5c3e => generated 278 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:18.637843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:18.652107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5948c3a-d960-4d62-b00c-32c2f5cedde4 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:18.658754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:18.660010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:18.660761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:18.661231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/644b7532bbf34a1cb20f69953c0d5c3e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:18.689366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=644b7532bbf34a1cb20f69953c0d5c3e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:18.692153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:18.796501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:18.807355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:18.946996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e', 'request_id': 'req-805a6436-d5b3-4fde-a25f-04ee7283331c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:18.956712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e', 'request_id': 'req-805a6436-d5b3-4fde-a25f-04ee7283331c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:18.972680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e', 'request_id': 'req-805a6436-d5b3-4fde-a25f-04ee7283331c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:18.997287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] The token cache is being invalidate because project 644b7532bbf34a1cb20f69953c0d5c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:19.000381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.001360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.007626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:19.021896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e', 'request_id': 'req-805a6436-d5b3-4fde-a25f-04ee7283331c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.026157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e', 'request_id': 'req-805a6436-d5b3-4fde-a25f-04ee7283331c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.028266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '644b7532bbf34a1cb20f69953c0d5c3e', 'request_id': 'req-805a6436-d5b3-4fde-a25f-04ee7283331c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.041178 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3551/7089] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:18 2022] DELETE /identity/v3/domains/644b7532bbf34a1cb20f69953c0d5c3e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:19.048879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:19.053712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:19.080487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f218ee9-92ca-48f7-8d05-71f00ae68827 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:19.194584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:19.195328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:19.195614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:19.195886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/362b7f678fc945e98fc44a543a807edd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:19.208369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=362b7f678fc945e98fc44a543a807edd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:19.210126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:19.220729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:19.236596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '362b7f678fc945e98fc44a543a807edd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.245642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '362b7f678fc945e98fc44a543a807edd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.295129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '362b7f678fc945e98fc44a543a807edd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.295948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '362b7f678fc945e98fc44a543a807edd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.300327 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3539/7090] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:19 2022] PATCH /identity/v3/domains/362b7f678fc945e98fc44a543a807edd => generated 276 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:19.308557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:19.324953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a6436-d5b3-4fde-a25f-04ee7283331c tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:19.422002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:19.422706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:19.423141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:19.423420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/362b7f678fc945e98fc44a543a807edd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:19.436175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=362b7f678fc945e98fc44a543a807edd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:19.439629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:19.496233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '362b7f678fc945e98fc44a543a807edd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.500419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '362b7f678fc945e98fc44a543a807edd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.604393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362b7f678fc945e98fc44a543a807edd', 'request_id': 'req-d4923f31-aef2-4cbf-97a7-95019763caa6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.612082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362b7f678fc945e98fc44a543a807edd', 'request_id': 'req-d4923f31-aef2-4cbf-97a7-95019763caa6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.623616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362b7f678fc945e98fc44a543a807edd', 'request_id': 'req-d4923f31-aef2-4cbf-97a7-95019763caa6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.637566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] The token cache is being invalidate because project 362b7f678fc945e98fc44a543a807edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:19.637870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.642204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.657871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '362b7f678fc945e98fc44a543a807edd', 'request_id': 'req-d4923f31-aef2-4cbf-97a7-95019763caa6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.681305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '362b7f678fc945e98fc44a543a807edd', 'request_id': 'req-d4923f31-aef2-4cbf-97a7-95019763caa6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.682418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '362b7f678fc945e98fc44a543a807edd', 'request_id': 'req-d4923f31-aef2-4cbf-97a7-95019763caa6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.698950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3552/7091] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:19 2022] DELETE /identity/v3/domains/362b7f678fc945e98fc44a543a807edd => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:19.703336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:19.705916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:19.725284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4087-c9c8-44a2-bcab-9be7d9398cc6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:19.790806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:19.791459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:19.791755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:19.792028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/f114c8c9eaae44349c5cba88333536f5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:19.804672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f114c8c9eaae44349c5cba88333536f5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:19.806298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:19.833687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.834734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.863298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.864109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:19.867023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3540/7092] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:19 2022] PATCH /identity/v3/domains/f114c8c9eaae44349c5cba88333536f5 => generated 276 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:19.871503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:19.873016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:19.887014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4923f31-aef2-4cbf-97a7-95019763caa6 tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:19.911022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:19.958413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b5bd95a2abe4454ae4554525df6ed20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '594e226345d149fc8265b0658218fc9c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:19.959067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:19.959389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:19.960410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] PATH_INFO: `/v3/domains/f114c8c9eaae44349c5cba88333536f5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:19.972941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f114c8c9eaae44349c5cba88333536f5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:19.975157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:20.001295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.004849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.070760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5', 'request_id': 'req-f67f7d8d-64af-486c-80d8-1287e62e87ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.084129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5', 'request_id': 'req-f67f7d8d-64af-486c-80d8-1287e62e87ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.095488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5', 'request_id': 'req-f67f7d8d-64af-486c-80d8-1287e62e87ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.118445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] The token cache is being invalidate because project f114c8c9eaae44349c5cba88333536f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:20.119511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.132915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.153708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5', 'request_id': 'req-f67f7d8d-64af-486c-80d8-1287e62e87ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.175347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5', 'request_id': 'req-f67f7d8d-64af-486c-80d8-1287e62e87ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.177334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f67f7d8d-64af-486c-80d8-1287e62e87ed tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f114c8c9eaae44349c5cba88333536f5', 'request_id': 'req-f67f7d8d-64af-486c-80d8-1287e62e87ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.185637 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3553/7093] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:19 2022] DELETE /identity/v3/domains/f114c8c9eaae44349c5cba88333536f5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:20.189431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:20.190978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:20.289056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71e7af-68c9-4464-8d5b-6e80be2b92df tempest-DomainsTestJSON-329640433 tempest-DomainsTestJSON-329640433-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:20.360416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:20.429152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:20.430000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:20.430373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:20.430743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] PATH_INFO: `/v3/users/e1994544002e4642a84e430aea8e92e0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:20.432004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1994544002e4642a84e430aea8e92e0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:20.435035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:20.536710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1994544002e4642a84e430aea8e92e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.577036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1994544002e4642a84e430aea8e92e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.615090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1994544002e4642a84e430aea8e92e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.616062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1994544002e4642a84e430aea8e92e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.616995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1994544002e4642a84e430aea8e92e0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:20.639776 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3541/7094] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:20 2022] DELETE /identity/v3/users/e1994544002e4642a84e430aea8e92e0 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:20.641764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:20.641764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:20.641938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:20.675842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:20.677110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:20.846188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:20.876678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3554/7095] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:20.887218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:20.888678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:20.999037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11de11e-a9fa-4954-b789-cbe11b1bc28e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:21.003943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:21.005060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:21.005400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:21.005769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:21.021242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:21.023531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:21.036778 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3542/7096] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:21.492182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:21.495624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:21.520909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48932aa8-b8b4-49af-98d1-fa65c1553150 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:21.542934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:21.543773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:21.544125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:21.544528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] PATH_INFO: `/v3/projects/8f3662c7980740c9a1095c77901be860` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:21.546112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f3662c7980740c9a1095c77901be860)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:21.550075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:21.620843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f3662c7980740c9a1095c77901be860', 'request_id': 'req-eb88ad6c-d21c-40a4-9504-97a20efa8e92'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:21.629506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f3662c7980740c9a1095c77901be860', 'request_id': 'req-eb88ad6c-d21c-40a4-9504-97a20efa8e92'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:21.642628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f3662c7980740c9a1095c77901be860', 'request_id': 'req-eb88ad6c-d21c-40a4-9504-97a20efa8e92'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:21.660436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] The token cache is being invalidate because project 8f3662c7980740c9a1095c77901be860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:21.661300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:21.662167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:21.664239 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3555/7097] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:21 2022] DELETE /identity/v3/projects/8f3662c7980740c9a1095c77901be860 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:21.668767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:21.687488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8e2e9-394e-443b-95a4-664e6af5136c service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:21.806540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:21.825359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:21.827128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:21.828285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:21.829585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] PATH_INFO: `/v3/users/5b5bd95a2abe4454ae4554525df6ed20` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:21.834191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5bd95a2abe4454ae4554525df6ed20)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:21.839572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:21.952327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5bd95a2abe4454ae4554525df6ed20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:21.971976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5bd95a2abe4454ae4554525df6ed20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.027088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5bd95a2abe4454ae4554525df6ed20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.027088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5bd95a2abe4454ae4554525df6ed20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.027088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-038ae4df-ebe2-48a1-bd93-5bceb4713a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5bd95a2abe4454ae4554525df6ed20'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.059312 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3543/7098] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:21 2022] DELETE /identity/v3/users/5b5bd95a2abe4454ae4554525df6ed20 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:22.512084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:22.514522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:22.539236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ad6c-d21c-40a4-9504-97a20efa8e92 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:22.642173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:22.643112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:22.643484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:22.643853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] PATH_INFO: `/v3/projects/594e226345d149fc8265b0658218fc9c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:22.645098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594e226345d149fc8265b0658218fc9c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:22.647676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:22.789041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594e226345d149fc8265b0658218fc9c', 'request_id': 'req-ed891993-ff68-4cc6-b6d5-18763df18c96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.797883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594e226345d149fc8265b0658218fc9c', 'request_id': 'req-ed891993-ff68-4cc6-b6d5-18763df18c96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.810207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594e226345d149fc8265b0658218fc9c', 'request_id': 'req-ed891993-ff68-4cc6-b6d5-18763df18c96'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.833693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] The token cache is being invalidate because project 594e226345d149fc8265b0658218fc9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:22.834051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.834792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed891993-ff68-4cc6-b6d5-18763df18c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:22.836394 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3556/7099] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:22 2022] DELETE /identity/v3/projects/594e226345d149fc8265b0658218fc9c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:22.842139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff4a1-151b-4ea0-bbf5-c737f1aa2763 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:22.842424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff4a1-151b-4ea0-bbf5-c737f1aa2763 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:22.842706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff4a1-151b-4ea0-bbf5-c737f1aa2763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:22.875370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9c9ff4a1-151b-4ea0-bbf5-c737f1aa2763 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:22.877046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9c9ff4a1-151b-4ea0-bbf5-c737f1aa2763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.037407 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3544/7100] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:23.039132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8571f-8faa-4d72-9f0f-89caa7f2ef6f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.039418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8571f-8faa-4d72-9f0f-89caa7f2ef6f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.039700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8571f-8faa-4d72-9f0f-89caa7f2ef6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.068076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-80a8571f-8faa-4d72-9f0f-89caa7f2ef6f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:23.072082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-80a8571f-8faa-4d72-9f0f-89caa7f2ef6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.091967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff4a1-151b-4ea0-bbf5-c737f1aa2763 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.093793 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3557/7101] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:23.119173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff4a1-151b-4ea0-bbf5-c737f1aa2763 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.131645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.132805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.133323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.133880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.134665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:23.135456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.137221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.163156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3545/7102] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:23.165117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8571f-8faa-4d72-9f0f-89caa7f2ef6f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.170589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.172830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.177362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:23.185258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8571f-8faa-4d72-9f0f-89caa7f2ef6f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.191566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.192468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.194216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.194743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.195532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.197626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.204087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f540d0-bcd8-4011-8c8e-b8707bc5359f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.212340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.213001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.213280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.213595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] PATH_INFO: `/v3/users/608aaeabfdb24796832640f803c01abc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.214801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=608aaeabfdb24796832640f803c01abc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.216761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.248557 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3558/7103] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:23 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:23.253550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.315769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608aaeabfdb24796832640f803c01abc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:23.329416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:23.343650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608aaeabfdb24796832640f803c01abc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:23.352223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608aaeabfdb24796832640f803c01abc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:23.353498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608aaeabfdb24796832640f803c01abc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:23.354353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1293ef21-661a-4b7f-8a43-50d16ccdb226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608aaeabfdb24796832640f803c01abc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:23.372220 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3546/7104] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:23 2022] DELETE /identity/v3/users/608aaeabfdb24796832640f803c01abc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:23.377389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.380834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.381138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.392488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e814-4243-4574-9065-6f7a57151e85 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.405237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:23.406526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.480033 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3547/7105] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:23.489773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.491243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.491676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.492311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.492610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.493727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.495167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.497751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.524751 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3559/7106] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:23 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:23.528987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.531115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.543972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a32e8b-5431-44e0-8034-31f7b6eb9f67 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.548644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.549254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.549562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.549839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.550524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.552123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.557910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947f18d-672d-4a22-8cf8-e15650563ec6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.561668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.562322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.562596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.562866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.563130 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3560/7107] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:23.566967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.573347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.574871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.577729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e04a8-40aa-4afe-a69a-aa01b3f58e86 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.585157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3548/7108] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:23.585416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143caa17-5a95-4b4a-9670-178cc460a2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.585883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa17-5a95-4b4a-9670-178cc460a2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.586161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa17-5a95-4b4a-9670-178cc460a2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.586439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa17-5a95-4b4a-9670-178cc460a2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.587076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143caa17-5a95-4b4a-9670-178cc460a2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.588748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143caa17-5a95-4b4a-9670-178cc460a2f5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.600172 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3561/7109] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:23.604421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.614650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006aaa1-b373-4b7d-a4e1-9a32f8f251b6 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.621624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.622344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.622590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.622862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] PATH_INFO: `/v3/projects/331ad21b034444c9a3bc2a029060c420/users/333e4238006349ed99312ac3a8ab4a96/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.789407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=331ad21b034444c9a3bc2a029060c420, user_id=333e4238006349ed99312ac3a8ab4a96, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.792254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:23.814925 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3549/7110] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:23 2022] PUT /identity/v3/projects/331ad21b034444c9a3bc2a029060c420/users/333e4238006349ed99312ac3a8ab4a96/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:23.818135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0cc62-87ba-491b-b659-45f155333a4e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.818422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0cc62-87ba-491b-b659-45f155333a4e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.818716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0cc62-87ba-491b-b659-45f155333a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.848964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dee0cc62-87ba-491b-b659-45f155333a4e None None] MFA Rules not processed for user `333e4238006349ed99312ac3a8ab4a96`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:23.850322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dee0cc62-87ba-491b-b659-45f155333a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.888799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.890585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:23.915260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:23.937576 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3562/7111] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:23.942031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0cc62-87ba-491b-b659-45f155333a4e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:23.948824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:23.963360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77e161-8d81-4f5a-bf10-d00f0c369f71 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:23.968277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:23.969018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:23.969363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:23.969684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] PATH_INFO: `/v3/projects/d4a92ab682ff4f62bf589544eee3e0f4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:23.970850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4a92ab682ff4f62bf589544eee3e0f4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:23.972808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.010948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0cc62-87ba-491b-b659-45f155333a4e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.021342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4a92ab682ff4f62bf589544eee3e0f4', 'request_id': 'req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.032443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4a92ab682ff4f62bf589544eee3e0f4', 'request_id': 'req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.040166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4a92ab682ff4f62bf589544eee3e0f4', 'request_id': 'req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.055408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] The token cache is being invalidate because project d4a92ab682ff4f62bf589544eee3e0f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:24.056253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.057091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.060069 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3550/7112] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:23 2022] DELETE /identity/v3/projects/d4a92ab682ff4f62bf589544eee3e0f4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:24.064218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.065952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:24.076265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.076869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.077143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.077425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.078085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.080028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.114086 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3563/7113] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:23 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:24.118169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.120547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:24.140176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8f2cf-bb7e-41fc-9b97-2933b864fc1d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.190845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edbb5b-8e2a-408d-ac3f-97a1a04a52c7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.196770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.197607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.197840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.198219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.198754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.201664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.203981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.204675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.205008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.205333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] PATH_INFO: `/v3/users/88bffac8cf38485c8fa7cc203ff80bf0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.206607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88bffac8cf38485c8fa7cc203ff80bf0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.208721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.226123 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3564/7114] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:24 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:24.230066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.240348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ce242-74fd-4c7c-8ece-a10bb7234ebd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.246272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.246965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.247265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.247617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.248366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.250319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.251802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:24.252667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:24.265169 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3565/7115] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:24.265861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88bffac8cf38485c8fa7cc203ff80bf0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.269099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.278358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88bffac8cf38485c8fa7cc203ff80bf0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.279096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2dd06-134a-4cec-90a1-54f2789ddc1d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.284164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.284838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.285242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.285434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] PATH_INFO: `/v3/projects/80e2df26ab06415fac47b46b8aa77a50/users/d355151d8d564245b489773924ea4372/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.286960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88bffac8cf38485c8fa7cc203ff80bf0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.287744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88bffac8cf38485c8fa7cc203ff80bf0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.288558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88bffac8cf38485c8fa7cc203ff80bf0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.301016 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3551/7116] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:24 2022] DELETE /identity/v3/users/88bffac8cf38485c8fa7cc203ff80bf0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:24.302138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e2df26ab06415fac47b46b8aa77a50, user_id=d355151d8d564245b489773924ea4372, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.304872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.320900 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3566/7117] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:24 2022] PUT /identity/v3/projects/80e2df26ab06415fac47b46b8aa77a50/users/d355151d8d564245b489773924ea4372/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:24.326769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.328308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:24.406345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc258-35a0-487f-9bae-c2366dce148b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.462177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba963d30-2214-43a7-9921-bc3935e18c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.462931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba963d30-2214-43a7-9921-bc3935e18c80 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.463276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba963d30-2214-43a7-9921-bc3935e18c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.463616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba963d30-2214-43a7-9921-bc3935e18c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.464266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba963d30-2214-43a7-9921-bc3935e18c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.465911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba963d30-2214-43a7-9921-bc3935e18c80 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.476895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3552/7118] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:24.481247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.492771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8dea-2227-4ca5-b3d3-ec553233d00b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.497999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.498662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.498937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.499213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] PATH_INFO: `/v3/projects/80e2df26ab06415fac47b46b8aa77a50/users/d355151d8d564245b489773924ea4372/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.501174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e2df26ab06415fac47b46b8aa77a50, user_id=d355151d8d564245b489773924ea4372, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.503889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.518979 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3567/7119] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:24 2022] PUT /identity/v3/projects/80e2df26ab06415fac47b46b8aa77a50/users/d355151d8d564245b489773924ea4372/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:24.524734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e67e-8702-4aab-9f96-e6df3a82c2d7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.525207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e67e-8702-4aab-9f96-e6df3a82c2d7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.525711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e67e-8702-4aab-9f96-e6df3a82c2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.558954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c1b0e67e-8702-4aab-9f96-e6df3a82c2d7 None None] MFA Rules not processed for user `d355151d8d564245b489773924ea4372`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:24.560067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1b0e67e-8702-4aab-9f96-e6df3a82c2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:24.616430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:24.629017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.630969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:24.673801 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3553/7120] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:24.673801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e67e-8702-4aab-9f96-e6df3a82c2d7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.683185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e67e-8702-4aab-9f96-e6df3a82c2d7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.693098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.693098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.693098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.693607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.693607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.693607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.712946 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3554/7121] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:24 2022] POST /identity/v3/services => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:24.717360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.746699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb261f-f0bc-4755-a2cf-8116d70c53db tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.753451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.754216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.754339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6443b-058c-47ea-b5e4-b7d457997675 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.754504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.754918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.755626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.756887 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:24.756887 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:24.756887 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:24.757878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.770902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.771651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.772051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.772344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] PATH_INFO: `/v3/projects/8f73dea8dbac459ca86e0258382f7630` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.773517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f73dea8dbac459ca86e0258382f7630)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.775463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.827890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f73dea8dbac459ca86e0258382f7630', 'request_id': 'req-7ce5768a-ce34-4fda-b09c-75db034c4b08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.831759 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3555/7122] 10.222.0.26 () {60 vars in 1213 bytes} [Sat Sep 17 21:19:24 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:24.833067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.835685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f73dea8dbac459ca86e0258382f7630', 'request_id': 'req-7ce5768a-ce34-4fda-b09c-75db034c4b08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.844866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf6143-f393-431e-ba18-8ffbcfba6759 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:24.846354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f73dea8dbac459ca86e0258382f7630', 'request_id': 'req-7ce5768a-ce34-4fda-b09c-75db034c4b08'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.860213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] The token cache is being invalidate because project 8f73dea8dbac459ca86e0258382f7630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:24.860542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.861173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7ce5768a-ce34-4fda-b09c-75db034c4b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:24.862949 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3568/7123] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:24 2022] DELETE /identity/v3/projects/8f73dea8dbac459ca86e0258382f7630 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:24.871364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664edeb-d726-45cc-a2f1-ca78587a85fd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.871665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664edeb-d726-45cc-a2f1-ca78587a85fd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.872073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664edeb-d726-45cc-a2f1-ca78587a85fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.896135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a664edeb-d726-45cc-a2f1-ca78587a85fd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:24.897781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a664edeb-d726-45cc-a2f1-ca78587a85fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:24.941488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:24.942869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:24.943949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:24.944879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:24.946327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:24.948734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:24.958808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:24.974792 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3556/7124] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:24 2022] POST /identity/v3/services => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:24.979212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:24.980962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:25.006062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cccce-383c-491f-a166-0eeb98e5b63e tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.007986 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3569/7125] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3504 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:25.015504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fb512-6394-4a9b-8996-59d03146c7fa None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.015792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fb512-6394-4a9b-8996-59d03146c7fa None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.016173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fb512-6394-4a9b-8996-59d03146c7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.017357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.018356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.018356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.018572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.019210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.020815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.031740 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3557/7126] 10.222.0.26 () {60 vars in 1213 bytes} [Sat Sep 17 21:19:24 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:25.035994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.043439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7b0fb512-6394-4a9b-8996-59d03146c7fa None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:25.044693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7b0fb512-6394-4a9b-8996-59d03146c7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:25.048139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f429-62f3-40c9-aefd-a74085bafa89 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.060978 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3570/7127] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3504 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:25.063203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.064091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.064384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.064720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.065361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.065838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fb512-6394-4a9b-8996-59d03146c7fa None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.066612 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:25.066612 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:25.066612 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:25.067392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.075625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fb512-6394-4a9b-8996-59d03146c7fa None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.078798 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3558/7128] 10.222.0.26 () {58 vars in 1191 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 764 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:25.080626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.081390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.081793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.081977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.082626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:25.082750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.082920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.084584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.093210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f11f6-7aed-4980-9fc4-0ee328bfbb63 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.097388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3571/7129] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:25.097806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.098494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.098775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.099043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.099626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=109a5aee28714246a1d38da232084610)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.101070 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:25.101070 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:25.101070 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:25.101926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.102060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.107983 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3559/7130] 10.222.0.26 () {58 vars in 1290 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610 => generated 333 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:25.111961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.113008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c26b05-c629-4889-8392-727c39eddf3c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.120212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.120936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.121320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.121632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.122282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.123429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75e383-b40d-4f5b-bcbe-142ab3594769 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.125203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.128448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.129398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.129767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.130229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.130889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=109a5aee28714246a1d38da232084610)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.132581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.137437 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3560/7131] 10.222.0.26 () {58 vars in 1291 bytes} [Sat Sep 17 21:19:25 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610 => generated 0 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 17 21:19:25.141623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.152225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b0821-a323-48c1-ab86-a0fd63c7231c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.155391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3572/7132] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:25.159645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.160301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.161125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.161492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.161911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.162702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=109a5aee28714246a1d38da232084610)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.163787 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:25.163787 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:25.163787 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:25.164822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.177357 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3561/7133] 10.222.0.26 () {58 vars in 1293 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:25.181521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.192828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a3cf-4420-436b-a89a-f8a5e3a098b4 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.233810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c4bf9-8322-43d2-9b96-6e5201c2fdc6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.239029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.240194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.240481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.240806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.241587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.244774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.250783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:25.270345 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3573/7134] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:25.271255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.271893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.272486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.272764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.273403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.275286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.277845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.281500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3562/7135] 10.222.0.26 () {58 vars in 1191 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:25.288689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.294863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed90646-eaaa-4496-be56-90fd3949c40a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.301859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfa661-e748-40d4-b22f-79bb735d5282 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.304410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.305273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.305704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.305981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.306627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.307274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.307941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.308200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.308400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.308536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.309049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=109a5aee28714246a1d38da232084610)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.310853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.321514 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3574/7136] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:25.325136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Could not find Endpoint Group: 109a5aee28714246a1d38da232084610.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 109a5aee28714246a1d38da232084610. Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Sep 17 21:19:25.325633 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Sep 17 21:19:25.326672 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Sep 17 21:19:25.326672 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Sep 17 21:19:25.326672 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Sep 17 21:19:25.326672 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Sep 17 21:19:25.326672 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 109a5aee28714246a1d38da232084610. Sep 17 21:19:25.326672 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:19:25.327085 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3563/7137] 10.222.0.26 () {58 vars in 1293 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/109a5aee28714246a1d38da232084610 => generated 120 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:25.330829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.338379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264df4c-0c3d-43d1-ab2d-f796985b2ce3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.343968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f850b-71dc-46b5-8366-dc01fcef0163 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.344193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.344811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.345094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.345408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.346034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.347677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.348767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.349712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.349989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.350334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services/44cbc6bba6ce44d3b69d2e0dbdaa1079` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.351353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_service(service_id=44cbc6bba6ce44d3b69d2e0dbdaa1079)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.353112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.364227 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3575/7138] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:25.364533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.378454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb53651-ff1a-41a6-9c8f-de0f8a30fab4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.384789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cffe3a-85a5-493f-9a10-f8c39112231b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.385496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cffe3a-85a5-493f-9a10-f8c39112231b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.385776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cffe3a-85a5-493f-9a10-f8c39112231b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.386148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cffe3a-85a5-493f-9a10-f8c39112231b admin admin] PATH_INFO: `/v3/projects/abe83fefe40d41f3810dc91f22180c63/users/92e222cd0a9b44108d0b25ca50f012b8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.389722 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3564/7139] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/services/44cbc6bba6ce44d3b69d2e0dbdaa1079 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:25.391751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.412859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cffe3a-85a5-493f-9a10-f8c39112231b admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe83fefe40d41f3810dc91f22180c63, user_id=92e222cd0a9b44108d0b25ca50f012b8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.413005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84981b42-5061-477d-974f-3e9bef3732e8 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.416278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cffe3a-85a5-493f-9a10-f8c39112231b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.419055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.420372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.421058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.421409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.422320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.424020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.433013 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3576/7140] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:25 2022] PUT /identity/v3/projects/abe83fefe40d41f3810dc91f22180c63/users/92e222cd0a9b44108d0b25ca50f012b8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:25.444833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b6ae-eee8-4ce5-97ff-6a1b0000909f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.445148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b6ae-eee8-4ce5-97ff-6a1b0000909f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.445422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b6ae-eee8-4ce5-97ff-6a1b0000909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.448659 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3565/7141] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/services => generated 287 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:25.452597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.473907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0245c6-eeb9-4c2d-a7f2-19c28b50c32b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.474162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1356b6ae-eee8-4ce5-97ff-6a1b0000909f None None] MFA Rules not processed for user `92e222cd0a9b44108d0b25ca50f012b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:25.475406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1356b6ae-eee8-4ce5-97ff-6a1b0000909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:25.500305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:25.530792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.531412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.531688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.531991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.532635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.534279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.541517 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3577/7142] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3525 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:25.545555 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3566/7143] 10.222.0.26 () {60 vars in 1213 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 331 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:25.548328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356b6ae-eee8-4ce5-97ff-6a1b0000909f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.553173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.555858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:25.559132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356b6ae-eee8-4ce5-97ff-6a1b0000909f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.563940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.565205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.565499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.565887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.566554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.568208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.587297 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3578/7144] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/services => generated 289 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:25.588186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.609311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad77e7-6a3a-4bf8-a360-22dd75e2ab1b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.614158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.614796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.615076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.615351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/2acd36e0e5734f24988d544e4f54a71a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.616111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=2acd36e0e5734f24988d544e4f54a71a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.617130 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:25.617130 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:19:25.617130 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:19:25.618063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.627841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fad6c-4611-47ee-bf6f-4bcb8b508a7c tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.636379 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3579/7145] 10.222.0.26 () {60 vars in 1313 bytes} [Sat Sep 17 21:19:25 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/2acd36e0e5734f24988d544e4f54a71a => generated 336 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:25.641246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.646998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.647618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.647895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.648165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.651573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae4d1f-df1e-4ddc-afaf-78f2895940f7 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.656234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.656816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.656941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.657120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.657389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services/5b4512923240462a96f41797b8aca3dc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.658134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_service(service_id=5b4512923240462a96f41797b8aca3dc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.658270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.659725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.667565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:25.680986 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3567/7146] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/auth/tokens => generated 3659 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:25.688577 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3580/7147] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/services/5b4512923240462a96f41797b8aca3dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:25.692966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.701581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.706276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c134c2c-db6f-406c-80f7-9a5d60b8b021 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.720792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.721428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.721813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.722094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/2acd36e0e5734f24988d544e4f54a71a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.722668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=2acd36e0e5734f24988d544e4f54a71a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.724269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.734559 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3568/7148] 10.222.0.26 () {58 vars in 1293 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/2acd36e0e5734f24988d544e4f54a71a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:25.738742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.750007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d1ba9-8d25-4056-9ff3-9030faec8256 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.755926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.756645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.756926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.757198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services/a96eb6456cf14ef38f84e2765f043cb2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.757867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_service(service_id=a96eb6456cf14ef38f84e2765f043cb2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.759490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.775619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126630-41ea-4278-ae5f-2d567943750f tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.781200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.781502 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3569/7149] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/services/a96eb6456cf14ef38f84e2765f043cb2 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:25.781854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.782112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.782451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] PATH_INFO: `/v3/users/92e222cd0a9b44108d0b25ca50f012b8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.784114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e222cd0a9b44108d0b25ca50f012b8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.785614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.786507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.806597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34f97d-bccf-4d4a-a9db-0c301305974b tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.811484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.812213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.812541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.812869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/e131511e1d224c6782fee5c024d7aa9c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.813572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=e131511e1d224c6782fee5c024d7aa9c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.815211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.824860 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3570/7150] 10.222.0.26 () {58 vars in 1293 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/e131511e1d224c6782fee5c024d7aa9c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:25.828768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.835531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e222cd0a9b44108d0b25ca50f012b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:25.835809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e222cd0a9b44108d0b25ca50f012b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:25.838791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381d57a-da13-4f95-89e3-8fcb1796e9fe tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.844173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd355151d8d564245b489773924ea4372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e2df26ab06415fac47b46b8aa77a50', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.844485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e222cd0a9b44108d0b25ca50f012b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:25.844778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.845055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.845165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e222cd0a9b44108d0b25ca50f012b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:25.845346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] PATH_INFO: `/v3/services/a138134f030d4f91aec313c9c0849cac` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.845932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorizing `identity:delete_service(service_id=a138134f030d4f91aec313c9c0849cac)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.847714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.850940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:UPDATE=1,keystone:DELETE=11 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:25.858978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0d52bf27-9a3e-477b-850f-9b93abf58994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e222cd0a9b44108d0b25ca50f012b8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:25.870298 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3571/7151] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/services/a138134f030d4f91aec313c9c0849cac => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:25.874269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.875852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:25.875988 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3581/7152] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/users/92e222cd0a9b44108d0b25ca50f012b8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:25.880121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.880579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.880909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.898922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1650719-70e2-4ef7-9877-56e6bd55a146 tempest-EndPointGroupsTest-1310139288 tempest-EndPointGroupsTest-1310139288-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:25.901709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:25.902974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:25.954464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:25.955153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:25.955430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:25.955703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] PATH_INFO: `/v3/users/333e4238006349ed99312ac3a8ab4a96` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:25.956825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333e4238006349ed99312ac3a8ab4a96)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:25.958798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:25.961610 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3582/7153] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:25.971561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:25.972928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.005694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333e4238006349ed99312ac3a8ab4a96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.011755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=16 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:26.019637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333e4238006349ed99312ac3a8ab4a96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.030015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333e4238006349ed99312ac3a8ab4a96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.030679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333e4238006349ed99312ac3a8ab4a96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.031336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e0e38f2a-a3aa-46f2-bc53-b7140fcd5527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333e4238006349ed99312ac3a8ab4a96'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.044089 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3572/7154] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:25 2022] DELETE /identity/v3/users/333e4238006349ed99312ac3a8ab4a96 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:26.044555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfd5c2-18fb-4e8a-8726-968965304eb8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:26.048190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:26.048679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:26.048792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:26.068545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:26.069835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.099731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:26.100789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:26.101072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:26.101324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:26.103123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.133378 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3573/7155] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:26.143254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:26.145606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.153325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:26.162270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ddba6-a204-451c-8483-cdab8bbecc1d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:26.167124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:26.168119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:26.168686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:26.169372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:26.179867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:26.180513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:26.181448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:26.182417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:26.192463 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3574/7156] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:26.197656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3583/7157] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:26.674527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:26.677570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.709122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817d049-7475-40c3-a2ea-e3befa1e865f service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:26.727117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:26.727744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:26.728866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:26.729381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] PATH_INFO: `/v3/projects/331ad21b034444c9a3bc2a029060c420` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:26.732103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] RBAC: Authorizing `identity:delete_project(project_id=331ad21b034444c9a3bc2a029060c420)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:26.735084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:26.749825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:26.753307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.774444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041330d3-4edd-4a53-b890-df5a99493006 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:26.780879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:26.781962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:26.782306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:26.782717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] PATH_INFO: `/v3/projects/abe83fefe40d41f3810dc91f22180c63` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:26.784214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe83fefe40d41f3810dc91f22180c63)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:26.786740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:26.833289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '331ad21b034444c9a3bc2a029060c420', 'request_id': 'req-b1edbfb6-6a99-414e-a3b9-629bf358823f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.845037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '331ad21b034444c9a3bc2a029060c420', 'request_id': 'req-b1edbfb6-6a99-414e-a3b9-629bf358823f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.852696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '331ad21b034444c9a3bc2a029060c420', 'request_id': 'req-b1edbfb6-6a99-414e-a3b9-629bf358823f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.859331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe83fefe40d41f3810dc91f22180c63', 'request_id': 'req-caa8983f-22a6-4866-b27f-fca8871f1d23'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.867090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] The token cache is being invalidate because project 331ad21b034444c9a3bc2a029060c420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:26.867388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.868014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe83fefe40d41f3810dc91f22180c63', 'request_id': 'req-caa8983f-22a6-4866-b27f-fca8871f1d23'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.868162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.870779 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3575/7158] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:26 2022] DELETE /identity/v3/projects/331ad21b034444c9a3bc2a029060c420 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:26.875949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:26.877542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.882567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe83fefe40d41f3810dc91f22180c63', 'request_id': 'req-caa8983f-22a6-4866-b27f-fca8871f1d23'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.896922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] The token cache is being invalidate because project abe83fefe40d41f3810dc91f22180c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:26.897243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.897929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-caa8983f-22a6-4866-b27f-fca8871f1d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:26.899433 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3584/7159] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:26 2022] DELETE /identity/v3/projects/abe83fefe40d41f3810dc91f22180c63 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:26.904910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac4a1b-5033-4d53-8d04-0d1e4b50d7d1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:26.905196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac4a1b-5033-4d53-8d04-0d1e4b50d7d1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:26.905569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac4a1b-5033-4d53-8d04-0d1e4b50d7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:26.927325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-03ac4a1b-5033-4d53-8d04-0d1e4b50d7d1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:26.928818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-03ac4a1b-5033-4d53-8d04-0d1e4b50d7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:26.932961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:26.957083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edbfb6-6a99-414e-a3b9-629bf358823f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:26.998259 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3585/7160] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:27.007934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8174b-bf2b-4aa1-a975-61ae4ea5fa8c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.008221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8174b-bf2b-4aa1-a975-61ae4ea5fa8c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.008604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8174b-bf2b-4aa1-a975-61ae4ea5fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.020541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.021179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.021473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.021767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] PATH_INFO: `/v3/users/d355151d8d564245b489773924ea4372` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.022966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d355151d8d564245b489773924ea4372)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.024943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.030108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-60a8174b-bf2b-4aa1-a975-61ae4ea5fa8c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:27.031330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-60a8174b-bf2b-4aa1-a975-61ae4ea5fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.040173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3586/7161] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:27.046486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8174b-bf2b-4aa1-a975-61ae4ea5fa8c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.058662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8174b-bf2b-4aa1-a975-61ae4ea5fa8c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.064945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.065718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.065999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.066387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.066890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:27.067199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.071845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.082271 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3587/7162] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:27.084790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd355151d8d564245b489773924ea4372'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.086511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.098928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46190e2e-c195-425f-ac74-a5c662b3a8fe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.099704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd355151d8d564245b489773924ea4372'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.103754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.104857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.105161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.105438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.106145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.108185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.109868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd355151d8d564245b489773924ea4372'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.110664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd355151d8d564245b489773924ea4372'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.111196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd355151d8d564245b489773924ea4372'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.129499 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3576/7163] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:26 2022] DELETE /identity/v3/users/d355151d8d564245b489773924ea4372 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:27.154446 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3588/7164] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/projects => generated 361 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:27.159217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.160891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.240048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85064fcd-dc21-4294-af8f-7e23903d8160 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.256514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:27.318163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.320921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.321550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.322077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.322801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.325445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.361369 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3577/7165] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/users => generated 281 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:27.365449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.377518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e75702-e84e-463b-aeeb-201085c5409b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.383100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.383840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.384221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.384622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.385410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.387240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.399047 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3589/7166] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:27.402994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.413834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199638a-2d7e-41b9-99da-3597ea5be521 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.419353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.419826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.420461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.420461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.421853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.422788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.435969 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3578/7167] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:27.441122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.454519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f629b6-7b47-459f-a2b6-1ea6b29a6227 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.457754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.461426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.471133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac88a4-d3c8-494b-a370-1358d477f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.472287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac88a4-d3c8-494b-a370-1358d477f8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.473311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac88a4-d3c8-494b-a370-1358d477f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.473926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac88a4-d3c8-494b-a370-1358d477f8e6 admin admin] PATH_INFO: `/v3/projects/3cb1bde2947440679a887963f9bf91db/users/35e804bc0ac947c48e1f2dbea960fe2e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.476821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:27.481324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ec750-5da9-444f-8e50-5a8067a57ed8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.492738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.493400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.494075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.494569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] PATH_INFO: `/v3/projects/80e2df26ab06415fac47b46b8aa77a50` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.496408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e2df26ab06415fac47b46b8aa77a50)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.498574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac88a4-d3c8-494b-a370-1358d477f8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb1bde2947440679a887963f9bf91db, user_id=35e804bc0ac947c48e1f2dbea960fe2e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.502095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac88a4-d3c8-494b-a370-1358d477f8e6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.506092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.517251 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3590/7168] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:27 2022] PUT /identity/v3/projects/3cb1bde2947440679a887963f9bf91db/users/35e804bc0ac947c48e1f2dbea960fe2e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:27.528021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.528379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.528578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.558272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] MFA Rules not processed for user `35e804bc0ac947c48e1f2dbea960fe2e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:27.559496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.569131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e2df26ab06415fac47b46b8aa77a50', 'request_id': 'req-9fc321e7-5107-4de8-b983-be755843b277'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.581397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e2df26ab06415fac47b46b8aa77a50', 'request_id': 'req-9fc321e7-5107-4de8-b983-be755843b277'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.588720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e2df26ab06415fac47b46b8aa77a50', 'request_id': 'req-9fc321e7-5107-4de8-b983-be755843b277'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.606702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] The token cache is being invalidate because project 80e2df26ab06415fac47b46b8aa77a50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:27.607059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.607917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9fc321e7-5107-4de8-b983-be755843b277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.610657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3579/7169] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:27 2022] DELETE /identity/v3/projects/80e2df26ab06415fac47b46b8aa77a50 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:27.615560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03750d7-9788-4fe7-a243-3a1f45cc56c9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.615962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03750d7-9788-4fe7-a243-3a1f45cc56c9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.616300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03750d7-9788-4fe7-a243-3a1f45cc56c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.639436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f03750d7-9788-4fe7-a243-3a1f45cc56c9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:27.640780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f03750d7-9788-4fe7-a243-3a1f45cc56c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.641602 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3591/7170] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:27.675849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.677884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.723937 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3580/7171] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:27.732981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebccfb3-20b2-4964-813c-1f9d896daba4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.733270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebccfb3-20b2-4964-813c-1f9d896daba4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.733863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebccfb3-20b2-4964-813c-1f9d896daba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.762058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044e7b4-c2dc-45a6-885d-9cd3735eca7c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.765568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3ebccfb3-20b2-4964-813c-1f9d896daba4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:27.766105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.766829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.767094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3ebccfb3-20b2-4964-813c-1f9d896daba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.767219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.767519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.778038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.779344 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3581/7172] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:27.779697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.783390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebccfb3-20b2-4964-813c-1f9d896daba4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.794719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebccfb3-20b2-4964-813c-1f9d896daba4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.795743 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3592/7173] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:27.800150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.800812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.801131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.801403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.802149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:27.802548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.810962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.812909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:27.813116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.815291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:27.819745 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3582/7174] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:27.823805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.832878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273623-48d3-4040-a087-32c146f3feac service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.837438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3644a-dc44-4b81-b289-f5248a6e926c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:27.839744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.840566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.840849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.841123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] PATH_INFO: `/v3/users/35e804bc0ac947c48e1f2dbea960fe2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.842409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e804bc0ac947c48e1f2dbea960fe2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.842777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:27.843480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.844104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.844449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.844549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.845163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:27.847415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:27.892615 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3583/7175] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:27.896508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:27.917569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e804bc0ac947c48e1f2dbea960fe2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.918354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e804bc0ac947c48e1f2dbea960fe2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.930007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e804bc0ac947c48e1f2dbea960fe2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.930647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e804bc0ac947c48e1f2dbea960fe2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.937992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:27.947474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13582b68-0859-4ad1-9237-f2b08343af75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e804bc0ac947c48e1f2dbea960fe2e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:27.961154 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3593/7176] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:27 2022] DELETE /identity/v3/users/35e804bc0ac947c48e1f2dbea960fe2e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:27.973557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:27.973557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:27.973557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:27.983187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b08a3-4475-4ef8-a2ad-bbb513f6cdd7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.005292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:28.006614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:28.045271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.045919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.046198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.046531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.047187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.049156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.073060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3584/7177] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:28.077280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.077982 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3594/7178] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:28.079252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:28.088323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.089915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:28.092468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0797cea-d067-4c6c-be6e-6ea7707ec012 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.101411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.102600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.103096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.103398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.103954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.106095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.121838 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3585/7179] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:28.122934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.134736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270360f7-0d98-4ee8-b4ed-333c7bbf953f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.140685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.141423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.141735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.142009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.142792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.144891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.157504 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3586/7180] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:28.163994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.168586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7929f-efc9-4cd7-917b-33792b20feea None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.175185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc088d5-91cc-47bd-a4c2-7934029e8c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.176079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc088d5-91cc-47bd-a4c2-7934029e8c69 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.176372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7c6ef-eacd-4de8-a914-5143274a83b8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.176826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc088d5-91cc-47bd-a4c2-7934029e8c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.177318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc088d5-91cc-47bd-a4c2-7934029e8c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.183102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.183791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.184041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.184367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] PATH_INFO: `/v3/projects/2fcacf5d47d84aa2aa6a477d08befd01/users/8adbcc377a9b46d2a647b6e39ed0fdaa/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.193343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc088d5-91cc-47bd-a4c2-7934029e8c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.195309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc088d5-91cc-47bd-a4c2-7934029e8c69 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.204280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fcacf5d47d84aa2aa6a477d08befd01, user_id=8adbcc377a9b46d2a647b6e39ed0fdaa, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.207982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.211362 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3595/7181] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:28.231127 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3587/7182] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:28 2022] PUT /identity/v3/projects/2fcacf5d47d84aa2aa6a477d08befd01/users/8adbcc377a9b46d2a647b6e39ed0fdaa/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:28.240079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a6f28-9351-460b-9b70-68480c1166a1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.240413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a6f28-9351-460b-9b70-68480c1166a1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.240672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a6f28-9351-460b-9b70-68480c1166a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.277329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4e5a6f28-9351-460b-9b70-68480c1166a1 None None] MFA Rules not processed for user `8adbcc377a9b46d2a647b6e39ed0fdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:28.279837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4e5a6f28-9351-460b-9b70-68480c1166a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:28.394682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.396208 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3596/7183] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:28.431639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:28.473428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09635e2-c698-4c16-aa82-1de9cede1a6b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.479428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.480262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.480562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.480938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.481862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.484161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.523298 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3588/7184] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:28 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:28.527489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a6f28-9351-460b-9b70-68480c1166a1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.542432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a6f28-9351-460b-9b70-68480c1166a1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.554482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:28.663465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.666117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:28.667915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.668554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.668839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.669113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.669791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.677267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.712105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27c2fa-7355-42e0-952e-dda4a8c8a542 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.715850 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3597/7185] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:28 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:28.720937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.724133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.724868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.726600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.726749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] PATH_INFO: `/v3/projects/3cb1bde2947440679a887963f9bf91db` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.728570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb1bde2947440679a887963f9bf91db)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.733863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.743203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae746012-450f-48d3-bdb0-b2326581e455 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.781113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.781781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.782055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.782482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.783016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.786544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.809158 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3598/7186] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:28.809957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.832509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eba8db-7b30-4085-9541-52ab5f15c5db admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:28.847554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb1bde2947440679a887963f9bf91db', 'request_id': 'req-3b434df2-bf2d-4863-bd4f-a2b3891f949b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:28.860277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb1bde2947440679a887963f9bf91db', 'request_id': 'req-3b434df2-bf2d-4863-bd4f-a2b3891f949b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:28.883165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb1bde2947440679a887963f9bf91db', 'request_id': 'req-3b434df2-bf2d-4863-bd4f-a2b3891f949b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:28.901953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] The token cache is being invalidate because project 3cb1bde2947440679a887963f9bf91db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:28.902358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:28.905070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b434df2-bf2d-4863-bd4f-a2b3891f949b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:28.913509 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3589/7187] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:28 2022] DELETE /identity/v3/projects/3cb1bde2947440679a887963f9bf91db => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:28.918523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aa952-21f3-481b-a6a1-99fc018113f0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.918812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aa952-21f3-481b-a6a1-99fc018113f0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.920416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5aa952-21f3-481b-a6a1-99fc018113f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.930142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:28.930848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.931131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.931411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] PATH_INFO: `/v3/projects/8d3502689d8346d78953b5cb78475b4a/users/f8e12980387c446a9a86aa3862049fe2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.950869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3502689d8346d78953b5cb78475b4a, user_id=f8e12980387c446a9a86aa3862049fe2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:28.952411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ae5aa952-21f3-481b-a6a1-99fc018113f0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:28.954617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ae5aa952-21f3-481b-a6a1-99fc018113f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:28.954796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:28.966459 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3590/7188] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:28.976069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d5bb-6621-4db6-83f2-9f8f58c47038 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:28.976589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d5bb-6621-4db6-83f2-9f8f58c47038 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:28.977017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d5bb-6621-4db6-83f2-9f8f58c47038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:28.980129 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3599/7189] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:28 2022] PUT /identity/v3/projects/8d3502689d8346d78953b5cb78475b4a/users/f8e12980387c446a9a86aa3862049fe2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:28.983918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:28.986218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:29.003643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1a47d5bb-6621-4db6-83f2-9f8f58c47038 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:29.004993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1a47d5bb-6621-4db6-83f2-9f8f58c47038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:29.068445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72d591-a288-4cc0-b920-836261cd06d2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.075580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.075580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.075580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.075580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.076253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.077627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.086341 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3591/7190] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:29.091614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d5bb-6621-4db6-83f2-9f8f58c47038 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.094284 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3600/7191] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:29.098358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.108138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d5bb-6621-4db6-83f2-9f8f58c47038 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.109387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e32aa-b26e-4440-851a-b30250bbcc7e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.114434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44fce3-5247-4f2f-8314-71ff2b39ab27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.114689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.115115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44fce3-5247-4f2f-8314-71ff2b39ab27 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.115394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44fce3-5247-4f2f-8314-71ff2b39ab27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.115516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.115823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44fce3-5247-4f2f-8314-71ff2b39ab27 admin admin] PATH_INFO: `/v3/projects/8d3502689d8346d78953b5cb78475b4a/users/f8e12980387c446a9a86aa3862049fe2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.115991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.116359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.117100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:29.117493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.119076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44fce3-5247-4f2f-8314-71ff2b39ab27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d3502689d8346d78953b5cb78475b4a, user_id=f8e12980387c446a9a86aa3862049fe2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.119727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.121304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44fce3-5247-4f2f-8314-71ff2b39ab27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.125357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:29.134102 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3601/7192] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:29 2022] PUT /identity/v3/projects/8d3502689d8346d78953b5cb78475b4a/users/f8e12980387c446a9a86aa3862049fe2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:29.146733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c9e8f-75b6-445a-b022-9fed16ba7627 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.147449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c9e8f-75b6-445a-b022-9fed16ba7627 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.147727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c9e8f-75b6-445a-b022-9fed16ba7627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.148797 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3592/7193] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:29.149765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.163991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fa20b-365a-4c46-8007-79ba061f6f2f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.183565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-683c9e8f-75b6-445a-b022-9fed16ba7627 None None] MFA Rules not processed for user `f8e12980387c446a9a86aa3862049fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:29.183984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:29.184941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-683c9e8f-75b6-445a-b022-9fed16ba7627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:29.245537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.246265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.246551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.246824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.247468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.249440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.286625 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3593/7194] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:29 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:29.287819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.296507 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3602/7195] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:29.301643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c9e8f-75b6-445a-b022-9fed16ba7627 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.311968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c9e8f-75b6-445a-b022-9fed16ba7627 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.366564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910704b-6e79-4277-81d3-52fb3742e9a7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.371923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.372617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.372992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.373271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.373936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.374070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.374631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.374984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.375327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.375892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.376163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.377927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.398038 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3603/7196] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:19:29 2022] POST /identity/v3/domains => generated 275 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:29.402633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.405566 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3594/7197] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:29 2022] POST /identity/v3/users => generated 282 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:29.409947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.423853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ba48-0dbe-4c75-a655-f9f623dea535 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.491003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.491682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.491961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.492235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.492896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.495181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.508149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3595/7198] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:29.517796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243328d6-963d-41f7-b11e-e46faea40279 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.528787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.530323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.530716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.531161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.531781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.538899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.548012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.559680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57eba-25ce-4632-85d7-66823215ef8b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.566633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:29.567546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.568358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.568662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.569067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.570078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.572409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.585487 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3604/7199] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:29 2022] POST /identity/v3/users => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:29.586642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.593509 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3596/7200] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:29.597040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.598665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719ebcc-b4d2-4678-bd92-db7459912fd9 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.605111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.605859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.606144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.606422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.607074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.608049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec8ea-6842-43a5-af87-b993f3ed53c5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.608966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.612915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa040db-52ca-42df-9680-f8d16e654042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.613716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa040db-52ca-42df-9680-f8d16e654042 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.614404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa040db-52ca-42df-9680-f8d16e654042 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.614404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa040db-52ca-42df-9680-f8d16e654042 admin admin] PATH_INFO: `/v3/projects/c8d2ef6638184f178a760075af39cc8b/users/53aa66e4f5134dcf9adb7aa238ba43de/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.633561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:29.637510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa040db-52ca-42df-9680-f8d16e654042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d2ef6638184f178a760075af39cc8b, user_id=53aa66e4f5134dcf9adb7aa238ba43de, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.643399 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3605/7201] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:29 2022] POST /identity/v3/users => generated 323 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:29.643730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa040db-52ca-42df-9680-f8d16e654042 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.644731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.655771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15596beb-df13-4c98-ab9a-337bfe8e0eb1 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.658691 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3597/7202] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:29 2022] PUT /identity/v3/projects/c8d2ef6638184f178a760075af39cc8b/users/53aa66e4f5134dcf9adb7aa238ba43de/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:29.673725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.673725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.673725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.713011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] MFA Rules not processed for user `53aa66e4f5134dcf9adb7aa238ba43de`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:29.714806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:29.724621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.724982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.725310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.725659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users/a4f3bf9d0d1347398bee6c193305ee17` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.741374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:get_user(user_id=a4f3bf9d0d1347398bee6c193305ee17)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.744209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.746290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3606/7203] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/users/a4f3bf9d0d1347398bee6c193305ee17 => generated 347 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:29.753170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.764337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d4954-952d-489e-971c-409bc414c374 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.769259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.770215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.770735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.770858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.771567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Adding query filter params (domain_id=9086eedbab884090926a30f57e21fbfe) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:29.771905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.773143 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:29.773143 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:19:29.773143 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:19:29.774846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.787111 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3607/7204] 10.222.0.26 () {58 vars in 1207 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/users?domain_id=9086eedbab884090926a30f57e21fbfe => generated 484 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:29.791989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.797913 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3598/7205] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:29.805510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314d006-4bbe-49c7-b453-a761833f3509 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.811302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.811935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.812273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.812557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.813242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.815169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.818234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.819867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:29.860400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3608/7206] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/users => generated 8007 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:29.861357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.873696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864cf25-0253-484b-97df-ae8c54c8f427 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.878923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.879692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.879980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.880848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.881660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Adding query filter params (name=tempest-test_user-670738631) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:29.882000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.883928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.891093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f9536-5921-46e4-b162-f952ae777359 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.895746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.896368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.896646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.897374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.906361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.908275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.918465 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3599/7207] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:29.924477 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3609/7208] 10.222.0.26 () {58 vars in 1187 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/users?name=tempest-test_user-670738631 => generated 474 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:29.929973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:29.941933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297f420-e0e4-4162-8124-e8002ec46a4f service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:29.947083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c41c4a-5f31-48f2-9195-e9999b6a9f66 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:29.947918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c41c4a-5f31-48f2-9195-e9999b6a9f66 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:29.948102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c41c4a-5f31-48f2-9195-e9999b6a9f66 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:29.948396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c41c4a-5f31-48f2-9195-e9999b6a9f66 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:29.949105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c41c4a-5f31-48f2-9195-e9999b6a9f66 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:29.949408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c41c4a-5f31-48f2-9195-e9999b6a9f66 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:29.951578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c41c4a-5f31-48f2-9195-e9999b6a9f66 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:29.992369 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3600/7209] 10.222.0.26 () {58 vars in 1149 bytes} [Sat Sep 17 21:19:29 2022] GET /identity/v3/users?enabled=False => generated 431 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:29.997222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:30.009055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f823f67-0e31-49c2-8a10-04a6320e2313 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:30.011083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.011464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.011667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:30.015655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:30.016373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.016577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.016856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users/0c38256ef17f4e928e1e246567f54371` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:30.033968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:30.034564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0c38256ef17f4e928e1e246567f54371)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:30.035206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:30.036980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:30.087039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c38256ef17f4e928e1e246567f54371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.088387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c38256ef17f4e928e1e246567f54371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.097806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c38256ef17f4e928e1e246567f54371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.098656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c38256ef17f4e928e1e246567f54371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.119404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c38256ef17f4e928e1e246567f54371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.121303 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3601/7210] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:30.125360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:30.138469 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3610/7211] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:29 2022] DELETE /identity/v3/users/0c38256ef17f4e928e1e246567f54371 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:30.138653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:30.139704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:30.140142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:30.142967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:30.151846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:30.227626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67211ff-da96-46ad-ae78-cc90e31ce78d tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:30.233233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:30.234218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.234516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.234808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/users/a4f3bf9d0d1347398bee6c193305ee17` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:30.236005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a4f3bf9d0d1347398bee6c193305ee17)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:30.238393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:30.239614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41c10a-7d1b-4c81-acfb-0b6d9b0c3d06 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:30.306660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f3bf9d0d1347398bee6c193305ee17'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.307732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f3bf9d0d1347398bee6c193305ee17'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.320057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:30.320737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.321013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.321283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f3bf9d0d1347398bee6c193305ee17'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.321397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:30.322819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f3bf9d0d1347398bee6c193305ee17'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.323971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:30.340989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f3bf9d0d1347398bee6c193305ee17'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.354895 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3611/7212] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:30 2022] DELETE /identity/v3/users/a4f3bf9d0d1347398bee6c193305ee17 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:30.358956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:30.360529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:30.393980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:30.395708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:30.404591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3602/7213] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:19:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:19:30.436067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91f65e-6222-4de6-add4-23c0d6db3873 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:30.445822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:30.447078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.447774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.448765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/domains/9086eedbab884090926a30f57e21fbfe` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:30.450482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=9086eedbab884090926a30f57e21fbfe)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:30.454392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:30.482458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:30.487130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '9086eedbab884090926a30f57e21fbfe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.488433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '9086eedbab884090926a30f57e21fbfe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.526630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '9086eedbab884090926a30f57e21fbfe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.527439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '9086eedbab884090926a30f57e21fbfe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.530205 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3612/7214] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:19:30 2022] PATCH /identity/v3/domains/9086eedbab884090926a30f57e21fbfe => generated 276 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:30.534164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:30.546289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2b030-34b8-466b-ae2b-a3bc72f1ba66 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:30.593738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:30.613804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8e12980387c446a9a86aa3862049fe2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d3502689d8346d78953b5cb78475b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:30.614609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.614982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.615351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] PATH_INFO: `/v3/domains/9086eedbab884090926a30f57e21fbfe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:30.629105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=9086eedbab884090926a30f57e21fbfe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:30.631078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:30.655326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '9086eedbab884090926a30f57e21fbfe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.665490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '9086eedbab884090926a30f57e21fbfe'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.714109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9086eedbab884090926a30f57e21fbfe', 'request_id': 'req-fe4b1293-45af-4fd7-9f80-7ee99f48909b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.722272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9086eedbab884090926a30f57e21fbfe', 'request_id': 'req-fe4b1293-45af-4fd7-9f80-7ee99f48909b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.728316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9086eedbab884090926a30f57e21fbfe', 'request_id': 'req-fe4b1293-45af-4fd7-9f80-7ee99f48909b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.740639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] The token cache is being invalidate because project 9086eedbab884090926a30f57e21fbfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:30.741010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.742118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.752609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '9086eedbab884090926a30f57e21fbfe', 'request_id': 'req-fe4b1293-45af-4fd7-9f80-7ee99f48909b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.756267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '9086eedbab884090926a30f57e21fbfe', 'request_id': 'req-fe4b1293-45af-4fd7-9f80-7ee99f48909b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.756979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fe4b1293-45af-4fd7-9f80-7ee99f48909b tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '9086eedbab884090926a30f57e21fbfe', 'request_id': 'req-fe4b1293-45af-4fd7-9f80-7ee99f48909b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.762671 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3603/7215] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:30 2022] DELETE /identity/v3/domains/9086eedbab884090926a30f57e21fbfe => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:30.766827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:30.768273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:30.834238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bc7db-1338-4b5d-bc8d-0ff165738ce2 tempest-UsersV3TestJSON-1048853495 tempest-UsersV3TestJSON-1048853495-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:30.892427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:30.893069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.893348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.893652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] PATH_INFO: `/v3/users/8adbcc377a9b46d2a647b6e39ed0fdaa` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:30.894796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8adbcc377a9b46d2a647b6e39ed0fdaa)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:30.896744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:30.946240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8adbcc377a9b46d2a647b6e39ed0fdaa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.946861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8adbcc377a9b46d2a647b6e39ed0fdaa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.955321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8adbcc377a9b46d2a647b6e39ed0fdaa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.956006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8adbcc377a9b46d2a647b6e39ed0fdaa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.969086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8adbcc377a9b46d2a647b6e39ed0fdaa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:30.982735 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3613/7216] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:30 2022] DELETE /identity/v3/users/8adbcc377a9b46d2a647b6e39ed0fdaa => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:30.987523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:30.987798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:30.988089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:31.008703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:31.009948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:31.069983 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3604/7217] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:31.079938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:31.081514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:31.089136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:31.146532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd50d3e-49ba-4ffa-b7f2-8143ec2e5b24 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:31.151152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:31.151779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:31.152052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:31.152320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:31.162762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:31.164241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:31.174108 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3614/7218] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:31.464786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:31.466345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:31.481796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432750e-0baa-4c76-8c1c-a05609597d3a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:31.487870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:31.488557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:31.490655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:31.491030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] PATH_INFO: `/v3/projects/2fcacf5d47d84aa2aa6a477d08befd01` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:31.492484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fcacf5d47d84aa2aa6a477d08befd01)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:31.495686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:31.513702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:31.547260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fcacf5d47d84aa2aa6a477d08befd01', 'request_id': 'req-ff31339b-e09d-4475-ba47-b6d6e23e29ff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.559006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fcacf5d47d84aa2aa6a477d08befd01', 'request_id': 'req-ff31339b-e09d-4475-ba47-b6d6e23e29ff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.565673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fcacf5d47d84aa2aa6a477d08befd01', 'request_id': 'req-ff31339b-e09d-4475-ba47-b6d6e23e29ff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.579977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] The token cache is being invalidate because project 2fcacf5d47d84aa2aa6a477d08befd01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:31.580303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.581072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.582818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3605/7219] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:31 2022] DELETE /identity/v3/projects/2fcacf5d47d84aa2aa6a477d08befd01 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:31.586746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:31.597063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84f0bd-24a9-4815-bbd0-90cf3b530214 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:31.672069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:31.673668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:31.674510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:31.674510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] PATH_INFO: `/v3/users/f8e12980387c446a9a86aa3862049fe2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:31.676219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e12980387c446a9a86aa3862049fe2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:31.678348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:31.744773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e12980387c446a9a86aa3862049fe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.750096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e12980387c446a9a86aa3862049fe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.760757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e12980387c446a9a86aa3862049fe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.761667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e12980387c446a9a86aa3862049fe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.774700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e82e8dc2-6186-41ed-b3c4-aee49ed1190d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e12980387c446a9a86aa3862049fe2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:31.788018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3615/7220] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:31 2022] DELETE /identity/v3/users/f8e12980387c446a9a86aa3862049fe2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:32.090536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.092523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:32.115515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31339b-e09d-4475-ba47-b6d6e23e29ff admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.183722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.184397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.184669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.184934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] PATH_INFO: `/v3/projects/8d3502689d8346d78953b5cb78475b4a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.186118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d3502689d8346d78953b5cb78475b4a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.188109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.237698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d3502689d8346d78953b5cb78475b4a', 'request_id': 'req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:32.247556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d3502689d8346d78953b5cb78475b4a', 'request_id': 'req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:32.253370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d3502689d8346d78953b5cb78475b4a', 'request_id': 'req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:32.264469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] The token cache is being invalidate because project 8d3502689d8346d78953b5cb78475b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:32.264755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:32.265513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-22f0cea5-b513-41ed-bbd2-9b3a6d96f385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:32.267137 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3606/7221] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:32 2022] DELETE /identity/v3/projects/8d3502689d8346d78953b5cb78475b4a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:32.272479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9cb31-c029-4ea3-825b-dca61faf8e70 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.273017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9cb31-c029-4ea3-825b-dca61faf8e70 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.273017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9cb31-c029-4ea3-825b-dca61faf8e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.294360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9af9cb31-c029-4ea3-825b-dca61faf8e70 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:32.295566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9af9cb31-c029-4ea3-825b-dca61faf8e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:32.316276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:32.364867 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3616/7222] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:32.373031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445858c-9634-4dcb-b706-f528533c5618 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.373313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445858c-9634-4dcb-b706-f528533c5618 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.373626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445858c-9634-4dcb-b706-f528533c5618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.394291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a445858c-9634-4dcb-b706-f528533c5618 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:32.395495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a445858c-9634-4dcb-b706-f528533c5618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:32.404856 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3607/7223] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:32.409220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9cb31-c029-4ea3-825b-dca61faf8e70 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.419744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9cb31-c029-4ea3-825b-dca61faf8e70 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.425420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.426094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.426369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.426633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.427285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:32.427579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.429198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.441067 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3617/7224] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:32.445625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445858c-9634-4dcb-b706-f528533c5618 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.455302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445858c-9634-4dcb-b706-f528533c5618 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.460994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.461604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.461906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.462173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.462795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.464693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.496350 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3608/7225] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:32 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:32.500595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.511110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503d268-1194-4da9-a396-e1ef7b4759ae admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.571329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.572043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.572318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.572588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.573205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.575189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.598998 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3618/7226] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:32 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:32.603429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.614329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bdf3f-bc1b-4918-a663-e6b98a6abad6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.620224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.620876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.621182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.621451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.622123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.623802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.625071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:32.635578 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3609/7227] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:32.640053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.651033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da606-9712-45c2-828e-dcbceeec4812 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.657033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb8ccd-666c-4eda-90d0-798999e5c3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.657681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb8ccd-666c-4eda-90d0-798999e5c3cc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.657961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb8ccd-666c-4eda-90d0-798999e5c3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.658242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb8ccd-666c-4eda-90d0-798999e5c3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.658860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb8ccd-666c-4eda-90d0-798999e5c3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.660622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb8ccd-666c-4eda-90d0-798999e5c3cc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.672539 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3619/7228] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:32.677073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.688108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d09e1-9fd1-4fd0-a7fb-47857afdc51f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.692963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.693655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.693930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.694200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] PATH_INFO: `/v3/projects/7732befdc807468393bf15aac300bf45/users/c0e9c75950f940b19a34db658bad0430/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.711281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7732befdc807468393bf15aac300bf45, user_id=c0e9c75950f940b19a34db658bad0430, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.714507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.733896 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3610/7229] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:32 2022] PUT /identity/v3/projects/7732befdc807468393bf15aac300bf45/users/c0e9c75950f940b19a34db658bad0430/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:32.752464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b179ae-2def-4f1e-8756-5f267a6d352f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.753068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b179ae-2def-4f1e-8756-5f267a6d352f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.753486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b179ae-2def-4f1e-8756-5f267a6d352f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.792596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-61b179ae-2def-4f1e-8756-5f267a6d352f None None] MFA Rules not processed for user `c0e9c75950f940b19a34db658bad0430`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:32.794009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-61b179ae-2def-4f1e-8756-5f267a6d352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:32.868263 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3620/7230] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:32.872986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.941008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d53015-ea0e-414d-a163-cbe70812cd4d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:32.946483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:32.947200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:32.947478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:32.947746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:32.948454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:32.950822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:32.980230 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3611/7231] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:32 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:32.984316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b179ae-2def-4f1e-8756-5f267a6d352f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:32.996456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b179ae-2def-4f1e-8756-5f267a6d352f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.016340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:33.059013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.059635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.059909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.060179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.060974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.063061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.093030 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3621/7232] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:32 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:33.097135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.112722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8659c-6244-484d-a702-136f1c921d72 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.115357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.115972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.116275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.116544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.117204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.119383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.134553 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3612/7233] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:33.139757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.154674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a579b-8c3e-4583-a64a-5160512b5483 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.160425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.161189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.161645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.161981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] PATH_INFO: `/v3/projects/498f4d66d23b4800922dff39a28c7d6f/users/f26edde148b94e74a20daef86a4f4aea/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.179097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] RBAC: Authorizing `identity:create_grant(project_id=498f4d66d23b4800922dff39a28c7d6f, user_id=f26edde148b94e74a20daef86a4f4aea, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.182052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.197992 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3622/7234] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:33 2022] PUT /identity/v3/projects/498f4d66d23b4800922dff39a28c7d6f/users/f26edde148b94e74a20daef86a4f4aea/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:33.202173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.212850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c48c9e-5efd-440d-b520-11a1b20cbd51 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.273297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c803f70-e1e7-48c0-a6f6-f151aeb112d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.274105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c803f70-e1e7-48c0-a6f6-f151aeb112d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.274407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c803f70-e1e7-48c0-a6f6-f151aeb112d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.274760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c803f70-e1e7-48c0-a6f6-f151aeb112d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.275488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c803f70-e1e7-48c0-a6f6-f151aeb112d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.277256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c803f70-e1e7-48c0-a6f6-f151aeb112d7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.288997 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3613/7235] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:33.294082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.304873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5c98e-009f-4c26-b142-f0524f5958be admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.309972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.310799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.311109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.311470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] PATH_INFO: `/v3/projects/498f4d66d23b4800922dff39a28c7d6f/users/f26edde148b94e74a20daef86a4f4aea/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.313819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=498f4d66d23b4800922dff39a28c7d6f, user_id=f26edde148b94e74a20daef86a4f4aea, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.316533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.327737 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3623/7236] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:33 2022] PUT /identity/v3/projects/498f4d66d23b4800922dff39a28c7d6f/users/f26edde148b94e74a20daef86a4f4aea/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:33.339439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c2b9-705f-41f5-93a1-15a2a385553b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.339680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c2b9-705f-41f5-93a1-15a2a385553b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.340034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c2b9-705f-41f5-93a1-15a2a385553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.370397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6857c2b9-705f-41f5-93a1-15a2a385553b None None] MFA Rules not processed for user `f26edde148b94e74a20daef86a4f4aea`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:33.371547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6857c2b9-705f-41f5-93a1-15a2a385553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:33.404079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:33.449524 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3614/7237] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:33.454879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.467809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3cf23-a8ee-4fce-9f38-0420c63972b6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.473137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.473882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.474490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.474766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.475982 ncdv-master-1 devstack@keystone.service[99167]: WARNING oslo_log.versionutils [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 17 21:19:33.476777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.478102 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:33.478102 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:19:33.478102 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:19:33.479140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.509913 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3624/7238] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:19:33 2022] POST /identity/v3/policies => generated 229 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:33.513960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857c2b9-705f-41f5-93a1-15a2a385553b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.524842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857c2b9-705f-41f5-93a1-15a2a385553b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.534742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.535389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.535665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.535934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies/0b74da98daaa434b925ae71a26bbd074` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.536909 ncdv-master-1 devstack@keystone.service[99166]: WARNING oslo_log.versionutils [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 17 21:19:33.537607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=0b74da98daaa434b925ae71a26bbd074)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.538893 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:33.538893 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:33.538893 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:33.539781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.565802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3615/7239] 10.222.0.26 () {60 vars in 1252 bytes} [Sat Sep 17 21:19:33 2022] PATCH /identity/v3/policies/0b74da98daaa434b925ae71a26bbd074 => generated 235 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:33.570440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.582576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4e246-3b10-4ecc-8f11-30b7b5866959 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.593857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.594544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.594836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.595154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies/0b74da98daaa434b925ae71a26bbd074` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.595986 ncdv-master-1 devstack@keystone.service[99167]: WARNING oslo_log.versionutils [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 17 21:19:33.596580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=0b74da98daaa434b925ae71a26bbd074)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.597921 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:33.597921 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:19:33.597921 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:19:33.599152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.609657 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3625/7240] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:33 2022] GET /identity/v3/policies/0b74da98daaa434b925ae71a26bbd074 => generated 235 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:33.614125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.625847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147453ec-5816-4b05-ac38-d8943ea25479 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.631123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.631773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.632114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.632391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies/0b74da98daaa434b925ae71a26bbd074` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.633170 ncdv-master-1 devstack@keystone.service[99166]: WARNING oslo_log.versionutils [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 17 21:19:33.633769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=0b74da98daaa434b925ae71a26bbd074)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.635133 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:33.635133 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:33.635133 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:33.636032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.654773 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3616/7241] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:33 2022] DELETE /identity/v3/policies/0b74da98daaa434b925ae71a26bbd074 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:33.660070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.671110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340ad3-257e-4da7-b66b-09dea62a1c38 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.676065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.676708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.676990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.677263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.678496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.680538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.711739 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3626/7242] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:19:33 2022] POST /identity/v3/policies => generated 229 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:33.713449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.730248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aef9b-5a07-4f26-a1c8-a1e161fabcae tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.735192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.735802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.736082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.736353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.737306 ncdv-master-1 devstack@keystone.service[99166]: WARNING oslo_log.versionutils [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 17 21:19:33.737828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.739066 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:33.739066 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:33.739066 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:33.740035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.762316 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3617/7243] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:19:33 2022] POST /identity/v3/policies => generated 230 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:33.766905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.778552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a23717-a22d-4c4e-86d3-d523489ccaf2 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.783365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.784444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.784843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.785455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.786788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.789585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.810407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3627/7244] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:19:33 2022] POST /identity/v3/policies => generated 227 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:33.812122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.823273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdabd5-0e16-4ebc-8256-ffb32bc33225 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.828813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.829436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.829750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.830024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.830817 ncdv-master-1 devstack@keystone.service[99166]: WARNING oslo_log.versionutils [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 17 21:19:33.831367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.832620 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:33.832620 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:33.832620 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:33.833551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.843736 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3618/7245] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:19:33 2022] GET /identity/v3/policies => generated 762 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:33.848199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.861811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433d255-8ad9-428d-8e51-0f4c9b2ffd26 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.868222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.869255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.869587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.869861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies/9c9a97cc2be44968a9502740c330d2de` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.870675 ncdv-master-1 devstack@keystone.service[99167]: WARNING oslo_log.versionutils [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 17 21:19:33.871226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=9c9a97cc2be44968a9502740c330d2de)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.872457 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:33.872457 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:19:33.872457 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:19:33.873338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.892351 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3628/7246] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:33 2022] DELETE /identity/v3/policies/9c9a97cc2be44968a9502740c330d2de => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:33.894342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.905150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52241083-b5c6-423c-8584-4edb84434783 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.910791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.911434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.911716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.911993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies/d2b5b3cca5ef49ad9f3510a7815a51fd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.913110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=d2b5b3cca5ef49ad9f3510a7815a51fd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.915113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.931218 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3619/7247] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:33 2022] DELETE /identity/v3/policies/d2b5b3cca5ef49ad9f3510a7815a51fd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:33.935693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:33.946750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009ed74-024f-44dd-b289-33b9d27aca2a tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:33.951465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300893e-3905-4169-a2e9-9eeb17f98937 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f26edde148b94e74a20daef86a4f4aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '498f4d66d23b4800922dff39a28c7d6f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:33.952104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300893e-3905-4169-a2e9-9eeb17f98937 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:33.952380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300893e-3905-4169-a2e9-9eeb17f98937 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:33.952653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300893e-3905-4169-a2e9-9eeb17f98937 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] PATH_INFO: `/v3/policies/b1cf9a6d5c57463aa95dbb574cdb2078` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:33.953774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300893e-3905-4169-a2e9-9eeb17f98937 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=b1cf9a6d5c57463aa95dbb574cdb2078)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:33.955724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300893e-3905-4169-a2e9-9eeb17f98937 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:33.971471 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3629/7248] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:33 2022] DELETE /identity/v3/policies/b1cf9a6d5c57463aa95dbb574cdb2078 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:33.976823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:34.041401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f039aab-883b-481b-8052-1c397a680b66 tempest-PoliciesTestJSON-952990500 tempest-PoliciesTestJSON-952990500-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:34.046148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:34.046785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:34.047067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:34.047340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] PATH_INFO: `/v3/users/c0e9c75950f940b19a34db658bad0430` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:34.048471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e9c75950f940b19a34db658bad0430)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:34.050464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:34.098652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e9c75950f940b19a34db658bad0430'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.111039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:34.112338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e9c75950f940b19a34db658bad0430'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.127546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e9c75950f940b19a34db658bad0430'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.128602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e9c75950f940b19a34db658bad0430'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.129407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e9c75950f940b19a34db658bad0430'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.142494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3620/7249] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:33 2022] DELETE /identity/v3/users/c0e9c75950f940b19a34db658bad0430 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:34.148044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:34.148400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:34.148773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:34.170517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:34.172194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:34.227265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:INSERT=8,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:34.248779 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3630/7250] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:34.254657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:34.256596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:34.339994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfa9d4-d00c-46ec-849e-9074700afb6c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:34.344779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:34.345343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:34.345805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:34.346038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:34.357910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:34.359633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:34.371303 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3621/7251] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:34.684873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:34.696058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:34.710404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a92ee9-8c9f-40e5-b267-48146c0a0101 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:34.727314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:34.728170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:34.728452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:34.728746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] PATH_INFO: `/v3/projects/7732befdc807468393bf15aac300bf45` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:34.729988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7732befdc807468393bf15aac300bf45)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:34.731949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:34.793779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7732befdc807468393bf15aac300bf45', 'request_id': 'req-591757f4-0903-4c3f-8653-a03f55c33476'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.800208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7732befdc807468393bf15aac300bf45', 'request_id': 'req-591757f4-0903-4c3f-8653-a03f55c33476'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.808620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7732befdc807468393bf15aac300bf45', 'request_id': 'req-591757f4-0903-4c3f-8653-a03f55c33476'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.819510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] The token cache is being invalidate because project 7732befdc807468393bf15aac300bf45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:34.819802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.820537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.822212 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3631/7252] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:34 2022] DELETE /identity/v3/projects/7732befdc807468393bf15aac300bf45 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:34.826275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:34.837055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae7c46-2bed-4a19-9a8f-8a542bf1e073 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:34.898353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:34.898990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:34.899267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:34.899542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] PATH_INFO: `/v3/users/f26edde148b94e74a20daef86a4f4aea` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:34.900718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26edde148b94e74a20daef86a4f4aea)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:34.902955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:34.952857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26edde148b94e74a20daef86a4f4aea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.964802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26edde148b94e74a20daef86a4f4aea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.974109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26edde148b94e74a20daef86a4f4aea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.974733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26edde148b94e74a20daef86a4f4aea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.975340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6efd402e-c11b-4999-98b2-26a1c36b9329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26edde148b94e74a20daef86a4f4aea'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:34.984089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:34.989266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3622/7253] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:34 2022] DELETE /identity/v3/users/f26edde148b94e74a20daef86a4f4aea => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:35.279355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:35.280815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:35.295156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591757f4-0903-4c3f-8653-a03f55c33476 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:35.355018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:35.355666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.355976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.356253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] PATH_INFO: `/v3/projects/498f4d66d23b4800922dff39a28c7d6f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.357378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=498f4d66d23b4800922dff39a28c7d6f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:35.359335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:35.408828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '498f4d66d23b4800922dff39a28c7d6f', 'request_id': 'req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:35.415405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '498f4d66d23b4800922dff39a28c7d6f', 'request_id': 'req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:35.424139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '498f4d66d23b4800922dff39a28c7d6f', 'request_id': 'req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:35.435623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] The token cache is being invalidate because project 498f4d66d23b4800922dff39a28c7d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:35.435912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:35.436672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a16a1ab-c98b-4ec8-8261-1c6cc573e1ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:35.438524 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3632/7254] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:35 2022] DELETE /identity/v3/projects/498f4d66d23b4800922dff39a28c7d6f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:35.443786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada062-551f-4287-99a2-62b654161549 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.444079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada062-551f-4287-99a2-62b654161549 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.444427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada062-551f-4287-99a2-62b654161549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.465822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-88ada062-551f-4287-99a2-62b654161549 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:35.467044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-88ada062-551f-4287-99a2-62b654161549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:35.529370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3623/7255] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:35.538013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e26d-9eac-49c1-aba4-a5990eb8e96c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.538334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e26d-9eac-49c1-aba4-a5990eb8e96c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.538604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e26d-9eac-49c1-aba4-a5990eb8e96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.562538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9675e26d-9eac-49c1-aba4-a5990eb8e96c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:35.563754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9675e26d-9eac-49c1-aba4-a5990eb8e96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:35.573006 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3633/7256] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:35.577233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada062-551f-4287-99a2-62b654161549 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:35.587000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada062-551f-4287-99a2-62b654161549 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:35.592047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:35.592692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.593030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.593317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.594001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:35.594299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:35.595889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:35.611303 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3624/7257] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:35.615585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e26d-9eac-49c1-aba4-a5990eb8e96c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:35.620589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:35.628654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e26d-9eac-49c1-aba4-a5990eb8e96c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:35.634906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:35.638328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.638995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.639308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.640016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:35.642217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:35.677246 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3634/7258] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:35 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:35.678423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:35.692836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb668-8848-44f4-ad78-7b4c6dbde1a3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:35.757973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:35.758609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.758834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.759110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.759734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:35.761734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:35.787994 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3625/7259] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:35 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:35.796673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:35.816319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb7a83-ec88-4154-97e4-8aadb12b025e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:35.821813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:35.822466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.822742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.823027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.823861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:35.825755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:35.837414 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3635/7260] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:35.841629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:35.852224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32d0e5-b578-49dc-b723-a11c80752b15 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:35.856880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c262d-b689-48bc-a3da-c475ac120ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:35.857523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c262d-b689-48bc-a3da-c475ac120ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.857805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c262d-b689-48bc-a3da-c475ac120ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.858116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c262d-b689-48bc-a3da-c475ac120ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.858747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c262d-b689-48bc-a3da-c475ac120ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:35.860366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c262d-b689-48bc-a3da-c475ac120ff6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:35.872656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3626/7261] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:35.876887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:35.887343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb867b5a-a825-4b02-965e-f65ab0d1bedb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:35.892008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:35.892663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.892944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.893213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] PATH_INFO: `/v3/projects/e50e41d313324b8a82c339c0a903bf43/users/7fe12a837df74ed397681e077e25db89/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.908079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50e41d313324b8a82c339c0a903bf43, user_id=7fe12a837df74ed397681e077e25db89, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:35.910792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:35.924900 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3636/7262] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:35 2022] PUT /identity/v3/projects/e50e41d313324b8a82c339c0a903bf43/users/7fe12a837df74ed397681e077e25db89/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:35.936567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09788dba-3a9d-4fc0-9a5e-f5b5eb890047 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:35.936886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09788dba-3a9d-4fc0-9a5e-f5b5eb890047 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:35.937123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09788dba-3a9d-4fc0-9a5e-f5b5eb890047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:35.973451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-09788dba-3a9d-4fc0-9a5e-f5b5eb890047 None None] MFA Rules not processed for user `7fe12a837df74ed397681e077e25db89`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:35.974641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-09788dba-3a9d-4fc0-9a5e-f5b5eb890047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:36.029591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:36.039545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3627/7263] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:36.043980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.105981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c76ec-ffd1-47fe-b00b-98dbf415904a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.112547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.113621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.114087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.114541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.115656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.119528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.150557 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3637/7264] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:36 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:36.156711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09788dba-3a9d-4fc0-9a5e-f5b5eb890047 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.170186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09788dba-3a9d-4fc0-9a5e-f5b5eb890047 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.232499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.233080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.233341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.233664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.234288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.236276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.258244 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3628/7265] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:36 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:36.262436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.273313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e77e1-787a-4ad3-b627-5b67f9014477 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.279030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.279629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.279900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.280165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.280806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.282430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.293421 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3638/7266] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:36.297843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.308295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bedfe-a28a-45ed-af95-aa960c01a026 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.313641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.314288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.314563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.314833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] PATH_INFO: `/v3/projects/e582c50f64294673b4c31aae2c9a7e57/users/ea1f148b9b304c97b16e8087e3e1cf3b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.329786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e582c50f64294673b4c31aae2c9a7e57, user_id=ea1f148b9b304c97b16e8087e3e1cf3b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.332445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.346729 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3629/7267] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:36 2022] PUT /identity/v3/projects/e582c50f64294673b4c31aae2c9a7e57/users/ea1f148b9b304c97b16e8087e3e1cf3b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:36.350579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.361113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b1c1d-18cc-4b3d-8136-adcbb9782619 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.420110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63878fd7-fa53-4b77-bb21-57260d15e5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.420722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878fd7-fa53-4b77-bb21-57260d15e5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.421004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878fd7-fa53-4b77-bb21-57260d15e5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.421268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63878fd7-fa53-4b77-bb21-57260d15e5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.421942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63878fd7-fa53-4b77-bb21-57260d15e5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.423567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63878fd7-fa53-4b77-bb21-57260d15e5a4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.433398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3639/7268] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:36.437645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.448714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e90bec-be36-4fc0-a45e-9bbe7c4b0b2d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.453779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.454430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.454711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.454979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] PATH_INFO: `/v3/projects/e582c50f64294673b4c31aae2c9a7e57/users/ea1f148b9b304c97b16e8087e3e1cf3b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.457016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e582c50f64294673b4c31aae2c9a7e57, user_id=ea1f148b9b304c97b16e8087e3e1cf3b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.459731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.470399 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3630/7269] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:36 2022] PUT /identity/v3/projects/e582c50f64294673b4c31aae2c9a7e57/users/ea1f148b9b304c97b16e8087e3e1cf3b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:36.561814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b06bf7-af62-4b53-82f2-384c60239698 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.562081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b06bf7-af62-4b53-82f2-384c60239698 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.562244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b06bf7-af62-4b53-82f2-384c60239698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.591799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c8b06bf7-af62-4b53-82f2-384c60239698 None None] MFA Rules not processed for user `ea1f148b9b304c97b16e8087e3e1cf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:36.593015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8b06bf7-af62-4b53-82f2-384c60239698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:36.597171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:36.667086 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3640/7270] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:36.671293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.682968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea755f-d7e7-473f-a2c3-249c2e9cd364 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.687962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.688685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.688933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.689343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.690102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.691189 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:36.691189 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:19:36.691189 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:19:36.692173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.712431 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3631/7271] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:19:36 2022] POST /identity/v3/regions => generated 224 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:36.716519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b06bf7-af62-4b53-82f2-384c60239698 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.726082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b06bf7-af62-4b53-82f2-384c60239698 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.741176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.741975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.742317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.742594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.743265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.744801 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:36.744801 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:19:36.744801 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:19:36.745584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.767671 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3641/7272] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:19:36 2022] POST /identity/v3/regions => generated 224 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:36.772419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.783492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74202ff-d467-4160-9249-049db1c3d214 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.798595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.799312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.799596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.799869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/55536554-6901-4c32-ba86-146f03607a0b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.800527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:create_region(region_id=55536554-6901-4c32-ba86-146f03607a0b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.802220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.825095 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3632/7273] 10.222.0.26 () {60 vars in 1259 bytes} [Sat Sep 17 21:19:36 2022] PUT /identity/v3/regions/55536554-6901-4c32-ba86-146f03607a0b => generated 231 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:36.829273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.840695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf595608-f4f1-4978-b8c6-d91eafe2f0d7 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.856620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.857274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.857639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.857914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/55536554-6901-4c32-ba86-146f03607a0b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.858492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:delete_region(region_id=55536554-6901-4c32-ba86-146f03607a0b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.860094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.878959 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3642/7274] 10.222.0.26 () {58 vars in 1242 bytes} [Sat Sep 17 21:19:36 2022] DELETE /identity/v3/regions/55536554-6901-4c32-ba86-146f03607a0b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:36.884288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.895968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b4720-a1cd-4806-90d8-02a088568cb8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.913716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.914331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.914605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.914872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.915534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.917150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.939145 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3633/7275] 10.222.0.26 () {60 vars in 1150 bytes} [Sat Sep 17 21:19:36 2022] POST /identity/v3/regions => generated 254 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:36.943417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.955015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbce7f-6972-45a8-8fba-5a925b52ad8c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.971775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.972433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.972692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.973503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/38c8f335707f441e9b2616a3617c3c85` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.973936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:update_region(region_id=38c8f335707f441e9b2616a3617c3c85)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.975024 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:19:36.975024 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:19:36.975024 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:19:36.976241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.001994 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3643/7276] 10.222.0.26 () {60 vars in 1250 bytes} [Sat Sep 17 21:19:36 2022] PATCH /identity/v3/regions/38c8f335707f441e9b2616a3617c3c85 => generated 253 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:37.006173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.019261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cf713-b775-4c59-a83c-24c3bcb58f59 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.041687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.042699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.043046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.043388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/38c8f335707f441e9b2616a3617c3c85` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.044109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:get_region(region_id=38c8f335707f441e9b2616a3617c3c85)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.045256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.051492 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3634/7277] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:19:37 2022] GET /identity/v3/regions/38c8f335707f441e9b2616a3617c3c85 => generated 253 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:37.055699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.066285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696fd60-43a3-4f73-b165-8d70a1a06b69 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.071087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.072165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.072165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.072397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/38c8f335707f441e9b2616a3617c3c85` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.072850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:delete_region(region_id=38c8f335707f441e9b2616a3617c3c85)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.074495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.095019 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3644/7278] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:37 2022] DELETE /identity/v3/regions/38c8f335707f441e9b2616a3617c3c85 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:37.099242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.109613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca578611-ab3f-457f-b879-fca432d2e75f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.125393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.126141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.126517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.126834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.127504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.128655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.131149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:37.139887 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3635/7279] 10.222.0.26 () {58 vars in 1128 bytes} [Sat Sep 17 21:19:37 2022] GET /identity/v3/regions => generated 670 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:37.141444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.153015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9fb76-b5c3-4ec5-a1f0-86230c57e55f tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.158162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.158900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.159253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.159556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/38c8f335707f441e9b2616a3617c3c85` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.160147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:delete_region(region_id=38c8f335707f441e9b2616a3617c3c85)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.161860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Could not find region: 38c8f335707f441e9b2616a3617c3c85.: keystone.exception.RegionNotFound: Could not find region: 38c8f335707f441e9b2616a3617c3c85. Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 38c8f335707f441e9b2616a3617c3c85. Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:37.174688 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 38c8f335707f441e9b2616a3617c3c85. Sep 17 21:19:37.175982 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:37.176988 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3645/7280] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:37 2022] DELETE /identity/v3/regions/38c8f335707f441e9b2616a3617c3c85 => generated 112 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:37.181395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.192519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bff6e6-9935-42b0-8e1f-ebdbd6f2bbbf tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.198739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.199368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.199653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.199930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.200672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.201800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.208042 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3636/7281] 10.222.0.26 () {58 vars in 1128 bytes} [Sat Sep 17 21:19:37 2022] GET /identity/v3/regions => generated 670 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:37.213513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.224651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443833db-bef0-4b2f-b0ac-3aa29374d102 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.230229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.230884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.231269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.231588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.232343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.234194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.256080 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3646/7282] 10.222.0.26 () {60 vars in 1150 bytes} [Sat Sep 17 21:19:37 2022] POST /identity/v3/regions => generated 254 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:37.260283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.270380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864f268-dcb8-4320-99d8-6a50c67cb1e8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.285049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.285785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.286072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.286349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.287024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Adding query filter params (parent_region_id=47fc1d09f5774a0d8974b3461474b9e6) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:37.287332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.288449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.293310 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3637/7283] 10.222.0.26 () {58 vars in 1227 bytes} [Sat Sep 17 21:19:37 2022] GET /identity/v3/regions?parent_region_id=47fc1d09f5774a0d8974b3461474b9e6 => generated 400 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:19:37.297570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.308659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a19ba-974b-49f2-80ea-5b2961922d8e tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.313704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.314334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.314602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.314873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/34fffc9f596b4c8c8f10354044ea84d2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.315457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:delete_region(region_id=34fffc9f596b4c8c8f10354044ea84d2)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.317091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.337897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3647/7284] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:37 2022] DELETE /identity/v3/regions/34fffc9f596b4c8c8f10354044ea84d2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:37.342958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.353177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c74b7a-3650-4cc3-98f9-ec3257737ff3 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.367977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.368940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.369255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.369548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/a0a1ac0e558445d68ddcbf6cce82505e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.370126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:delete_region(region_id=a0a1ac0e558445d68ddcbf6cce82505e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.371731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.392415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3638/7285] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:37 2022] DELETE /identity/v3/regions/a0a1ac0e558445d68ddcbf6cce82505e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:37.396793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.402179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:37.409451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732ae62-8005-42d1-82fa-600ff320150c tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.426323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ee174-e18b-46f0-9487-a96a722aaa44 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea1f148b9b304c97b16e8087e3e1cf3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e582c50f64294673b4c31aae2c9a7e57', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.426948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ee174-e18b-46f0-9487-a96a722aaa44 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.427256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ee174-e18b-46f0-9487-a96a722aaa44 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.427542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ee174-e18b-46f0-9487-a96a722aaa44 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] PATH_INFO: `/v3/regions/47fc1d09f5774a0d8974b3461474b9e6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.428139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ee174-e18b-46f0-9487-a96a722aaa44 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorizing `identity:delete_region(region_id=47fc1d09f5774a0d8974b3461474b9e6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.429892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ee174-e18b-46f0-9487-a96a722aaa44 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.450266 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3648/7286] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:19:37 2022] DELETE /identity/v3/regions/47fc1d09f5774a0d8974b3461474b9e6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:37.454310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.538780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc5b47-975f-4cf5-9f8d-409dc7151ba8 tempest-RegionsTestJSON-980280245 tempest-RegionsTestJSON-980280245-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.554217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.554853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.555131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.555395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] PATH_INFO: `/v3/users/7fe12a837df74ed397681e077e25db89` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.556595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe12a837df74ed397681e077e25db89)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.558615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.609239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe12a837df74ed397681e077e25db89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:37.621675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe12a837df74ed397681e077e25db89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:37.630813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe12a837df74ed397681e077e25db89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:37.631495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe12a837df74ed397681e077e25db89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:37.632269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe12a837df74ed397681e077e25db89'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:37.644456 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3639/7287] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:37 2022] DELETE /identity/v3/users/7fe12a837df74ed397681e077e25db89 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:37.649058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.649424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.649828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.670347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:37.671555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:37.733400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3649/7288] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:37.744121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:37.746040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:37.754920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:37.834687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbe135-f540-4096-ba92-3f362e885747 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:37.838974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:37.839622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:37.839885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:37.840159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:37.850948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:37.852397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:37.862311 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3640/7289] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:38.138268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:38.139710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:38.153378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c403e93-4c6d-4092-95f5-46c3d61611f8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:38.159833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:38.161497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:38.161850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:38.162263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] PATH_INFO: `/v3/projects/e50e41d313324b8a82c339c0a903bf43` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:38.164071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50e41d313324b8a82c339c0a903bf43)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:38.167252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:38.221937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50e41d313324b8a82c339c0a903bf43', 'request_id': 'req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.229081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50e41d313324b8a82c339c0a903bf43', 'request_id': 'req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.238290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50e41d313324b8a82c339c0a903bf43', 'request_id': 'req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.249725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] The token cache is being invalidate because project e50e41d313324b8a82c339c0a903bf43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:38.250050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.250653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.252188 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3650/7290] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:38 2022] DELETE /identity/v3/projects/e50e41d313324b8a82c339c0a903bf43 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:38.256317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:38.267175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2727f-2487-4c1c-a176-bf6a9878b3f1 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:38.329377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:38.330045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:38.330319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:38.330587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] PATH_INFO: `/v3/users/ea1f148b9b304c97b16e8087e3e1cf3b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:38.331702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1f148b9b304c97b16e8087e3e1cf3b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:38.333668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:38.383995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1f148b9b304c97b16e8087e3e1cf3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.395830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1f148b9b304c97b16e8087e3e1cf3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.403844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1f148b9b304c97b16e8087e3e1cf3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.404446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1f148b9b304c97b16e8087e3e1cf3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.405036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-cc7cacfe-dd04-43bd-8cda-ececb1aaef57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1f148b9b304c97b16e8087e3e1cf3b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.417180 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3641/7291] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:38 2022] DELETE /identity/v3/users/ea1f148b9b304c97b16e8087e3e1cf3b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:38.700308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:38.701988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:38.719925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8a671-b5b2-426d-941e-be28bc6ae0b6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:38.773171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:38.780973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:38.781663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:38.781951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:38.782219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] PATH_INFO: `/v3/projects/e582c50f64294673b4c31aae2c9a7e57` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:38.783436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e582c50f64294673b4c31aae2c9a7e57)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:38.785863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:38.834028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e582c50f64294673b4c31aae2c9a7e57', 'request_id': 'req-c7651afd-13ee-4638-9863-15d16203faa7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.840756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e582c50f64294673b4c31aae2c9a7e57', 'request_id': 'req-c7651afd-13ee-4638-9863-15d16203faa7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.851720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e582c50f64294673b4c31aae2c9a7e57', 'request_id': 'req-c7651afd-13ee-4638-9863-15d16203faa7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.863670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] The token cache is being invalidate because project e582c50f64294673b4c31aae2c9a7e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:38.863963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.864761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c7651afd-13ee-4638-9863-15d16203faa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:38.866555 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3651/7292] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:38 2022] DELETE /identity/v3/projects/e582c50f64294673b4c31aae2c9a7e57 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:38.873225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734a672-eb5d-4539-a11e-77eb149a5efd None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:38.873338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734a672-eb5d-4539-a11e-77eb149a5efd None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:38.873604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734a672-eb5d-4539-a11e-77eb149a5efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:38.894209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9734a672-eb5d-4539-a11e-77eb149a5efd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:38.895686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9734a672-eb5d-4539-a11e-77eb149a5efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:38.909104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:38.959702 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3642/7293] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:38.972204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b32b-1872-457f-ae73-e2d3b2622368 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:38.972612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b32b-1872-457f-ae73-e2d3b2622368 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:38.973023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b32b-1872-457f-ae73-e2d3b2622368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:38.999980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bb02b32b-1872-457f-ae73-e2d3b2622368 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:39.001174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bb02b32b-1872-457f-ae73-e2d3b2622368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:39.009917 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3652/7294] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:39.014413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734a672-eb5d-4539-a11e-77eb149a5efd None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.024657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734a672-eb5d-4539-a11e-77eb149a5efd None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.029641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.030239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.030550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.030789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.031475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:39.031790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.033409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.045542 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3643/7295] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:39.050070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02b32b-1872-457f-ae73-e2d3b2622368 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.059944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02b32b-1872-457f-ae73-e2d3b2622368 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.065606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.066211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.066487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.066759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.067393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.069400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.102511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3653/7296] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:39.107172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.117872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f185e8f-b97d-4b9b-a664-46c882fb46d2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.188013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.188637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.188920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.189191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.189863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.191851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.216021 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3644/7297] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:39.220202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.231845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7d0ee-2fea-4b65-9204-2e82c2e92e29 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.238261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.239058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.239345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.239618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.240286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.241981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.255429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3654/7298] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:39.260434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.273092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f62b50-5da2-4841-bb03-ada8a036ea39 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.278551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d618ad-5391-4ea6-88c1-4e4a547009a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.279170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d618ad-5391-4ea6-88c1-4e4a547009a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.279443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d618ad-5391-4ea6-88c1-4e4a547009a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.279836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d618ad-5391-4ea6-88c1-4e4a547009a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.280351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d618ad-5391-4ea6-88c1-4e4a547009a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.281981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d618ad-5391-4ea6-88c1-4e4a547009a8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.295119 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3645/7299] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:39.300001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.311448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de175b0-5589-4834-bf6f-fe70daf693fa admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.316338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.317065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.317358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.317669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] PATH_INFO: `/v3/projects/08516bf7c42948c8810ff77186b017c1/users/27bb5f8bf702429bb718b2b4b62449dd/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.335964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08516bf7c42948c8810ff77186b017c1, user_id=27bb5f8bf702429bb718b2b4b62449dd, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.338916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.356626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3655/7300] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:39 2022] PUT /identity/v3/projects/08516bf7c42948c8810ff77186b017c1/users/27bb5f8bf702429bb718b2b4b62449dd/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:39.366368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15505c0-4994-4654-a9bf-b7c4b5fbb07e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.366657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15505c0-4994-4654-a9bf-b7c4b5fbb07e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.366926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15505c0-4994-4654-a9bf-b7c4b5fbb07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.394908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e15505c0-4994-4654-a9bf-b7c4b5fbb07e None None] MFA Rules not processed for user `27bb5f8bf702429bb718b2b4b62449dd`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:39.396110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e15505c0-4994-4654-a9bf-b7c4b5fbb07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:39.461154 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3646/7301] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:39.466189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.476852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454c3f7-5ecd-4799-ada5-5a4651e2e38d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.481488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.482133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.482409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.482678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.483294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.484795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.505547 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3656/7302] 10.222.0.26 () {60 vars in 1299 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2 => generated 367 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:39.510353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15505c0-4994-4654-a9bf-b7c4b5fbb07e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.519681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15505c0-4994-4654-a9bf-b7c4b5fbb07e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.523920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.524568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.524844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.525117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/d0a5e138b3f54b36bdc4d6e457044c1f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.528593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd, credential_id=d0a5e138b3f54b36bdc4d6e457044c1f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.529049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.542561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:39.548488 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3647/7303] 10.222.0.26 () {58 vars in 1380 bytes} [Sat Sep 17 21:19:39 2022] DELETE /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/d0a5e138b3f54b36bdc4d6e457044c1f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:39.554147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.565249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab325c3-5159-4db9-a6c5-a8b860faacf2 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.570850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.571537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.571812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.572066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.572705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.574401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.596129 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3657/7304] 10.222.0.26 () {60 vars in 1299 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2 => generated 367 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:39.597570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.608732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fca5b-2052-475f-8305-1dbc98219e34 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.613787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.614591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.614979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.615327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/ca184138f3d14cddbc6e2c99df793f6f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.617789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd, credential_id=ca184138f3d14cddbc6e2c99df793f6f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.619709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.639258 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3648/7305] 10.222.0.26 () {58 vars in 1380 bytes} [Sat Sep 17 21:19:39 2022] DELETE /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/ca184138f3d14cddbc6e2c99df793f6f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:39.643708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.654441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc8b4d-95ed-4e80-95cc-6ba6cf2ab5ef tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.659803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca3c79-6beb-4f91-a59f-4d6575f83cda tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.660395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca3c79-6beb-4f91-a59f-4d6575f83cda tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.660671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca3c79-6beb-4f91-a59f-4d6575f83cda tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.660930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca3c79-6beb-4f91-a59f-4d6575f83cda tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/ca184138f3d14cddbc6e2c99df793f6f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-9bca3c79-6beb-4f91-a59f-4d6575f83cda tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Could not find credential: 02a6e8d4ed97a148f7d490fe23edae5ce7357c1dfd33cb2c5db10119f978e91a.: keystone.exception.CredentialNotFound: Could not find credential: 02a6e8d4ed97a148f7d490fe23edae5ce7357c1dfd33cb2c5db10119f978e91a. Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Sep 17 21:19:39.683363 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:19:39.684539 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: 02a6e8d4ed97a148f7d490fe23edae5ce7357c1dfd33cb2c5db10119f978e91a. Sep 17 21:19:39.685941 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:19:39.689705 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3658/7306] 10.222.0.26 () {58 vars in 1377 bytes} [Sat Sep 17 21:19:39 2022] GET /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/ca184138f3d14cddbc6e2c99df793f6f => generated 148 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:19:39.690783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.701045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285741f6-b291-4261-bf78-7fd88fa6c92f tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.705424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.706097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.706369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.706637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.707257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.708947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.727323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3649/7307] 10.222.0.26 () {60 vars in 1299 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2 => generated 367 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:39.731323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca3c79-6beb-4f91-a59f-4d6575f83cda tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.741655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca3c79-6beb-4f91-a59f-4d6575f83cda tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.747256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.747882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.748161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.748769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.749487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.751175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.768615 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3659/7308] 10.222.0.26 () {60 vars in 1299 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2 => generated 367 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:39.772996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.783006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90960017-81e4-4242-9272-8a300ef9a39e tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.787578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.788398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.788676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.788949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.789692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=27bb5f8bf702429bb718b2b4b62449dd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.791380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.805529 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3650/7309] 10.222.0.26 () {58 vars in 1278 bytes} [Sat Sep 17 21:19:39 2022] GET /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2 => generated 865 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:39.808413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.818718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8f7e0-8fce-46b9-b27f-c957fffef3a4 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.823663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.823982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.824093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.824360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/a08a82bc316b47afaadb2b1d691a0576` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.826284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd, credential_id=a08a82bc316b47afaadb2b1d691a0576)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.827700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.857985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3660/7310] 10.222.0.26 () {58 vars in 1380 bytes} [Sat Sep 17 21:19:39 2022] DELETE /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/a08a82bc316b47afaadb2b1d691a0576 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:39.858099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.861174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9508ec-f831-4a46-9fb8-f78c81fc67d0 tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.866096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.866846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.867129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.867407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/086960645c254b9d8ae2fe7e92fac09d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.869867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd, credential_id=086960645c254b9d8ae2fe7e92fac09d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.871440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.891612 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3651/7311] 10.222.0.26 () {58 vars in 1380 bytes} [Sat Sep 17 21:19:39 2022] DELETE /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/086960645c254b9d8ae2fe7e92fac09d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:39.903867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.916390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5f656-9b44-4e9f-a450-f38a42fc2ddc tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.920596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.921437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.921856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.922196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.922899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.924540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.940472 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3661/7312] 10.222.0.26 () {60 vars in 1299 bytes} [Sat Sep 17 21:19:39 2022] POST /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2 => generated 367 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:39.944756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.954735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9425f-159b-49f1-a1a1-88a42218438d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.958873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.959631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.959946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.960220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/35f5dbe5e22f4702b6060e7731d2caab` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.962323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd, credential_id=35f5dbe5e22f4702b6060e7731d2caab)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.963867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:39.966834 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3652/7313] 10.222.0.26 () {58 vars in 1377 bytes} [Sat Sep 17 21:19:39 2022] GET /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/35f5dbe5e22f4702b6060e7731d2caab => generated 367 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:39.970814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:39.977263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=8,keystone:INSERT=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:39.980820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae343611-e0f5-4d5b-be25-ca72e384359d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:39.986324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b182b-e1c5-4903-9c5a-6fe86fd3c69d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27bb5f8bf702429bb718b2b4b62449dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08516bf7c42948c8810ff77186b017c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:39.987074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b182b-e1c5-4903-9c5a-6fe86fd3c69d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:39.987476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b182b-e1c5-4903-9c5a-6fe86fd3c69d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:39.988048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b182b-e1c5-4903-9c5a-6fe86fd3c69d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/35f5dbe5e22f4702b6060e7731d2caab` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:39.990304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b182b-e1c5-4903-9c5a-6fe86fd3c69d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=27bb5f8bf702429bb718b2b4b62449dd, credential_id=35f5dbe5e22f4702b6060e7731d2caab)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:39.991732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b182b-e1c5-4903-9c5a-6fe86fd3c69d tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:40.013503 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3662/7314] 10.222.0.26 () {58 vars in 1380 bytes} [Sat Sep 17 21:19:39 2022] DELETE /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd/credentials/OS-EC2/35f5dbe5e22f4702b6060e7731d2caab => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:40.015911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:40.087856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ed665-35ea-4750-b70d-58f9139e659b tempest-EC2CredentialsTest-1658760261 tempest-EC2CredentialsTest-1658760261-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:40.092770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:40.093416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:40.093724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:40.094001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] PATH_INFO: `/v3/users/27bb5f8bf702429bb718b2b4b62449dd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:40.095143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27bb5f8bf702429bb718b2b4b62449dd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:40.097104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:40.148290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27bb5f8bf702429bb718b2b4b62449dd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:40.160866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27bb5f8bf702429bb718b2b4b62449dd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:40.170251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27bb5f8bf702429bb718b2b4b62449dd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:40.170843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27bb5f8bf702429bb718b2b4b62449dd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:40.171513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27bb5f8bf702429bb718b2b4b62449dd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:40.184738 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3653/7315] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:40 2022] DELETE /identity/v3/users/27bb5f8bf702429bb718b2b4b62449dd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:40.189509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:40.189791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:40.190028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:40.210804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:40.212002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:40.274429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3663/7316] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:40.284616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:40.286202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:40.310070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:40.348776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e2a6-53c5-43b7-bac6-1b36b7a8c2a6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:40.352714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91d6d6-b069-49cc-aea0-607276075f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:40.353333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91d6d6-b069-49cc-aea0-607276075f4e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:40.353649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91d6d6-b069-49cc-aea0-607276075f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:40.353948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91d6d6-b069-49cc-aea0-607276075f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:40.363600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91d6d6-b069-49cc-aea0-607276075f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:40.365119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91d6d6-b069-49cc-aea0-607276075f4e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:40.375602 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3654/7317] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:40.953941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:40.955609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:40.985221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569d44b-66e5-4635-9e96-c4bd0a89cb78 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:40.990737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:40.991539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:40.992011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:40.992892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] PATH_INFO: `/v3/projects/08516bf7c42948c8810ff77186b017c1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:40.994056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08516bf7c42948c8810ff77186b017c1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:40.996065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.056103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08516bf7c42948c8810ff77186b017c1', 'request_id': 'req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:41.065532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08516bf7c42948c8810ff77186b017c1', 'request_id': 'req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:41.075436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08516bf7c42948c8810ff77186b017c1', 'request_id': 'req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:41.089500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] The token cache is being invalidate because project 08516bf7c42948c8810ff77186b017c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:41.090256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:41.091268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3ee7c7c1-13fb-4c78-bab4-9aedb2776d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:41.093533 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3664/7318] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:40 2022] DELETE /identity/v3/projects/08516bf7c42948c8810ff77186b017c1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:41.098564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595da37-bad0-4670-9d4c-a6ebfc0a66d4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.098956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595da37-bad0-4670-9d4c-a6ebfc0a66d4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.099282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595da37-bad0-4670-9d4c-a6ebfc0a66d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.126384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8595da37-bad0-4670-9d4c-a6ebfc0a66d4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:41.128123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8595da37-bad0-4670-9d4c-a6ebfc0a66d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:41.197514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3655/7319] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:41.204177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb56c52-91bb-41d0-9200-4184c7ac105f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.204572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb56c52-91bb-41d0-9200-4184c7ac105f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.204944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb56c52-91bb-41d0-9200-4184c7ac105f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.227484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8bb56c52-91bb-41d0-9200-4184c7ac105f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:41.229072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8bb56c52-91bb-41d0-9200-4184c7ac105f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:41.241838 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3665/7320] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:41.244010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595da37-bad0-4670-9d4c-a6ebfc0a66d4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.255617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595da37-bad0-4670-9d4c-a6ebfc0a66d4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.260894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.261548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.261824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.262093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.262768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:41.263144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.264898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.279469 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3656/7321] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:41.283993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb56c52-91bb-41d0-9200-4184c7ac105f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.294375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb56c52-91bb-41d0-9200-4184c7ac105f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.298990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.299613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.299881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.300154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.300786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.302735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.331499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3666/7322] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:41.336085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.347265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139fd14-542a-44c4-b96b-ba0acc2ab4e5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.408789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.409417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.409728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.410002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.410635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.412590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.436799 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3657/7323] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:41.441667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.453114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c5f8b-a2ef-430f-9dec-38432a2ed233 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.457859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.458465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.458838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.459001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.459811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.461433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.472718 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3667/7324] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:41.476888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.489677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33108e7-ed8e-410f-9a66-53c922c90c46 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.495254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdfb89-a8a9-4299-a10b-d695cda53309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.495904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdfb89-a8a9-4299-a10b-d695cda53309 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.496208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdfb89-a8a9-4299-a10b-d695cda53309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.496481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdfb89-a8a9-4299-a10b-d695cda53309 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.497152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdfb89-a8a9-4299-a10b-d695cda53309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.498858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdfb89-a8a9-4299-a10b-d695cda53309 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.514684 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3658/7325] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:41.515925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.526653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dcab4-2513-44d3-8d8e-378ca8bebf13 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.531272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.531924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.532197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.532464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] PATH_INFO: `/v3/projects/ad472c744b334344b08c0a08f952a429/users/6bb6f1dfc62640f58c0bdac08bafc497/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.547100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad472c744b334344b08c0a08f952a429, user_id=6bb6f1dfc62640f58c0bdac08bafc497, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.549719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.565396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3668/7326] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:41 2022] PUT /identity/v3/projects/ad472c744b334344b08c0a08f952a429/users/6bb6f1dfc62640f58c0bdac08bafc497/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:41.576240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136106-6908-4cae-a1bd-9d319bf3437c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.576511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136106-6908-4cae-a1bd-9d319bf3437c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.576789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136106-6908-4cae-a1bd-9d319bf3437c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.605196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ec136106-6908-4cae-a1bd-9d319bf3437c None None] MFA Rules not processed for user `6bb6f1dfc62640f58c0bdac08bafc497`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:41.606420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ec136106-6908-4cae-a1bd-9d319bf3437c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:41.628922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:41.680819 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3659/7327] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:41.685638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.696133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:41.753280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcae197-fc45-4539-ac8a-2fc09a83a15f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.758948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.759601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.759893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.760158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.760840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.762819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.793417 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3669/7328] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:41.797821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136106-6908-4cae-a1bd-9d319bf3437c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.809007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136106-6908-4cae-a1bd-9d319bf3437c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.868257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.869227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.869513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.869785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.870414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.872358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.895449 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3660/7329] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:41.899566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.910308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346f03-0a9c-453f-8581-52dbdd6ef38e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.915191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.915785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.916057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.916323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.916939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.918594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.929560 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3670/7330] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:41.933427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.943983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12310a-4477-4724-8d16-832a6d6bae87 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.949346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efebd23-94e5-47c9-a1f7-1afc3cc330eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.950006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efebd23-94e5-47c9-a1f7-1afc3cc330eb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.950248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efebd23-94e5-47c9-a1f7-1afc3cc330eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.950520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efebd23-94e5-47c9-a1f7-1afc3cc330eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.951168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efebd23-94e5-47c9-a1f7-1afc3cc330eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.952785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efebd23-94e5-47c9-a1f7-1afc3cc330eb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.967165 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3661/7331] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:41.968554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.979326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043394b-f5c6-4f43-8d03-52385f000310 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.986734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.987478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.987710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.988123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] PATH_INFO: `/v3/projects/64f76453b7d748d4a66438c2e1c3763a/users/5970cae3d5324ed0b7ee1d1b7053154f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.005005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f76453b7d748d4a66438c2e1c3763a, user_id=5970cae3d5324ed0b7ee1d1b7053154f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:42.007953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:42.023446 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3671/7332] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:41 2022] PUT /identity/v3/projects/64f76453b7d748d4a66438c2e1c3763a/users/5970cae3d5324ed0b7ee1d1b7053154f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:42.034481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9354a-163c-48b1-8d52-526d705b9641 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:42.034953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9354a-163c-48b1-8d52-526d705b9641 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:42.035294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9354a-163c-48b1-8d52-526d705b9641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.065698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-19b9354a-163c-48b1-8d52-526d705b9641 None None] MFA Rules not processed for user `5970cae3d5324ed0b7ee1d1b7053154f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:42.066847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-19b9354a-163c-48b1-8d52-526d705b9641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:42.134074 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3662/7333] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:42.140113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:42.209260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fae46-d2c9-48fb-882a-d7f4a17a9d44 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:42.213979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e0dbf-fca7-4442-854e-9530b4db02ce tempest-IdentityV3ProjectsTest-409540974 tempest-IdentityV3ProjectsTest-409540974-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb6f1dfc62640f58c0bdac08bafc497', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad472c744b334344b08c0a08f952a429', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:42.214770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e0dbf-fca7-4442-854e-9530b4db02ce tempest-IdentityV3ProjectsTest-409540974 tempest-IdentityV3ProjectsTest-409540974-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:42.215620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e0dbf-fca7-4442-854e-9530b4db02ce tempest-IdentityV3ProjectsTest-409540974 tempest-IdentityV3ProjectsTest-409540974-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:42.215620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e0dbf-fca7-4442-854e-9530b4db02ce tempest-IdentityV3ProjectsTest-409540974 tempest-IdentityV3ProjectsTest-409540974-project] PATH_INFO: `/v3/users/6bb6f1dfc62640f58c0bdac08bafc497/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.216858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e0dbf-fca7-4442-854e-9530b4db02ce tempest-IdentityV3ProjectsTest-409540974 tempest-IdentityV3ProjectsTest-409540974-project] RBAC: Authorizing `identity:list_user_projects(user_id=6bb6f1dfc62640f58c0bdac08bafc497)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:42.218880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e0dbf-fca7-4442-854e-9530b4db02ce tempest-IdentityV3ProjectsTest-409540974 tempest-IdentityV3ProjectsTest-409540974-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:42.265779 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3672/7334] 10.222.0.26 () {58 vars in 1248 bytes} [Sat Sep 17 21:19:42 2022] GET /identity/v3/users/6bb6f1dfc62640f58c0bdac08bafc497/projects => generated 515 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:42.271108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5153-134c-44e6-a132-0fd10d468ad1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:42.271467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5153-134c-44e6-a132-0fd10d468ad1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:42.271813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5153-134c-44e6-a132-0fd10d468ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.307090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0f4b5153-134c-44e6-a132-0fd10d468ad1 None None] MFA Rules not processed for user `6bb6f1dfc62640f58c0bdac08bafc497`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:42.309027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0f4b5153-134c-44e6-a132-0fd10d468ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:42.321571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3663/7335] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:42.327396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d794b-143f-45f6-aa6c-19470872de72 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:42.327913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d794b-143f-45f6-aa6c-19470872de72 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:42.328033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d794b-143f-45f6-aa6c-19470872de72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.349521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2e9d794b-143f-45f6-aa6c-19470872de72 None None] MFA Rules not processed for user `6bb6f1dfc62640f58c0bdac08bafc497`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:42.350957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2e9d794b-143f-45f6-aa6c-19470872de72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:42.381581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:42.409534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.models.token_model [None req-2e9d794b-143f-45f6-aa6c-19470872de72 None None] User 6bb6f1dfc62640f58c0bdac08bafc497 has no access to project 64f76453b7d748d4a66438c2e1c3763a {{(pid=99167) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Sep 17 21:19:42.413392 ncdv-master-1 devstack@keystone.service[99167]: WARNING keystone.server.flask.application [None req-2e9d794b-143f-45f6-aa6c-19470872de72 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.26: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 17 21:19:42.414877 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3673/7336] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:42 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 90 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 17 21:19:42.420321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5153-134c-44e6-a132-0fd10d468ad1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:42.435634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:42.492575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5153-134c-44e6-a132-0fd10d468ad1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:42.498075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:42.498721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:42.498995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:42.499267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] PATH_INFO: `/v3/users/6bb6f1dfc62640f58c0bdac08bafc497` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.500522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb6f1dfc62640f58c0bdac08bafc497)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:42.502547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:42.559097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb6f1dfc62640f58c0bdac08bafc497'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:42.574233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb6f1dfc62640f58c0bdac08bafc497'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:42.582757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb6f1dfc62640f58c0bdac08bafc497'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:42.583539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb6f1dfc62640f58c0bdac08bafc497'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:42.584398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb6f1dfc62640f58c0bdac08bafc497'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:42.600328 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3664/7337] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:42 2022] DELETE /identity/v3/users/6bb6f1dfc62640f58c0bdac08bafc497 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:42.605280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:42.605765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:42.606329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.628213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:42.629553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:42.701917 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3674/7338] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:42.712118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:42.713554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:42.778153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4aeb5-32e2-48bb-9a49-bf4d4c34081b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:42.782413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:42.783031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:42.783307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:42.783575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:42.792868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:42.794355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:42.803952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3665/7339] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:43.091596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:43.093127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:43.106105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b35e3-8477-4fcf-a889-05bad24df081 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:43.110950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:43.111590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:43.111868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:43.112141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] PATH_INFO: `/v3/projects/ad472c744b334344b08c0a08f952a429` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:43.113267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad472c744b334344b08c0a08f952a429)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:43.115278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:43.167543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad472c744b334344b08c0a08f952a429', 'request_id': 'req-88b42388-1a37-45bc-b406-48977002dd39'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.173941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad472c744b334344b08c0a08f952a429', 'request_id': 'req-88b42388-1a37-45bc-b406-48977002dd39'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.183720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad472c744b334344b08c0a08f952a429', 'request_id': 'req-88b42388-1a37-45bc-b406-48977002dd39'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.194594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] The token cache is being invalidate because project ad472c744b334344b08c0a08f952a429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:43.194877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.195499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.197052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3675/7340] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:43 2022] DELETE /identity/v3/projects/ad472c744b334344b08c0a08f952a429 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:43.202467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:43.218374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba5f5f-3307-4003-8ab9-31a94d556e31 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:43.257821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:43.287226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:43.287912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:43.288193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:43.288466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] PATH_INFO: `/v3/users/5970cae3d5324ed0b7ee1d1b7053154f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:43.289710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5970cae3d5324ed0b7ee1d1b7053154f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:43.291812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:43.351908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5970cae3d5324ed0b7ee1d1b7053154f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.367440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5970cae3d5324ed0b7ee1d1b7053154f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.376935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5970cae3d5324ed0b7ee1d1b7053154f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.377594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5970cae3d5324ed0b7ee1d1b7053154f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.378236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a6b25096-25b2-47d2-b7b4-b9255a2af1ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5970cae3d5324ed0b7ee1d1b7053154f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.392134 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3666/7341] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:43 2022] DELETE /identity/v3/users/5970cae3d5324ed0b7ee1d1b7053154f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:43.670959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:43.672636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:43.700054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b42388-1a37-45bc-b406-48977002dd39 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:43.758959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:43.759631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:43.759905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:43.760345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] PATH_INFO: `/v3/projects/64f76453b7d748d4a66438c2e1c3763a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:43.761512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f76453b7d748d4a66438c2e1c3763a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:43.763464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:43.771997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:43.813284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f76453b7d748d4a66438c2e1c3763a', 'request_id': 'req-5da18de4-0be5-43e7-93c2-988cafa2449b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.819486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f76453b7d748d4a66438c2e1c3763a', 'request_id': 'req-5da18de4-0be5-43e7-93c2-988cafa2449b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.828280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f76453b7d748d4a66438c2e1c3763a', 'request_id': 'req-5da18de4-0be5-43e7-93c2-988cafa2449b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.838887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] The token cache is being invalidate because project 64f76453b7d748d4a66438c2e1c3763a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:43.839170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.839917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5da18de4-0be5-43e7-93c2-988cafa2449b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:43.841609 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3676/7342] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:43 2022] DELETE /identity/v3/projects/64f76453b7d748d4a66438c2e1c3763a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:43.847579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28993f-aa8c-4325-83d9-1da821a274a3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:43.848092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28993f-aa8c-4325-83d9-1da821a274a3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:43.848092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28993f-aa8c-4325-83d9-1da821a274a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:43.869843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7e28993f-aa8c-4325-83d9-1da821a274a3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:43.871034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7e28993f-aa8c-4325-83d9-1da821a274a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:43.935086 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3667/7343] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:43.944345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd81b-5005-4037-8f12-48b299360fb1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:43.944624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd81b-5005-4037-8f12-48b299360fb1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:43.944887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd81b-5005-4037-8f12-48b299360fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:43.967836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-866fd81b-5005-4037-8f12-48b299360fb1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:43.969041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-866fd81b-5005-4037-8f12-48b299360fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:43.978019 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3677/7344] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:43.982401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28993f-aa8c-4325-83d9-1da821a274a3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:43.992697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28993f-aa8c-4325-83d9-1da821a274a3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:43.998997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:43.999612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:43.999887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.000153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.000788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:44.001089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.002739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.017310 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3668/7345] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:44.019899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fd81b-5005-4037-8f12-48b299360fb1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.029794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fd81b-5005-4037-8f12-48b299360fb1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.034902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.035502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.035771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.036039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.036691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.038676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.071005 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3678/7346] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:44 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:44.075296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.086409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e4fe0-e8f9-4775-85c9-5514d847d557 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.127560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:44.146311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.146931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.147210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.147574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.148205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.150213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.172962 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3669/7347] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:44 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:44.177206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.188412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13d33e-0ff8-40f9-b529-89ae0e9246f0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.193596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.194201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.194481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.194764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.195400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.197007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.207307 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3679/7348] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:44.211958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.223054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88199800-b065-483f-a557-93f36340bbb7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.228317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.228975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.229250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.229558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] PATH_INFO: `/v3/projects/0668b395fa5847ba9548495f898c075c/users/d1b38f1fde3c474fab6c80fea05dd153/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.245751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0668b395fa5847ba9548495f898c075c, user_id=d1b38f1fde3c474fab6c80fea05dd153, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.248417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.263395 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3670/7349] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:44 2022] PUT /identity/v3/projects/0668b395fa5847ba9548495f898c075c/users/d1b38f1fde3c474fab6c80fea05dd153/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:44.267635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.277842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88de9f-0739-40a4-be61-2bdcf19f9f76 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.339915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52097518-19f2-4719-8e7d-7084f45c5eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.340722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097518-19f2-4719-8e7d-7084f45c5eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.341062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097518-19f2-4719-8e7d-7084f45c5eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.341453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52097518-19f2-4719-8e7d-7084f45c5eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.342459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52097518-19f2-4719-8e7d-7084f45c5eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.344720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52097518-19f2-4719-8e7d-7084f45c5eb9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.357622 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3680/7350] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:44.361168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.372894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689760de-c799-4cf4-a7b2-8b04d64b5c7b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.380384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.381190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.381740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.382033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] PATH_INFO: `/v3/projects/0668b395fa5847ba9548495f898c075c/users/d1b38f1fde3c474fab6c80fea05dd153/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.388889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0668b395fa5847ba9548495f898c075c, user_id=d1b38f1fde3c474fab6c80fea05dd153, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.395554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.408694 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3671/7351] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:44 2022] PUT /identity/v3/projects/0668b395fa5847ba9548495f898c075c/users/d1b38f1fde3c474fab6c80fea05dd153/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:44.419477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5b1b3-6237-4198-84b3-1c1155a8e222 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.419477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5b1b3-6237-4198-84b3-1c1155a8e222 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.419791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5b1b3-6237-4198-84b3-1c1155a8e222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.449672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-36d5b1b3-6237-4198-84b3-1c1155a8e222 None None] MFA Rules not processed for user `d1b38f1fde3c474fab6c80fea05dd153`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:44.450912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-36d5b1b3-6237-4198-84b3-1c1155a8e222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:44.497541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:44.528400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3681/7352] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:44.533005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.596254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b648a2e-1267-44e2-9dcd-c4e294865ffb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.601242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.601885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.602159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.602454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.603092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.605102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.632822 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3672/7353] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:44 2022] POST /identity/v3/projects => generated 395 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:44.637124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5b1b3-6237-4198-84b3-1c1155a8e222 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.647993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5b1b3-6237-4198-84b3-1c1155a8e222 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.713481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.714162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.714401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.714666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.715490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.717497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.745087 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3682/7354] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:44 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:44.748124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.760170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a0fb4-de6d-4410-b7c2-3788f0b9f9cf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.774016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.774016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.774016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.774016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.774414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.774414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.780760 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3673/7355] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:44.784672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.795854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d132f3-d11b-43ac-adb9-37f42e3d9087 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.800625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c01bd-22f4-46bb-9d55-e7074675695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.801236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c01bd-22f4-46bb-9d55-e7074675695c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.801548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c01bd-22f4-46bb-9d55-e7074675695c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.801938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c01bd-22f4-46bb-9d55-e7074675695c admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.802587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c01bd-22f4-46bb-9d55-e7074675695c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.804379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c01bd-22f4-46bb-9d55-e7074675695c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.818897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3683/7356] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:44.820362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.833825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b1d05-879b-4b22-bac4-34ee57f74e6b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.839313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.840634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.840831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.841111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] PATH_INFO: `/v3/projects/afc0699e86b146e5a7f5a75988fe534f/users/eda2d9d147814a6a935802b7be273eed/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.857805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc0699e86b146e5a7f5a75988fe534f, user_id=eda2d9d147814a6a935802b7be273eed, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.860556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.877078 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3674/7357] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:44 2022] PUT /identity/v3/projects/afc0699e86b146e5a7f5a75988fe534f/users/eda2d9d147814a6a935802b7be273eed/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:44.888157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa313a7-75b4-477c-b7e0-94e6f04a2d65 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.888464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa313a7-75b4-477c-b7e0-94e6f04a2d65 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.888885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa313a7-75b4-477c-b7e0-94e6f04a2d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.918212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3aa313a7-75b4-477c-b7e0-94e6f04a2d65 None None] MFA Rules not processed for user `eda2d9d147814a6a935802b7be273eed`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:44.919489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3aa313a7-75b4-477c-b7e0-94e6f04a2d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:44.990012 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3684/7358] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:45.002481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:45.004188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:45.061057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:45.087011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1447-38ff-441b-9af5-211bf31f5239 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:45.091412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d5d30-4900-4470-9328-876713f7ddba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:45.091963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d5d30-4900-4470-9328-876713f7ddba service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:45.092242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d5d30-4900-4470-9328-876713f7ddba service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:45.092508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d5d30-4900-4470-9328-876713f7ddba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:45.155942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d5d30-4900-4470-9328-876713f7ddba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:45.157402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d5d30-4900-4470-9328-876713f7ddba service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:45.167426 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3675/7359] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:45.587241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa313a7-75b4-477c-b7e0-94e6f04a2d65 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:45.639443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:45.656172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa313a7-75b4-477c-b7e0-94e6f04a2d65 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:45.661697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:45.662368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:45.662641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:45.662945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] PATH_INFO: `/v3/users/d1b38f1fde3c474fab6c80fea05dd153` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:45.664070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b38f1fde3c474fab6c80fea05dd153)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:45.666031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:45.720194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b38f1fde3c474fab6c80fea05dd153'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:45.721238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b38f1fde3c474fab6c80fea05dd153'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:45.732231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b38f1fde3c474fab6c80fea05dd153'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:45.732853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b38f1fde3c474fab6c80fea05dd153'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:45.746218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b38f1fde3c474fab6c80fea05dd153'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:45.760691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3685/7360] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:45 2022] DELETE /identity/v3/users/d1b38f1fde3c474fab6c80fea05dd153 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:45.765450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:45.765784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:45.766060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:45.786365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:45.787649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:45.849721 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3676/7361] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:45.859798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:45.861355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:45.923279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975503b2-fc17-4ed2-996a-452a6596c78c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:45.926994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:45.927595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:45.927889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:45.928165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:45.937419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:45.938896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:45.948508 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3686/7362] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:46.242559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:46.244499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:46.258032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770895c-e9dc-41b0-ba5b-72af3c121786 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:46.264432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:46.265086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:46.265365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:46.265750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] PATH_INFO: `/v3/projects/0668b395fa5847ba9548495f898c075c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:46.266992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0668b395fa5847ba9548495f898c075c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:46.268983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:46.332325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0668b395fa5847ba9548495f898c075c', 'request_id': 'req-f5bbfb1c-8bf2-4659-abe4-85439da08426'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.341186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0668b395fa5847ba9548495f898c075c', 'request_id': 'req-f5bbfb1c-8bf2-4659-abe4-85439da08426'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.348093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0668b395fa5847ba9548495f898c075c', 'request_id': 'req-f5bbfb1c-8bf2-4659-abe4-85439da08426'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.360504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] The token cache is being invalidate because project 0668b395fa5847ba9548495f898c075c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:46.360879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.361652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.363427 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3677/7363] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:46 2022] DELETE /identity/v3/projects/0668b395fa5847ba9548495f898c075c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:46.368079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:46.379573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bd1c-9ead-4adf-baf3-0686bc21832d service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:46.438019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:46.438664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:46.438949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:46.439225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] PATH_INFO: `/v3/users/eda2d9d147814a6a935802b7be273eed` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:46.440348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda2d9d147814a6a935802b7be273eed)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:46.442350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:46.453120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:46.494043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda2d9d147814a6a935802b7be273eed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.494788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda2d9d147814a6a935802b7be273eed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.504399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda2d9d147814a6a935802b7be273eed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.505150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda2d9d147814a6a935802b7be273eed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.518611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f18d7aba-f3f4-4421-bf24-e5ad5a7a9061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda2d9d147814a6a935802b7be273eed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.531069 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3687/7364] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:46 2022] DELETE /identity/v3/users/eda2d9d147814a6a935802b7be273eed => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:46.808518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:46.810244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:46.824852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbfb1c-8bf2-4659-abe4-85439da08426 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:46.830416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:46.889433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:46.889995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:46.890274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:46.890546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] PATH_INFO: `/v3/projects/afc0699e86b146e5a7f5a75988fe534f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:46.891670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc0699e86b146e5a7f5a75988fe534f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:46.893639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:46.947761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc0699e86b146e5a7f5a75988fe534f', 'request_id': 'req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.956928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc0699e86b146e5a7f5a75988fe534f', 'request_id': 'req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.963176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc0699e86b146e5a7f5a75988fe534f', 'request_id': 'req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.976652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] The token cache is being invalidate because project afc0699e86b146e5a7f5a75988fe534f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:46.976943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.977746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e0f5073-fd1c-47c5-9c9f-50e842f85d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:46.979349 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3678/7365] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:46 2022] DELETE /identity/v3/projects/afc0699e86b146e5a7f5a75988fe534f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:46.985216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06d0d-4ec8-442c-99ef-f38c28763ec4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:46.985743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06d0d-4ec8-442c-99ef-f38c28763ec4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:46.985957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06d0d-4ec8-442c-99ef-f38c28763ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.007016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5ac06d0d-4ec8-442c-99ef-f38c28763ec4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:47.008426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5ac06d0d-4ec8-442c-99ef-f38c28763ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:47.069816 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3688/7366] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:47.077498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875fa0c-8d88-402c-a540-44ffdc12c44a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.077804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875fa0c-8d88-402c-a540-44ffdc12c44a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.078073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875fa0c-8d88-402c-a540-44ffdc12c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.099290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0875fa0c-8d88-402c-a540-44ffdc12c44a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:47.100575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0875fa0c-8d88-402c-a540-44ffdc12c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:47.111480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3679/7367] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:47.122948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06d0d-4ec8-442c-99ef-f38c28763ec4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.133396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06d0d-4ec8-442c-99ef-f38c28763ec4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.139054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.139865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.140093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.140359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.141013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:47.141312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.142966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:47.159165 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3689/7368] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:47.160352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875fa0c-8d88-402c-a540-44ffdc12c44a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.171814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875fa0c-8d88-402c-a540-44ffdc12c44a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.176993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.177617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.177898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.178162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.178856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.180773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:47.211464 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3680/7369] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:47 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:47.215822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.226646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c415d2f-d07c-48ca-8300-d82c940c92fe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.287649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.288393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.288669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.288964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.289701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.291733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:47.318146 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3690/7370] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:47 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:47.319412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.330262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe706c59-5acf-43d6-8cd1-5bea26ad9f14 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.334997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.335622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.335899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.336176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.336809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.338457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:47.348737 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3681/7371] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:47.352923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.360398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:47.365603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc489-57e2-4ce5-a4d5-68ceb0a41952 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.374368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79064-5be3-4f9a-a070-eb460cfb10e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.374988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79064-5be3-4f9a-a070-eb460cfb10e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.375316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79064-5be3-4f9a-a070-eb460cfb10e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.375579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79064-5be3-4f9a-a070-eb460cfb10e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.376372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c79064-5be3-4f9a-a070-eb460cfb10e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.378243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c79064-5be3-4f9a-a070-eb460cfb10e7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:47.391506 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3691/7372] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:47.397766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.411149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d505cea-9a3d-4563-b14c-f2e96391cb60 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.416525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.417266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.417707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.418195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] PATH_INFO: `/v3/projects/feaf15c148b64fa6897c239f1ae3afe7/users/00c68bae87f64b4fa58632dc31ada766/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.435023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feaf15c148b64fa6897c239f1ae3afe7, user_id=00c68bae87f64b4fa58632dc31ada766, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.437791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:47.465715 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3682/7373] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:47 2022] PUT /identity/v3/projects/feaf15c148b64fa6897c239f1ae3afe7/users/00c68bae87f64b4fa58632dc31ada766/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:47.468210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820b18d-a134-4636-a87c-8155f88073bc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.468498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820b18d-a134-4636-a87c-8155f88073bc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.468778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820b18d-a134-4636-a87c-8155f88073bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.498061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a820b18d-a134-4636-a87c-8155f88073bc None None] MFA Rules not processed for user `00c68bae87f64b4fa58632dc31ada766`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:47.499349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a820b18d-a134-4636-a87c-8155f88073bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:47.563410 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3692/7374] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:47.575333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.577260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:47.621498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:47.644419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc70517-6732-4fbd-839a-a2ce574880c1 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.649057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a432726-79a9-4684-bd77-752aa52664f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.649737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432726-79a9-4684-bd77-752aa52664f8 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.650053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432726-79a9-4684-bd77-752aa52664f8 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.650328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a432726-79a9-4684-bd77-752aa52664f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.659497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a432726-79a9-4684-bd77-752aa52664f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.660974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a432726-79a9-4684-bd77-752aa52664f8 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:47.670992 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3683/7375] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:47.918315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820b18d-a134-4636-a87c-8155f88073bc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:47.982121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820b18d-a134-4636-a87c-8155f88073bc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:47.987524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:47.988143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:47.988433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:47.988700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] PATH_INFO: `/v3/users/00c68bae87f64b4fa58632dc31ada766` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:47.990045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c68bae87f64b4fa58632dc31ada766)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:47.992115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.045706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c68bae87f64b4fa58632dc31ada766'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.046750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c68bae87f64b4fa58632dc31ada766'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.059838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c68bae87f64b4fa58632dc31ada766'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.060897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c68bae87f64b4fa58632dc31ada766'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.076666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c68bae87f64b4fa58632dc31ada766'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.088660 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3693/7376] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:47 2022] DELETE /identity/v3/users/00c68bae87f64b4fa58632dc31ada766 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:48.093412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.093704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.093974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.114575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:48.115772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.176478 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3684/7377] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:48.186348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.187763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.193229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:48.251916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e7706-3bbf-4a09-a033-d0b94388c075 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:48.256268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9213e1-ead1-40e8-ba9f-eb2868832ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:48.256940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9213e1-ead1-40e8-ba9f-eb2868832ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.257219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9213e1-ead1-40e8-ba9f-eb2868832ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.257486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9213e1-ead1-40e8-ba9f-eb2868832ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.268030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9213e1-ead1-40e8-ba9f-eb2868832ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:48.269505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9213e1-ead1-40e8-ba9f-eb2868832ce7 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.279587 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3694/7378] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:48.560815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.562437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.586364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16520ab7-3887-4615-9e8c-0341b2484428 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:48.598996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:48.599650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.599924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.600191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] PATH_INFO: `/v3/projects/feaf15c148b64fa6897c239f1ae3afe7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.601307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feaf15c148b64fa6897c239f1ae3afe7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:48.603403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.663353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feaf15c148b64fa6897c239f1ae3afe7', 'request_id': 'req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.672928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feaf15c148b64fa6897c239f1ae3afe7', 'request_id': 'req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.681328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feaf15c148b64fa6897c239f1ae3afe7', 'request_id': 'req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.703762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] The token cache is being invalidate because project feaf15c148b64fa6897c239f1ae3afe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:48.704059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.704742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ec054d98-2f6b-4dd4-8cc3-bf12367ecc87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.706646 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3685/7379] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:48 2022] DELETE /identity/v3/projects/feaf15c148b64fa6897c239f1ae3afe7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:48.712074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d74ba-aa0f-4aaf-8896-bc5e564161c7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.712381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d74ba-aa0f-4aaf-8896-bc5e564161c7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.712660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d74ba-aa0f-4aaf-8896-bc5e564161c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.738399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-830d74ba-aa0f-4aaf-8896-bc5e564161c7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:48.740255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-830d74ba-aa0f-4aaf-8896-bc5e564161c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.808659 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3695/7380] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:48.817508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6539df-021a-464d-a21b-e67b4b7f44d5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.817803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6539df-021a-464d-a21b-e67b4b7f44d5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.818076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6539df-021a-464d-a21b-e67b4b7f44d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.838943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5f6539df-021a-464d-a21b-e67b4b7f44d5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:48.840253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f6539df-021a-464d-a21b-e67b4b7f44d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.848712 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3686/7381] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:48.853719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d74ba-aa0f-4aaf-8896-bc5e564161c7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.864431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d74ba-aa0f-4aaf-8896-bc5e564161c7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:48.869901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:48.870545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.870791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.871063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.871748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:48.872048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:48.873683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.889509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3696/7382] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:48.891331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6539df-021a-464d-a21b-e67b4b7f44d5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.901045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6539df-021a-464d-a21b-e67b4b7f44d5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:48.905889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:48.906496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.906766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.907035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.907734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:48.909723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.944901 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3687/7383] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:48 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:48.946181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.956843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501ddbc-519a-46d6-97c5-cd5f24d6199b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.017054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.017848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.017975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.018221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.018855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.020781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.046721 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3697/7384] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:48 2022] POST /identity/v3/users => generated 305 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:49.047918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.059612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d7b89-94b6-40a7-9088-951000604ceb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.065133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.065791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.066065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.066363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.067011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.068655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.081530 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3688/7385] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:49.083957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.095980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61bdb9-bf6a-42af-9bde-b40acc03c5e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.102099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88171476-bc92-4c60-aab1-6c8eb1c7d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.102826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88171476-bc92-4c60-aab1-6c8eb1c7d6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.103102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88171476-bc92-4c60-aab1-6c8eb1c7d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.103363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88171476-bc92-4c60-aab1-6c8eb1c7d6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.104028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88171476-bc92-4c60-aab1-6c8eb1c7d6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.105748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88171476-bc92-4c60-aab1-6c8eb1c7d6d6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.117726 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3698/7386] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:49.122378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.132760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba0a80-5667-439e-875d-d4b13a420616 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.137396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.138099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.138405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.138680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] PATH_INFO: `/v3/projects/ca5ae0c94f964d898892da77caa88497/users/489ca81fe86a45fdab620e32e2b6e3d4/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.153203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5ae0c94f964d898892da77caa88497, user_id=489ca81fe86a45fdab620e32e2b6e3d4, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.155926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.166626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:49.178878 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3689/7387] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:49 2022] PUT /identity/v3/projects/ca5ae0c94f964d898892da77caa88497/users/489ca81fe86a45fdab620e32e2b6e3d4/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:49.183518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3d095-800f-43c0-8974-6c612badf2f9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.183808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3d095-800f-43c0-8974-6c612badf2f9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.184084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3d095-800f-43c0-8974-6c612badf2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.205895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:49.214304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-83b3d095-800f-43c0-8974-6c612badf2f9 None None] MFA Rules not processed for user `489ca81fe86a45fdab620e32e2b6e3d4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:49.216220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-83b3d095-800f-43c0-8974-6c612badf2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:49.289849 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3699/7388] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:49 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:49.294637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.375125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced5f6-bc12-421d-8ce4-f49bbc10a50c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.380075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.380697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.380978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.381257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.381965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.383909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.421393 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3690/7389] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:49 2022] POST /identity/v3/projects => generated 411 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:49.428666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3d095-800f-43c0-8974-6c612badf2f9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.440193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3d095-800f-43c0-8974-6c612badf2f9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.505258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.505909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.506186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.506456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.507104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.509114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.530993 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3700/7390] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:49 2022] POST /identity/v3/users => generated 312 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:49.535157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.545637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0fa5-d68c-4de2-b2b5-275e1c60f47b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.550932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.551537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.551813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.552063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.552700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.554410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.565579 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3691/7391] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:49.569786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.580557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c2882-2b01-4dde-bf75-8b79b1a248d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.586243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.586907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.587183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.587457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] PATH_INFO: `/v3/projects/5db72514afd843c7846d50de222c7583/users/8a434bb8ef3446f3bfd83bf6b8cba9de/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.603297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db72514afd843c7846d50de222c7583, user_id=8a434bb8ef3446f3bfd83bf6b8cba9de, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.606030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.621852 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3701/7392] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:49 2022] PUT /identity/v3/projects/5db72514afd843c7846d50de222c7583/users/8a434bb8ef3446f3bfd83bf6b8cba9de/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:49.625906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.636620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eae878-2e6f-4d2b-aac5-f11055adc1c6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.697165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0c85-0fe3-407f-9295-e91ce12bc659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.697818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0c85-0fe3-407f-9295-e91ce12bc659 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.698122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0c85-0fe3-407f-9295-e91ce12bc659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.698390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0c85-0fe3-407f-9295-e91ce12bc659 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.699030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0c85-0fe3-407f-9295-e91ce12bc659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.700663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0c85-0fe3-407f-9295-e91ce12bc659 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.714381 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3692/7393] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:49.715476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.727023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f53e0-decb-4c0f-9e12-749b8a666c05 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.733764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.734626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.735673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.735950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] PATH_INFO: `/v3/projects/5db72514afd843c7846d50de222c7583/users/8a434bb8ef3446f3bfd83bf6b8cba9de/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.737975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db72514afd843c7846d50de222c7583, user_id=8a434bb8ef3446f3bfd83bf6b8cba9de, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.740752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.754480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3702/7394] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:49 2022] PUT /identity/v3/projects/5db72514afd843c7846d50de222c7583/users/8a434bb8ef3446f3bfd83bf6b8cba9de/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:49.765262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c8e6a-99c8-4d2b-be58-2265965e4a8c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.765582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c8e6a-99c8-4d2b-be58-2265965e4a8c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.765896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c8e6a-99c8-4d2b-be58-2265965e4a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.796791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f64c8e6a-99c8-4d2b-be58-2265965e4a8c None None] MFA Rules not processed for user `8a434bb8ef3446f3bfd83bf6b8cba9de`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:49.798276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f64c8e6a-99c8-4d2b-be58-2265965e4a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:49.859824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:49.872147 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3693/7395] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:49 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:49.882918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:49.884681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:49.947573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634175e6-c35c-4b2f-9c13-649d4a50af7f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.951949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b8ca8-a405-41f1-80a4-0ca798cd0b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.952579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8ca8-a405-41f1-80a4-0ca798cd0b4e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.952842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8ca8-a405-41f1-80a4-0ca798cd0b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.953117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8ca8-a405-41f1-80a4-0ca798cd0b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.960612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:49.963813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b8ca8-a405-41f1-80a4-0ca798cd0b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.965731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b8ca8-a405-41f1-80a4-0ca798cd0b4e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.977939 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3703/7396] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:49 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:51.646217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c8e6a-99c8-4d2b-be58-2265965e4a8c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:51.710017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c8e6a-99c8-4d2b-be58-2265965e4a8c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:51.715800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:51.716722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:51.717150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:51.717588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] PATH_INFO: `/v3/users/489ca81fe86a45fdab620e32e2b6e3d4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:51.719279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=489ca81fe86a45fdab620e32e2b6e3d4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:51.722487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:51.784005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '489ca81fe86a45fdab620e32e2b6e3d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:51.796080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '489ca81fe86a45fdab620e32e2b6e3d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:51.804315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '489ca81fe86a45fdab620e32e2b6e3d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:51.805007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '489ca81fe86a45fdab620e32e2b6e3d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:51.805780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '489ca81fe86a45fdab620e32e2b6e3d4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:51.818717 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3694/7397] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:51 2022] DELETE /identity/v3/users/489ca81fe86a45fdab620e32e2b6e3d4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:51.823453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:51.823947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:51.824045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:51.843475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:51.844677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:51.907197 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3704/7398] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:51.918679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:51.920354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:51.984857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff69ed-0dfd-4e39-a3cb-39bb47874139 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:51.988927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:51.989651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:51.989897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:51.990238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:52.001763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:52.003120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:52.014058 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3695/7399] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:52.317648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:52.322448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:52.338967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968545f0-f0fe-4a38-9efd-62e894f0d306 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:52.345843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:52.346848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:52.347155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:52.347401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] PATH_INFO: `/v3/projects/ca5ae0c94f964d898892da77caa88497` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:52.348642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5ae0c94f964d898892da77caa88497)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:52.350614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:52.409884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5ae0c94f964d898892da77caa88497', 'request_id': 'req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.417114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5ae0c94f964d898892da77caa88497', 'request_id': 'req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.425399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5ae0c94f964d898892da77caa88497', 'request_id': 'req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.436175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] The token cache is being invalidate because project ca5ae0c94f964d898892da77caa88497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:52.436460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.437147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.438705 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3705/7400] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:52 2022] DELETE /identity/v3/projects/ca5ae0c94f964d898892da77caa88497 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:52.442707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:52.452411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb7a39-4bfa-4dde-b5f8-cd09017dc08f service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:52.464916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:52.513264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:52.513912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:52.514194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:52.514462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] PATH_INFO: `/v3/users/8a434bb8ef3446f3bfd83bf6b8cba9de` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:52.515594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a434bb8ef3446f3bfd83bf6b8cba9de)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:52.517553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:52.570390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a434bb8ef3446f3bfd83bf6b8cba9de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.582315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a434bb8ef3446f3bfd83bf6b8cba9de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.590752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a434bb8ef3446f3bfd83bf6b8cba9de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.591357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a434bb8ef3446f3bfd83bf6b8cba9de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.591986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-70e53c01-e46b-4062-87d9-99ebcf168353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a434bb8ef3446f3bfd83bf6b8cba9de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.604571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3696/7401] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:52 2022] DELETE /identity/v3/users/8a434bb8ef3446f3bfd83bf6b8cba9de => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:52.781179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:52.782801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:52.798093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070b5ed-cbc7-45e7-93b8-5d50fd6ab570 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:52.865789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:52.866529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:52.866660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:52.866956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] PATH_INFO: `/v3/projects/5db72514afd843c7846d50de222c7583` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:52.868126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db72514afd843c7846d50de222c7583)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:52.870458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:52.893447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:52.919859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db72514afd843c7846d50de222c7583', 'request_id': 'req-eb4a1538-99fb-4aae-9f94-82e015b65bde'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.926036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db72514afd843c7846d50de222c7583', 'request_id': 'req-eb4a1538-99fb-4aae-9f94-82e015b65bde'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.935340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db72514afd843c7846d50de222c7583', 'request_id': 'req-eb4a1538-99fb-4aae-9f94-82e015b65bde'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.945719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] The token cache is being invalidate because project 5db72514afd843c7846d50de222c7583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:52.945941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.946549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-eb4a1538-99fb-4aae-9f94-82e015b65bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:52.948052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3706/7402] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:52 2022] DELETE /identity/v3/projects/5db72514afd843c7846d50de222c7583 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:52.953150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:52.953656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:52.953797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:52.973930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:52.975167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.036307 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3697/7403] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:53.045338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307068-949e-4d12-b804-d1cb570b725b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.045661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307068-949e-4d12-b804-d1cb570b725b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.045958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307068-949e-4d12-b804-d1cb570b725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.067224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d9307068-949e-4d12-b804-d1cb570b725b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:53.068416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d9307068-949e-4d12-b804-d1cb570b725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.078482 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3707/7404] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:53.082558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9307068-949e-4d12-b804-d1cb570b725b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.082683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.084023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.092173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9307068-949e-4d12-b804-d1cb570b725b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.096313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574a1fd-fbc9-458d-9e65-836af4ae19cc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.097287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.097929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.098200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.098500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.099146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:53.099447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.100971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.101109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.101618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.101895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.102164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] PATH_INFO: `/v3/users/53aa66e4f5134dcf9adb7aa238ba43de` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.103274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53aa66e4f5134dcf9adb7aa238ba43de)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.105219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.112991 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3708/7405] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:53.119093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.129647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc6442-a337-43a6-977c-5c275be055ee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.135216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.135951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.136258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.136529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.137174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.139142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.144859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:53.166832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53aa66e4f5134dcf9adb7aa238ba43de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.175820 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3709/7406] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:53.176712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.183211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53aa66e4f5134dcf9adb7aa238ba43de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.191672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53aa66e4f5134dcf9adb7aa238ba43de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.192333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53aa66e4f5134dcf9adb7aa238ba43de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.192996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c779f3d5-a6ca-4f28-9810-ce05e6fd22c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53aa66e4f5134dcf9adb7aa238ba43de'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.206313 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3698/7407] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:53 2022] DELETE /identity/v3/users/53aa66e4f5134dcf9adb7aa238ba43de => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:53.214748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.215033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.215558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.235794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:53.237008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.245390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216753-0f81-42c1-b4ea-c304ec1ea1d9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.302015 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3699/7408] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:53.308012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.308630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.308907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.309179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.310001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.311759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.312039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.313261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.335932 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3710/7409] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:53.340091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.341714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.349862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:53.353535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc596c-bd70-4368-bc87-eec80aaf0d09 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.359732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.360427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.360713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.360991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.361671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.363701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.376303 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3711/7410] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:53.379946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.380289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79069403-58c9-40b5-9b4f-2d0ac67f4602 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.384500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.385137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.385300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.385620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.390721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304957ff-2400-4fbe-88a9-519f47ea9e8b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.396737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc09047-d273-4b1b-9da5-6e4673bf1dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.397581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc09047-d273-4b1b-9da5-6e4673bf1dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.397868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc09047-d273-4b1b-9da5-6e4673bf1dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.398154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc09047-d273-4b1b-9da5-6e4673bf1dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.399361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc09047-d273-4b1b-9da5-6e4673bf1dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.400660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.400982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc09047-d273-4b1b-9da5-6e4673bf1dc8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.402147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.411923 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3700/7411] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:53.413571 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3712/7412] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:53.416603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.426248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ea472-1405-4729-afcd-df791b53deda service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.431084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.431616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.432083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.432351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] PATH_INFO: `/v3/projects/e5cc4b1067ce429ebbf9aae3db82b7f6/users/0392eca5c9644a71ad290fb49f421ce2/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.448422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5cc4b1067ce429ebbf9aae3db82b7f6, user_id=0392eca5c9644a71ad290fb49f421ce2, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.451220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.466327 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3701/7413] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:53 2022] PUT /identity/v3/projects/e5cc4b1067ce429ebbf9aae3db82b7f6/users/0392eca5c9644a71ad290fb49f421ce2/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:53.477505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd4103-e417-4459-aa62-65891ca722a6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.478143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd4103-e417-4459-aa62-65891ca722a6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.478422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd4103-e417-4459-aa62-65891ca722a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.508379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fcbd4103-e417-4459-aa62-65891ca722a6 None None] MFA Rules not processed for user `0392eca5c9644a71ad290fb49f421ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:53.509756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fcbd4103-e417-4459-aa62-65891ca722a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.577833 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3713/7414] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:53.580759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.611195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:53.647284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9a25-0749-4073-b142-c56dceb5c07d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.652044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.652660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.652941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.653214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.653906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.655879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.690192 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3702/7415] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:53.694616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd4103-e417-4459-aa62-65891ca722a6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.708210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.710983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.711120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd4103-e417-4459-aa62-65891ca722a6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.784721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.785331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.785658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.785966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.786581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.788561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.791747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef91b7-94ee-41d6-a9f9-09df5ee9b15a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.797049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.797720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.797990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.798252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] PATH_INFO: `/v3/projects/c8d2ef6638184f178a760075af39cc8b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.799424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d2ef6638184f178a760075af39cc8b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.801478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.812142 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3714/7416] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:53.816197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.826290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca427c-017f-4e81-a5e5-9aa3c0be9301 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.831191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.831811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.832070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.832337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.832977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.834648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.848007 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3715/7417] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:53.849036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.852032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d2ef6638184f178a760075af39cc8b', 'request_id': 'req-4272113f-ed62-443c-ad95-3163bad7bde2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.858855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5709f-455b-41cb-80be-580bfeba7ace admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.860112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d2ef6638184f178a760075af39cc8b', 'request_id': 'req-4272113f-ed62-443c-ad95-3163bad7bde2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.866269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d2ef6638184f178a760075af39cc8b', 'request_id': 'req-4272113f-ed62-443c-ad95-3163bad7bde2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.878684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] The token cache is being invalidate because project c8d2ef6638184f178a760075af39cc8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:53.879081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.879754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4272113f-ed62-443c-ad95-3163bad7bde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:53.881381 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3703/7418] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:53 2022] DELETE /identity/v3/projects/c8d2ef6638184f178a760075af39cc8b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:53.887610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c27d-286c-4c58-b0d0-04a869d11c99 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.887894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c27d-286c-4c58-b0d0-04a869d11c99 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.888162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c27d-286c-4c58-b0d0-04a869d11c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.907570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-19c0c27d-286c-4c58-b0d0-04a869d11c99 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:53.908782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-19c0c27d-286c-4c58-b0d0-04a869d11c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.909417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:53.920642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.921251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.921554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.921821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.922476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.924033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.934328 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3716/7419] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:53.938552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:53.940282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:53.950760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:53.952990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb008c-b3c4-49ca-b00d-c5cf76aacf7b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:53.958371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add0783-c903-4e1f-a3ab-ff88f0c942f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:53.959099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add0783-c903-4e1f-a3ab-ff88f0c942f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.959375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add0783-c903-4e1f-a3ab-ff88f0c942f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.959662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add0783-c903-4e1f-a3ab-ff88f0c942f8 admin admin] PATH_INFO: `/v3/projects/597d425901bb43f6b0d873cc52963550/users/a3d7e3cc7ec6406f8380d6b9d647cbdd/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.974528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add0783-c903-4e1f-a3ab-ff88f0c942f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=597d425901bb43f6b0d873cc52963550, user_id=a3d7e3cc7ec6406f8380d6b9d647cbdd, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:53.976218 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3704/7420] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:53.977144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add0783-c903-4e1f-a3ab-ff88f0c942f8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:53.985757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3df14-e8de-4389-b614-3392df14a8ed None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:53.986036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3df14-e8de-4389-b614-3392df14a8ed None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:53.986321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3df14-e8de-4389-b614-3392df14a8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:53.992923 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3717/7421] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:53 2022] PUT /identity/v3/projects/597d425901bb43f6b0d873cc52963550/users/a3d7e3cc7ec6406f8380d6b9d647cbdd/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:54.003645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab613-3ed9-4588-be12-c66dd89e66ee None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.003932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab613-3ed9-4588-be12-c66dd89e66ee None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.004203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab613-3ed9-4588-be12-c66dd89e66ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.006629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e8f3df14-e8de-4389-b614-3392df14a8ed None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:54.007813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8f3df14-e8de-4389-b614-3392df14a8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:54.032994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b12ab613-3ed9-4588-be12-c66dd89e66ee None None] MFA Rules not processed for user `a3d7e3cc7ec6406f8380d6b9d647cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:54.034252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b12ab613-3ed9-4588-be12-c66dd89e66ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:54.068960 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3705/7422] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:54.073415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3df14-e8de-4389-b614-3392df14a8ed None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.082911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3df14-e8de-4389-b614-3392df14a8ed None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.087681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.088471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.088704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.088972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.089653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:19:54.089958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.091580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.103255 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3718/7423] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:54.105606 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3706/7424] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:19:54.107555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.107703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ab613-3ed9-4588-be12-c66dd89e66ee None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.117569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f4be1-afa4-46e2-8b90-b4db2df0a96e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.118150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ab613-3ed9-4588-be12-c66dd89e66ee None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.122564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.123140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.123322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.123443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.123667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.124052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.124312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.124409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.124595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.125232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.126250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.127215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.157582 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3707/7425] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:54.157814 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3719/7426] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:54.160007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.160140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.222138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a261fee-9c03-4abd-bdf5-573f19bfcf34 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.222610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964d77d-ecb6-4835-9bde-cc22715e2599 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.227536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.227707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.228212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.228328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.228493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.228606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.228775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.228885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.229408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.229538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.231423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.231552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.256870 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3708/7427] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:54.261095 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3720/7428] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:54.261227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.262060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.271676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d4454-ee44-49af-94cb-96cb2f123b0e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.272082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb12759-9178-40ca-aa3e-bf1fec90dfeb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.276662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.276907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.277250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.277562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.277675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.277826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.277960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.278137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.278567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.278740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.280152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.280374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.290485 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3709/7429] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:54.290732 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3721/7430] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:54.295110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.295239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.305584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcd518-b76d-43c3-b24d-ed70ee3b158d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.305807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a07a-eec3-4a80-a2e6-7557861623db admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.310417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.311075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.311191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.311359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.311633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] PATH_INFO: `/v3/projects/00747b2f48df4523a4754c5f14f261e4/users/721524ee4f6249848cb3a08983969587/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.311756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.311954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.312228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.312869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.314512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.314841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:54.329585 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3710/7431] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:54.329786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00747b2f48df4523a4754c5f14f261e4, user_id=721524ee4f6249848cb3a08983969587, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.331405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.334292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.344928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0657-313c-4059-9a71-c0cb0464e65f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.346283 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3722/7432] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:54 2022] PUT /identity/v3/projects/00747b2f48df4523a4754c5f14f261e4/users/721524ee4f6249848cb3a08983969587/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:54.349446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.359834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:54.410078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc5d0a-2a4a-45bf-91da-6f88fb2b8352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.410740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc5d0a-2a4a-45bf-91da-6f88fb2b8352 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.411017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc5d0a-2a4a-45bf-91da-6f88fb2b8352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.411290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc5d0a-2a4a-45bf-91da-6f88fb2b8352 admin admin] PATH_INFO: `/v3/projects/0608df66a18a4fb3855b6cd557ab2998/users/2fc6b11f473e4231adf4278688619f24/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.415675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6db9b-fa82-4cd4-b649-82ed07b5aa98 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.425395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.426204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.426321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc5d0a-2a4a-45bf-91da-6f88fb2b8352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0608df66a18a4fb3855b6cd557ab2998, user_id=2fc6b11f473e4231adf4278688619f24, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.426503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.426743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.427408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.429009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc5d0a-2a4a-45bf-91da-6f88fb2b8352 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.429126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.443488 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3723/7433] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:19:54.443673 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3711/7434] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:54 2022] PUT /identity/v3/projects/0608df66a18a4fb3855b6cd557ab2998/users/2fc6b11f473e4231adf4278688619f24/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:54.444806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.452020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77340-8b14-4a28-9752-c88b865458bb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.452304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77340-8b14-4a28-9752-c88b865458bb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.452577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77340-8b14-4a28-9752-c88b865458bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.455093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2439c-53e4-4a71-88f3-f82d4b3615a6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.481914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-87a77340-8b14-4a28-9752-c88b865458bb None None] MFA Rules not processed for user `2fc6b11f473e4231adf4278688619f24`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:54.483063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87a77340-8b14-4a28-9752-c88b865458bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:54.515132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a5095-a701-447c-8d8b-2ca40642ddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.515838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5095-a701-447c-8d8b-2ca40642ddc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.516114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5095-a701-447c-8d8b-2ca40642ddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.516379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5095-a701-447c-8d8b-2ca40642ddc9 admin admin] PATH_INFO: `/v3/projects/00747b2f48df4523a4754c5f14f261e4/users/721524ee4f6249848cb3a08983969587/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.518669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a5095-a701-447c-8d8b-2ca40642ddc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00747b2f48df4523a4754c5f14f261e4, user_id=721524ee4f6249848cb3a08983969587, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.521338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a5095-a701-447c-8d8b-2ca40642ddc9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.533198 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3724/7435] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:54 2022] PUT /identity/v3/projects/00747b2f48df4523a4754c5f14f261e4/users/721524ee4f6249848cb3a08983969587/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:54.544685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881203b-6a3c-4359-a2a4-49dcf5b52087 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.544953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881203b-6a3c-4359-a2a4-49dcf5b52087 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.545610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881203b-6a3c-4359-a2a4-49dcf5b52087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.550698 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3712/7436] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:54.556692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a77340-8b14-4a28-9752-c88b865458bb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.568063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a77340-8b14-4a28-9752-c88b865458bb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.576989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a881203b-6a3c-4359-a2a4-49dcf5b52087 None None] MFA Rules not processed for user `721524ee4f6249848cb3a08983969587`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:54.578252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a881203b-6a3c-4359-a2a4-49dcf5b52087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:54.628363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.629091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.629249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.629554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.630196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.632167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.649753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:54.653865 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3725/7437] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:54.662989 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3713/7438] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:19:54.664072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881203b-6a3c-4359-a2a4-49dcf5b52087 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.664200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.665845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:54.674902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881203b-6a3c-4359-a2a4-49dcf5b52087 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.696033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:54.733843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.734386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.734667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.734990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.735620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.737636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.743301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538d27b-e252-4e71-9d92-2d1894fc9b0a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.747925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.748685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.748974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.749254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.751635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:54.761169 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3726/7439] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:19:54.765827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.775939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f23f9-49ba-4ddf-acc4-1614d3bc2dcc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.780741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.781365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.781668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.781937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.782577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.784182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.794309 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3727/7440] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:54.798361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.807883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34befe36-682a-4a2c-9584-51cdbeba3716 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.812625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.813252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.813560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.813831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] PATH_INFO: `/v3/projects/76c02d00830244688d278acc8fd2cecc/users/187f106f895d4af390c9cc8d7353f7af/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.816011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.817483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.826791 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3714/7441] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:19:54.829828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c02d00830244688d278acc8fd2cecc, user_id=187f106f895d4af390c9cc8d7353f7af, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.832622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.847427 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3728/7442] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:54 2022] PUT /identity/v3/projects/76c02d00830244688d278acc8fd2cecc/users/187f106f895d4af390c9cc8d7353f7af/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:54.851755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.862911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b1233-fade-49b9-9d8e-c7b821425c88 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.924405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4b63e-9719-4276-bd0e-b045fc2713b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.925022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b63e-9719-4276-bd0e-b045fc2713b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.925273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b63e-9719-4276-bd0e-b045fc2713b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.925570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4b63e-9719-4276-bd0e-b045fc2713b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.926243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b63e-9719-4276-bd0e-b045fc2713b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.927990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4b63e-9719-4276-bd0e-b045fc2713b3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.938842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3715/7443] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:19:54.943178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:54.953769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea14347-e1dd-4c5f-8e9f-a775ec3ecdce admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:54.958465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:54.959125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.959395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.959679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] PATH_INFO: `/v3/projects/76c02d00830244688d278acc8fd2cecc/users/187f106f895d4af390c9cc8d7353f7af/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:54.961743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c02d00830244688d278acc8fd2cecc, user_id=187f106f895d4af390c9cc8d7353f7af, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:54.964382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:54.974892 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3729/7444] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:19:54 2022] PUT /identity/v3/projects/76c02d00830244688d278acc8fd2cecc/users/187f106f895d4af390c9cc8d7353f7af/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:54.985830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fbfa0-0aab-48b5-8e09-c7631f6c6e2b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:54.986320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fbfa0-0aab-48b5-8e09-c7631f6c6e2b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:54.986320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fbfa0-0aab-48b5-8e09-c7631f6c6e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:55.016482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5d1fbfa0-0aab-48b5-8e09-c7631f6c6e2b None None] MFA Rules not processed for user `187f106f895d4af390c9cc8d7353f7af`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:55.018123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5d1fbfa0-0aab-48b5-8e09-c7631f6c6e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:55.077352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:55.101988 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3716/7445] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:55.113804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:55.115770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:55.178160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737330-05e4-459d-b95a-ec1e3ad5b846 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:55.182184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:55.182809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:55.183359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:55.183661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:55.247533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:55.250459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:55.252564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:55.263695 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3730/7446] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:57.586136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fbfa0-0aab-48b5-8e09-c7631f6c6e2b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:57.599526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fbfa0-0aab-48b5-8e09-c7631f6c6e2b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:57.609029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0964d0a-ba65-4cbd-95b9-fdca1e5b6ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:57.609917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0964d0a-ba65-4cbd-95b9-fdca1e5b6ebb service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:57.610323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0964d0a-ba65-4cbd-95b9-fdca1e5b6ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:57.610694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0964d0a-ba65-4cbd-95b9-fdca1e5b6ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:57.627379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0964d0a-ba65-4cbd-95b9-fdca1e5b6ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:57.629566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0964d0a-ba65-4cbd-95b9-fdca1e5b6ebb service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:57.641540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3717/7447] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:02.320793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:02.343327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80cd11-cca5-46cc-8acf-e7f1caf253c2 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:02.350960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:02.351757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:02.352640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:02.353016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:02.498748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:02.500439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:02.525763 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3731/7448] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:04.646752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5c0ce-113b-42fd-83f6-4dc1cb247491 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:04.647264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5c0ce-113b-42fd-83f6-4dc1cb247491 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:04.647396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5c0ce-113b-42fd-83f6-4dc1cb247491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:04.691329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bef5c0ce-113b-42fd-83f6-4dc1cb247491 None None] MFA Rules not processed for user `2fc6b11f473e4231adf4278688619f24`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:04.693257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bef5c0ce-113b-42fd-83f6-4dc1cb247491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:04.709178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3718/7449] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:08.429197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:08.447565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdaa58-a00b-4b3d-806b-926508f8b5b7 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:08.471585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:08.474176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:08.474802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:08.475844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:08.654308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:08.656706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:08.699702 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3732/7450] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:13.481861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5c0ce-113b-42fd-83f6-4dc1cb247491 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:13.494442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5c0ce-113b-42fd-83f6-4dc1cb247491 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:13.498840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:13.499457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:13.499734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:13.500003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:13.502008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:13.576348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:13.577955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:13.588753 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3719/7451] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:16.922334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:16.925027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:17.056568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c996-7fce-498a-8121-588deb372215 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:17.064573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe314f7c-611d-40c7-95c7-e54dda9621e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:17.065592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe314f7c-611d-40c7-95c7-e54dda9621e2 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:17.066064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe314f7c-611d-40c7-95c7-e54dda9621e2 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:17.066582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe314f7c-611d-40c7-95c7-e54dda9621e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:17.068923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fe314f7c-611d-40c7-95c7-e54dda9621e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:17.088499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe314f7c-611d-40c7-95c7-e54dda9621e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:17.090273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe314f7c-611d-40c7-95c7-e54dda9621e2 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:17.112716 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3733/7452] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:17.617576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:17.629434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5adba-da7b-4b1d-9fec-f3182f730e0a service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:17.635459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:17.636126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:17.636424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:17.636781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] PATH_INFO: `/v3/users/0392eca5c9644a71ad290fb49f421ce2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:17.638010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0392eca5c9644a71ad290fb49f421ce2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:17.640164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:17.698127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0392eca5c9644a71ad290fb49f421ce2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:17.710186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0392eca5c9644a71ad290fb49f421ce2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:17.717959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0392eca5c9644a71ad290fb49f421ce2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:17.718573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0392eca5c9644a71ad290fb49f421ce2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:17.719172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0392eca5c9644a71ad290fb49f421ce2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:17.731377 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3720/7453] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:17 2022] DELETE /identity/v3/users/0392eca5c9644a71ad290fb49f421ce2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:17.735886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:17.736239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:17.736577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:17.763316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:17.765435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:17.770286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:17.845795 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3734/7454] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:17.854601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:17.856023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:17.910111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:17.941855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305a8f6-b26f-4102-803d-14fe5c0525f3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:17.947329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:17.948853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:17.949235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:17.949640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:17.964712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:17.966430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:17.978586 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3721/7455] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:18.180032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:18.181664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:18.198751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf35ffe-be85-489f-810a-1f97f898ead2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:18.203415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:18.204067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:18.204347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:18.204623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] PATH_INFO: `/v3/projects/e5cc4b1067ce429ebbf9aae3db82b7f6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:18.205867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5cc4b1067ce429ebbf9aae3db82b7f6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:18.207796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:18.276148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5cc4b1067ce429ebbf9aae3db82b7f6', 'request_id': 'req-8afac299-7953-48d1-bf97-d35591fb1613'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.282816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5cc4b1067ce429ebbf9aae3db82b7f6', 'request_id': 'req-8afac299-7953-48d1-bf97-d35591fb1613'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.296055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5cc4b1067ce429ebbf9aae3db82b7f6', 'request_id': 'req-8afac299-7953-48d1-bf97-d35591fb1613'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.310184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] The token cache is being invalidate because project e5cc4b1067ce429ebbf9aae3db82b7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:18.310436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.311524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.314222 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3735/7456] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:18 2022] DELETE /identity/v3/projects/e5cc4b1067ce429ebbf9aae3db82b7f6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:18.318030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:18.331233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f886a51-15e0-489c-bc9b-e8af97f976f5 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:18.404258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:18.404894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:18.405174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:18.405438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] PATH_INFO: `/v3/users/a3d7e3cc7ec6406f8380d6b9d647cbdd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:18.406609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d7e3cc7ec6406f8380d6b9d647cbdd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:18.408528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:18.455429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d7e3cc7ec6406f8380d6b9d647cbdd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.468108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d7e3cc7ec6406f8380d6b9d647cbdd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.477703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d7e3cc7ec6406f8380d6b9d647cbdd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.478280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d7e3cc7ec6406f8380d6b9d647cbdd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.478954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d7e3cc7ec6406f8380d6b9d647cbdd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:18.491856 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3722/7457] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:18 2022] DELETE /identity/v3/users/a3d7e3cc7ec6406f8380d6b9d647cbdd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:18.841841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:18.843306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:18.849809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:18.852298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:18.877707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afac299-7953-48d1-bf97-d35591fb1613 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:18.941639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:18.941639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:18.941639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:18.941639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] PATH_INFO: `/v3/users/8ddb1d0a7bca457493d7ac9a58a01e64` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:18.943422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ddb1d0a7bca457493d7ac9a58a01e64)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:18.945522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:18.952188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:18.958097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daceaf-9ec8-4657-9387-58b20cbe8323 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:18.967987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:18.968569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:18.968851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:18.969124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] PATH_INFO: `/v3/projects/597d425901bb43f6b0d873cc52963550` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:18.970355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=597d425901bb43f6b0d873cc52963550)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:18.973811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:18.998514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:19.018073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ddb1d0a7bca457493d7ac9a58a01e64'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.019223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ddb1d0a7bca457493d7ac9a58a01e64'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.031126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ddb1d0a7bca457493d7ac9a58a01e64'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.032702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ddb1d0a7bca457493d7ac9a58a01e64'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.035528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '597d425901bb43f6b0d873cc52963550', 'request_id': 'req-a9698460-9ab3-4bce-8786-6543634d7bcf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.048079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '597d425901bb43f6b0d873cc52963550', 'request_id': 'req-a9698460-9ab3-4bce-8786-6543634d7bcf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.049027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ddb1d0a7bca457493d7ac9a58a01e64'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.059564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '597d425901bb43f6b0d873cc52963550', 'request_id': 'req-a9698460-9ab3-4bce-8786-6543634d7bcf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.071499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] The token cache is being invalidate because project 597d425901bb43f6b0d873cc52963550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:19.071957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.072934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.082660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3723/7459] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:18 2022] DELETE /identity/v3/projects/597d425901bb43f6b0d873cc52963550 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:19.085582 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3736/7459] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:18 2022] DELETE /identity/v3/users/8ddb1d0a7bca457493d7ac9a58a01e64 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:19.087129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:19.094486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:19.170120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43613b31-40aa-49ea-9564-e189056343d8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:19.174867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:19.175504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:19.175806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:19.176075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] PATH_INFO: `/v3/users/721524ee4f6249848cb3a08983969587` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:19.177813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] RBAC: Authorizing `identity:delete_user(user_id=721524ee4f6249848cb3a08983969587)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:19.179847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:19.236255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721524ee4f6249848cb3a08983969587'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.238031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721524ee4f6249848cb3a08983969587'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.253034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721524ee4f6249848cb3a08983969587'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.254014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721524ee4f6249848cb3a08983969587'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.273574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6843ea37-28d6-4a32-9b27-83dcc34db6be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721524ee4f6249848cb3a08983969587'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.292772 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3737/7460] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:19 2022] DELETE /identity/v3/users/721524ee4f6249848cb3a08983969587 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:19.519235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:19.521224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:19.609928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698460-9ab3-4bce-8786-6543634d7bcf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:19.700994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313751d-d575-439a-898e-28991752340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:19.701757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313751d-d575-439a-898e-28991752340c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:19.702131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313751d-d575-439a-898e-28991752340c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:19.702465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313751d-d575-439a-898e-28991752340c admin admin] PATH_INFO: `/v3/projects/720881c6f7304aadbf069698c97dc5c5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:19.703731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313751d-d575-439a-898e-28991752340c admin admin] RBAC: Authorizing `identity:delete_project(project_id=720881c6f7304aadbf069698c97dc5c5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:19.705800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313751d-d575-439a-898e-28991752340c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:19.759423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:19.775108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8313751d-d575-439a-898e-28991752340c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720881c6f7304aadbf069698c97dc5c5', 'request_id': 'req-8313751d-d575-439a-898e-28991752340c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.784788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8313751d-d575-439a-898e-28991752340c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720881c6f7304aadbf069698c97dc5c5', 'request_id': 'req-8313751d-d575-439a-898e-28991752340c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.791963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8313751d-d575-439a-898e-28991752340c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720881c6f7304aadbf069698c97dc5c5', 'request_id': 'req-8313751d-d575-439a-898e-28991752340c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.806044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8313751d-d575-439a-898e-28991752340c admin admin] The token cache is being invalidate because project 720881c6f7304aadbf069698c97dc5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:19.806044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8313751d-d575-439a-898e-28991752340c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.807498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8313751d-d575-439a-898e-28991752340c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:19.809572 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3724/7461] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:19 2022] DELETE /identity/v3/projects/720881c6f7304aadbf069698c97dc5c5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:19.815057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af284d-c844-40b2-829f-cb6533249232 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:19.815347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af284d-c844-40b2-829f-cb6533249232 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:19.815633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af284d-c844-40b2-829f-cb6533249232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:19.836902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-21af284d-c844-40b2-829f-cb6533249232 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:19.838177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21af284d-c844-40b2-829f-cb6533249232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:19.874012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:19.929068 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3738/7462] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:19.938775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af284d-c844-40b2-829f-cb6533249232 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:19.939196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f230a-8423-49af-a2b6-8387c9ab4fe9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:19.939659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f230a-8423-49af-a2b6-8387c9ab4fe9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:19.939743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f230a-8423-49af-a2b6-8387c9ab4fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:19.942338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21af284d-c844-40b2-829f-cb6533249232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:19.965657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af284d-c844-40b2-829f-cb6533249232 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:19.973110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:19.974153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:19.974725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:19.975740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] PATH_INFO: `/v3/projects/00747b2f48df4523a4754c5f14f261e4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:19.977214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=00747b2f48df4523a4754c5f14f261e4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:19.979655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.015633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f00f230a-8423-49af-a2b6-8387c9ab4fe9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:20.018447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f00f230a-8423-49af-a2b6-8387c9ab4fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:20.033246 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3725/7463] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:20.038688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f230a-8423-49af-a2b6-8387c9ab4fe9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.051101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f230a-8423-49af-a2b6-8387c9ab4fe9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.061032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00747b2f48df4523a4754c5f14f261e4', 'request_id': 'req-9d088b18-6a31-4ca4-988d-c50cdc4967ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:20.067353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00747b2f48df4523a4754c5f14f261e4', 'request_id': 'req-9d088b18-6a31-4ca4-988d-c50cdc4967ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:20.077812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00747b2f48df4523a4754c5f14f261e4', 'request_id': 'req-9d088b18-6a31-4ca4-988d-c50cdc4967ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:20.090598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] The token cache is being invalidate because project 00747b2f48df4523a4754c5f14f261e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:20.090891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:20.091568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d088b18-6a31-4ca4-988d-c50cdc4967ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:20.093148 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3739/7464] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:19 2022] DELETE /identity/v3/projects/00747b2f48df4523a4754c5f14f261e4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:20.099530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f9bf0-6b9f-45c3-a9fd-7f3211c25b78 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.099824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f9bf0-6b9f-45c3-a9fd-7f3211c25b78 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.100149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f9bf0-6b9f-45c3-a9fd-7f3211c25b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.124897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-534f9bf0-6b9f-45c3-a9fd-7f3211c25b78 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:20.126165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-534f9bf0-6b9f-45c3-a9fd-7f3211c25b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:20.141713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.142817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.143320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.143830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.144871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:20.145250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.148023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.172587 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3726/7465] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:20.173055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.176571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:20.200650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae32b0-d34f-42c9-a966-fbb2b26cf4d7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.209866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.210494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.211700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.211976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.212626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.215250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.228502 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3740/7466] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:20.237375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae6820-ac1f-48aa-8e63-45b8b1e6bb8c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.238048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae6820-ac1f-48aa-8e63-45b8b1e6bb8c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.238266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae6820-ac1f-48aa-8e63-45b8b1e6bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.253291 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3727/7467] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:20.257644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.267193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-52ae6820-ac1f-48aa-8e63-45b8b1e6bb8c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:20.268552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-52ae6820-ac1f-48aa-8e63-45b8b1e6bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:20.325705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:20.344163 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3741/7468] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:20.345165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae6820-ac1f-48aa-8e63-45b8b1e6bb8c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.354848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc7f90-2010-43ac-9a6e-078336dc76ad admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.360777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae6820-ac1f-48aa-8e63-45b8b1e6bb8c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.364816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.366226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.366896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.367663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.369152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.370561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.371617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.371617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.371961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.372655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:20.373021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.375354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.375494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.400906 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3742/7469] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:20.401951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.414521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f08649-7ed6-43fd-a1c1-488fd3ef40f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.425268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.425956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.426333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.426712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.427466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.430070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.438908 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3728/7470] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:20.443134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.459087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e63dc-aed9-491b-bea7-dd0b72e636ce admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.465669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.466578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.466863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.467233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.467901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.469663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.471027 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3743/7471] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:20.474001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.484876 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3729/7472] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:20.489986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.510217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089c551-f932-4fca-988a-c190ebf0b886 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.516717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:20.554997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9f62-e72f-47da-874f-1fad5f3646a0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.560215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.560844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.561121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.562196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.562927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.564921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.597829 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3744/7473] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:20.601947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.607390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.608021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.608297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.608567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.609264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.610962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.620123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca2213-ad75-4ea2-968e-e937959e50c0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.625045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.625756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.626031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.626377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.627035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.629019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.630092 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3730/7474] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:20.634869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.655063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c542a8-3fdb-48d1-87f0-86f156584db0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.656112 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3745/7475] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:20.666717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.666853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ee1f5-02bf-4c05-ab22-78415b3cc366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.667419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee1f5-02bf-4c05-ab22-78415b3cc366 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.669784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee1f5-02bf-4c05-ab22-78415b3cc366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.670110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ee1f5-02bf-4c05-ab22-78415b3cc366 admin admin] PATH_INFO: `/v3/projects/f2fd0343346a41008083043c91e87aab/users/b3f951e315234c69b0c1294ddac832ef/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.687570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365983a-45a1-4afe-bd22-e4a79e29df01 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.722123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.726124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ee1f5-02bf-4c05-ab22-78415b3cc366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fd0343346a41008083043c91e87aab, user_id=b3f951e315234c69b0c1294ddac832ef, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.726437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.726732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.727014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.728765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.728895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ee1f5-02bf-4c05-ab22-78415b3cc366 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.731711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.754996 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3731/7476] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:20 2022] PUT /identity/v3/projects/f2fd0343346a41008083043c91e87aab/users/b3f951e315234c69b0c1294ddac832ef/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:20.766101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.766464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.766709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.772221 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3746/7477] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:20.776415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.800600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea02a2-bd79-438b-9ce5-75b8dbdfbd17 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:20.829225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] MFA Rules not processed for user `b3f951e315234c69b0c1294ddac832ef`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:20.831452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:20.911419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966d975-7801-4752-972d-04b0aab9ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:20.912220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966d975-7801-4752-972d-04b0aab9ac66 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.912649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966d975-7801-4752-972d-04b0aab9ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.913021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966d975-7801-4752-972d-04b0aab9ac66 admin admin] PATH_INFO: `/v3/projects/e356478356274aacbc41e14a05b7819c/users/90e226926c024e0da6e1bff6c92105b4/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:20.944635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966d975-7801-4752-972d-04b0aab9ac66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e356478356274aacbc41e14a05b7819c, user_id=90e226926c024e0da6e1bff6c92105b4, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:20.947571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966d975-7801-4752-972d-04b0aab9ac66 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:20.955489 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3732/7478] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:20.966651 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3747/7479] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:20 2022] PUT /identity/v3/projects/e356478356274aacbc41e14a05b7819c/users/90e226926c024e0da6e1bff6c92105b4/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:20.968927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:20.971590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:20.979313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42db1-b4e2-4364-9549-0fe071ac44a4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:20.980171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42db1-b4e2-4364-9549-0fe071ac44a4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:20.980171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c42db1-b4e2-4364-9549-0fe071ac44a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.016411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:21.035862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a2c42db1-b4e2-4364-9549-0fe071ac44a4 None None] MFA Rules not processed for user `90e226926c024e0da6e1bff6c92105b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:21.038962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a2c42db1-b4e2-4364-9549-0fe071ac44a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:21.095902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:21.129717 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3748/7480] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:21.133758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42db1-b4e2-4364-9549-0fe071ac44a4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:21.140519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86186eb-9e89-4399-b8d0-0ffd9cad17fc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:21.232579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c42db1-b4e2-4364-9549-0fe071ac44a4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:21.238703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:21.239441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.241013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.241013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.241013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:21.243701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:21.260544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:21.261351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.261753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.262186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.295624 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3749/7481] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:21 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:21.299330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:21.381698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:21.383878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:21.394878 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3733/7482] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:21.414128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed2739-78fe-4160-8aa7-8dde2055e30c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:21.423463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:21.423463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.424491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.426695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.427458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:21.432220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:21.493595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:21.493843 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3750/7483] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:21 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:21.507040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a705-ce09-448a-a49f-e6e6ee8aef89 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:21.520009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:21.520655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.520936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.521217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.521978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:21.530449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:21.552308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3734/7484] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:21.557599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:21.580086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732684f-6edb-4b2d-b88e-d2ec0fcc6601 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:21.600881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:21.601614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.601936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.602217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] PATH_INFO: `/v3/projects/222c5b6a8ae94481aa9ca228914d9518/users/5e4eb93d88bc4dd9acb4ed0809331011/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.622888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:21.631892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222c5b6a8ae94481aa9ca228914d9518, user_id=5e4eb93d88bc4dd9acb4ed0809331011, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:21.640878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:21.701919 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3751/7485] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:21 2022] PUT /identity/v3/projects/222c5b6a8ae94481aa9ca228914d9518/users/5e4eb93d88bc4dd9acb4ed0809331011/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:21.706385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:21.724129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9bce9-d087-4395-8729-8d51397dcce8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:21.836971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccb0a7-f817-4cba-ac35-58b5dd520122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:21.838177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccb0a7-f817-4cba-ac35-58b5dd520122 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.838177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccb0a7-f817-4cba-ac35-58b5dd520122 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.838371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccb0a7-f817-4cba-ac35-58b5dd520122 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.839050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccb0a7-f817-4cba-ac35-58b5dd520122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:21.840988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccb0a7-f817-4cba-ac35-58b5dd520122 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:21.856509 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3735/7486] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:21.857766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:21.872307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae21c6-c7a1-4c42-8c5a-fa4a74adcd40 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:21.877538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:21.878779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.878779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.879020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] PATH_INFO: `/v3/projects/222c5b6a8ae94481aa9ca228914d9518/users/5e4eb93d88bc4dd9acb4ed0809331011/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.881532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] RBAC: Authorizing `identity:create_grant(project_id=222c5b6a8ae94481aa9ca228914d9518, user_id=5e4eb93d88bc4dd9acb4ed0809331011, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:21.884562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:21.901510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3752/7487] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:21 2022] PUT /identity/v3/projects/222c5b6a8ae94481aa9ca228914d9518/users/5e4eb93d88bc4dd9acb4ed0809331011/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:21.909937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299142e3-a13a-4f7e-b18f-b67340ab1d3c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:21.910301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299142e3-a13a-4f7e-b18f-b67340ab1d3c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:21.910691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299142e3-a13a-4f7e-b18f-b67340ab1d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:21.935252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:21.950725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-299142e3-a13a-4f7e-b18f-b67340ab1d3c None None] MFA Rules not processed for user `5e4eb93d88bc4dd9acb4ed0809331011`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:21.952098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-299142e3-a13a-4f7e-b18f-b67340ab1d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:22.083276 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3736/7488] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:22.095883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:22.098237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:22.182383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b0d73-85e7-4f89-bfc6-925b8cf2140b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:22.187977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b8bf8-b2ba-4fef-a35e-4856956b133f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:22.188615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b8bf8-b2ba-4fef-a35e-4856956b133f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:22.188895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b8bf8-b2ba-4fef-a35e-4856956b133f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:22.189183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b8bf8-b2ba-4fef-a35e-4856956b133f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:22.201412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b8bf8-b2ba-4fef-a35e-4856956b133f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:22.202915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b8bf8-b2ba-4fef-a35e-4856956b133f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:22.217009 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3753/7489] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:27.267316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299142e3-a13a-4f7e-b18f-b67340ab1d3c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:27.344431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299142e3-a13a-4f7e-b18f-b67340ab1d3c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:27.354126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:27.354784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:27.355061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:27.360044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] PATH_INFO: `/v3/users/b3f951e315234c69b0c1294ddac832ef` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:27.361843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f951e315234c69b0c1294ddac832ef)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:27.363692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:27.437905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f951e315234c69b0c1294ddac832ef'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:27.457572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f951e315234c69b0c1294ddac832ef'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:27.469541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f951e315234c69b0c1294ddac832ef'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:27.470321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f951e315234c69b0c1294ddac832ef'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:27.471808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f951e315234c69b0c1294ddac832ef'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:27.490852 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3737/7490] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:27 2022] DELETE /identity/v3/users/b3f951e315234c69b0c1294ddac832ef => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:27.495602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:27.496143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:27.496143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:27.517732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:27.518953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:27.599078 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3754/7491] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:27.612157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:27.614266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:27.633871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:27.689587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0ca98-c069-43f9-93fa-23589cd95ff0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:27.693174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:27.694207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:27.694770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:27.695107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:27.705159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:27.706692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:27.718324 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3738/7492] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:28.252943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:28.254961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:28.336566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:28.338681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:28.348657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9d46f-ce71-44cd-9537-d41ff021a418 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:28.356739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60c005-b12c-4ba5-983b-831cdd2a7cfa service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:28.362936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4eb93d88bc4dd9acb4ed0809331011', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '222c5b6a8ae94481aa9ca228914d9518', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:28.364068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.364982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.365672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.367077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:28.367485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:28.369790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:28.374543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:28.375452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.376113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.376596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] PATH_INFO: `/v3/projects/f2fd0343346a41008083043c91e87aab` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.378110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fd0343346a41008083043c91e87aab)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:28.382886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:28.402004 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3755/7493] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:28.403239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:28.425632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd2131-5df9-4703-9c09-d9ac27b9f522 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:28.434515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4eb93d88bc4dd9acb4ed0809331011', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '222c5b6a8ae94481aa9ca228914d9518', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:28.435637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.436719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.437224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.438948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:28.441884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:28.476149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fd0343346a41008083043c91e87aab', 'request_id': 'req-a7d76a8f-e288-430f-aac4-eb87693212bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:28.480991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:28.488339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fd0343346a41008083043c91e87aab', 'request_id': 'req-a7d76a8f-e288-430f-aac4-eb87693212bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:28.499400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3756/7494] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:28 2022] POST /identity/v3/projects => generated 346 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:28.503171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:28.508750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fd0343346a41008083043c91e87aab', 'request_id': 'req-a7d76a8f-e288-430f-aac4-eb87693212bf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:28.525128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] The token cache is being invalidate because project f2fd0343346a41008083043c91e87aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:28.525636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:28.526516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7d76a8f-e288-430f-aac4-eb87693212bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:28.531199 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3739/7495] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:28 2022] DELETE /identity/v3/projects/f2fd0343346a41008083043c91e87aab => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:28.533753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4615ca-99be-4be4-82db-9f71c224bbd7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.534191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4615ca-99be-4be4-82db-9f71c224bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.534691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4615ca-99be-4be4-82db-9f71c224bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.563940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9b4615ca-99be-4be4-82db-9f71c224bbd7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:28.565448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9b4615ca-99be-4be4-82db-9f71c224bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:28.592504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da58c2f-b5bd-404e-88f9-976654602e6f tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:28.599016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9aec7-c11c-4142-9a9c-ab43dfb497b3 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4eb93d88bc4dd9acb4ed0809331011', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '222c5b6a8ae94481aa9ca228914d9518', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:28.599528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9aec7-c11c-4142-9a9c-ab43dfb497b3 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.599940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9aec7-c11c-4142-9a9c-ab43dfb497b3 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.600314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9aec7-c11c-4142-9a9c-ab43dfb497b3 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.601131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9aec7-c11c-4142-9a9c-ab43dfb497b3 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:28.601551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9aec7-c11c-4142-9a9c-ab43dfb497b3 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:28.603574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9aec7-c11c-4142-9a9c-ab43dfb497b3 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:28.623852 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3757/7496] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:28.645405 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3740/7497] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:28.654415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2238ab-4a3a-4bdb-b71a-69a57a291c88 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.654679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2238ab-4a3a-4bdb-b71a-69a57a291c88 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.654861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2238ab-4a3a-4bdb-b71a-69a57a291c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.701660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bb2238ab-4a3a-4bdb-b71a-69a57a291c88 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:28.703043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bb2238ab-4a3a-4bdb-b71a-69a57a291c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:28.726565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4615ca-99be-4be4-82db-9f71c224bbd7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:28.727152 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3758/7498] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:28.737405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4615ca-99be-4be4-82db-9f71c224bbd7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:28.748406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:28.749069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.749367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.749672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.752537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:28.752843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:28.754921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:28.778642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3741/7499] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:28.782749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2238ab-4a3a-4bdb-b71a-69a57a291c88 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:28.794179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2238ab-4a3a-4bdb-b71a-69a57a291c88 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:28.799299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:28.799924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.800201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.800524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.801170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:28.803329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:28.839231 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3759/7500] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:28 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:28.843294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:28.854570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b854d45-694d-4750-878b-5dafbf833f61 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:28.902234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:28.953429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:28.955113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:28.955771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:28.956357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:28.959989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:28.962144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.002499 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3742/7501] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:28 2022] POST /identity/v3/users => generated 301 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:29.006627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.021700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc553-e690-43da-9de1-6fa9a09fc9ee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.032997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.034201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.034493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.034774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.035440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.037931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.052800 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3760/7502] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:29.056620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.067032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9460674-2dc8-4f8e-b951-92134f10d384 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.072161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.072782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.073059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.073330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.074022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.076060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.094464 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3743/7503] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:29.095328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.106759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf506-a973-4487-987f-f9f47150ee57 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.113470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ccfb2-6e6e-48cd-be6d-2dd5ef30a9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.116724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccfb2-6e6e-48cd-be6d-2dd5ef30a9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.117186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccfb2-6e6e-48cd-be6d-2dd5ef30a9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.117338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ccfb2-6e6e-48cd-be6d-2dd5ef30a9b5 admin admin] PATH_INFO: `/v3/projects/56926512f50b40669ecdbcf1e952eea6/users/c9b44775e0f945a9b0c7884ab9536858/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.137562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ccfb2-6e6e-48cd-be6d-2dd5ef30a9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56926512f50b40669ecdbcf1e952eea6, user_id=c9b44775e0f945a9b0c7884ab9536858, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.138567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.139716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ccfb2-6e6e-48cd-be6d-2dd5ef30a9b5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.140503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:29.159034 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3761/7504] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:29 2022] PUT /identity/v3/projects/56926512f50b40669ecdbcf1e952eea6/users/c9b44775e0f945a9b0c7884ab9536858/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:29.169335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4778-eaf9-46a1-977d-010e28119e07 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.169658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4778-eaf9-46a1-977d-010e28119e07 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.170054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4778-eaf9-46a1-977d-010e28119e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.216767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0caf4778-eaf9-46a1-977d-010e28119e07 None None] MFA Rules not processed for user `c9b44775e0f945a9b0c7884ab9536858`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:29.220235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0caf4778-eaf9-46a1-977d-010e28119e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:29.236698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00780a1e-f27b-4364-b334-3916faaf7f2f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.318920 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3762/7505] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:29 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:29.322664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4778-eaf9-46a1-977d-010e28119e07 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.323714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e4eb93d88bc4dd9acb4ed0809331011', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '222c5b6a8ae94481aa9ca228914d9518', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.324380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.324670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.324949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] PATH_INFO: `/v3/projects/141c89a770c1452cb42d28e6932919c2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.326161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorizing `identity:delete_project(project_id=141c89a770c1452cb42d28e6932919c2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.328883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.352412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:29.389657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141c89a770c1452cb42d28e6932919c2', 'request_id': 'req-1154c220-bf89-4cc8-9246-d2f86c4950f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:29.398981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141c89a770c1452cb42d28e6932919c2', 'request_id': 'req-1154c220-bf89-4cc8-9246-d2f86c4950f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:29.400778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:29.407494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141c89a770c1452cb42d28e6932919c2', 'request_id': 'req-1154c220-bf89-4cc8-9246-d2f86c4950f8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:29.417810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4778-eaf9-46a1-977d-010e28119e07 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.424044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] The token cache is being invalidate because project 141c89a770c1452cb42d28e6932919c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:29.424584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:29.425287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:29.427339 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3744/7506] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:29 2022] DELETE /identity/v3/projects/141c89a770c1452cb42d28e6932919c2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:29.438501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.440202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:29.490914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.491465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.491730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.491997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.492951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.495704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.508285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c220-bf89-4cc8-9246-d2f86c4950f8 tempest-RoutersIpV6AdminTest-1555691524 tempest-RoutersIpV6AdminTest-1555691524-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.512693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.513423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.513710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.514679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.517025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:29.531476 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3763/7507] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:29 2022] POST /identity/v3/projects => generated 403 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:29.532918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.535370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:29.590400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.592321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.601577 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3745/7508] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:29.613738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1611fce-f474-49e8-9e33-647034cc5625 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.619077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.619697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.619980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.620281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.620947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.622960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.651129 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3764/7509] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:29 2022] POST /identity/v3/users => generated 308 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:29.653530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.668454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0595e8-5f87-4c45-b857-37c64efd7ad4 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.680947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.681874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.682444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.683044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.683833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.687342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.708752 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3746/7510] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:29.712997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.728851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26f83e-b890-4c0a-be7f-22cf2f20bef5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.740963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.742022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.742372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.742709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] PATH_INFO: `/v3/projects/2f3afd57d23d4e31a687eba14e520af5/users/1c476453cc1644a1993e0a91ff23b64a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.770296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3afd57d23d4e31a687eba14e520af5, user_id=1c476453cc1644a1993e0a91ff23b64a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.773106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.807015 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3765/7511] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:29 2022] PUT /identity/v3/projects/2f3afd57d23d4e31a687eba14e520af5/users/1c476453cc1644a1993e0a91ff23b64a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:29.813019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.824867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c7ace-1cd7-4a0b-bd64-cfdced0497d5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.899632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe22080-974d-44f2-ace9-99e8b10991c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.900515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22080-974d-44f2-ace9-99e8b10991c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.900860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22080-974d-44f2-ace9-99e8b10991c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.901319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22080-974d-44f2-ace9-99e8b10991c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.902180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe22080-974d-44f2-ace9-99e8b10991c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.903906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe22080-974d-44f2-ace9-99e8b10991c8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.939028 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3747/7512] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:29.942997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:29.961271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23797b5-0766-4fb9-9bc3-f9f40d6512b9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:29.980568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:29.981816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:29.982102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:29.982607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] PATH_INFO: `/v3/projects/2f3afd57d23d4e31a687eba14e520af5/users/1c476453cc1644a1993e0a91ff23b64a/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:29.984901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3afd57d23d4e31a687eba14e520af5, user_id=1c476453cc1644a1993e0a91ff23b64a, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:29.988524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:29.997735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:30.014445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3766/7513] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:29 2022] PUT /identity/v3/projects/2f3afd57d23d4e31a687eba14e520af5/users/1c476453cc1644a1993e0a91ff23b64a/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:30.026287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9070d-9739-4def-9248-c3a429babb6a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:30.026572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9070d-9739-4def-9248-c3a429babb6a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:30.026846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9070d-9739-4def-9248-c3a429babb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:30.065579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e8e9070d-9739-4def-9248-c3a429babb6a None None] MFA Rules not processed for user `1c476453cc1644a1993e0a91ff23b64a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:30.066983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8e9070d-9739-4def-9248-c3a429babb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:30.085642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:30.202301 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3748/7514] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:30.308480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:30.324793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a306f-7f19-4139-89a4-cae32444ce50 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:30.333016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c476453cc1644a1993e0a91ff23b64a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f3afd57d23d4e31a687eba14e520af5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:30.333920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:30.334317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:30.335902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:30.336710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:30.339392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:30.376573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9070d-9739-4def-9248-c3a429babb6a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:30.376770 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3767/7515] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:30 2022] POST /identity/v3/roles => generated 240 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:30.407900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9070d-9739-4def-9248-c3a429babb6a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:30.415564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c476453cc1644a1993e0a91ff23b64a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f3afd57d23d4e31a687eba14e520af5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:30.416838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:30.417101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:30.417495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] PATH_INFO: `/v3/projects/56926512f50b40669ecdbcf1e952eea6/users/c9b44775e0f945a9b0c7884ab9536858/roles/e64322b39293468388dd9cd320f96634` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:30.420938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: Authorizing `identity:create_grant(project_id=56926512f50b40669ecdbcf1e952eea6, user_id=c9b44775e0f945a9b0c7884ab9536858, role_id=e64322b39293468388dd9cd320f96634)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:30.424601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:30.474070 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3749/7516] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:30 2022] PUT /identity/v3/projects/56926512f50b40669ecdbcf1e952eea6/users/c9b44775e0f945a9b0c7884ab9536858/roles/e64322b39293468388dd9cd320f96634 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:30.475812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:30.484042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:30.582382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:30.585034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:30.607792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c28b5-4d53-4934-8f8a-cb4aae2de414 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:30.700179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdc59a-d19a-4321-afc4-80c2c6bfc767 tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:30.705214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:30.705949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:30.706230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:30.706343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9e763-2bd7-4216-a25b-8264a1f1f554 tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b44775e0f945a9b0c7884ab9536858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56926512f50b40669ecdbcf1e952eea6', 'project_domain_id': 'default', 'roles': ['tempest-test_role-1125676626', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:30.706522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] PATH_INFO: `/v3/users/7ff5b886208a44c0891c2d1ba65dc083` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:30.706906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e763-2bd7-4216-a25b-8264a1f1f554 tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:30.707192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e763-2bd7-4216-a25b-8264a1f1f554 tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:30.707464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e763-2bd7-4216-a25b-8264a1f1f554 tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] PATH_INFO: `/v3/users/c9b44775e0f945a9b0c7884ab9536858/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:30.707750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff5b886208a44c0891c2d1ba65dc083)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:30.708167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9e763-2bd7-4216-a25b-8264a1f1f554 tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] RBAC: Authorizing `identity:create_application_credential(user_id=c9b44775e0f945a9b0c7884ab9536858)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:30.709698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9e763-2bd7-4216-a25b-8264a1f1f554 tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:30.709938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:30.768014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff5b886208a44c0891c2d1ba65dc083'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:30.785521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff5b886208a44c0891c2d1ba65dc083'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:30.794911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff5b886208a44c0891c2d1ba65dc083'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:30.795625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff5b886208a44c0891c2d1ba65dc083'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:30.796256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff5b886208a44c0891c2d1ba65dc083'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:30.803754 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3768/7517] 10.222.0.26 () {60 vars in 1315 bytes} [Sat Sep 17 21:20:30 2022] POST /identity/v3/users/c9b44775e0f945a9b0c7884ab9536858/application_credentials => generated 659 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:30.809178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:30.809497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:30.809788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:30.825645 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3750/7518] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:30 2022] DELETE /identity/v3/users/7ff5b886208a44c0891c2d1ba65dc083 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:30.839440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] MFA Rules not processed for user `c9b44775e0f945a9b0c7884ab9536858`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:30.840723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:30.853698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:30.981186 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3769/7519] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:30 2022] POST /identity/v3/auth/tokens => generated 3394 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 406 bytes (2 switches on core 0) Sep 17 21:20:30.985494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:30.987740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.019760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.021241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.022757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:31.088942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22fa7e-8808-4115-bc46-a1a4eb3aa4d4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.096192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9b44775e0f945a9b0c7884ab9536858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56926512f50b40669ecdbcf1e952eea6', 'project_domain_id': 'default', 'roles': ['tempest-test_role-1125676626', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.096948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.097283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.097923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] PATH_INFO: `/v3/users/c9b44775e0f945a9b0c7884ab9536858/application_credentials/2adf94b5130d45e69ccca79a6ea5d37b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.099588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] RBAC: Authorizing `identity:delete_application_credential(user_id=c9b44775e0f945a9b0c7884ab9536858, application_credential_id=2adf94b5130d45e69ccca79a6ea5d37b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.101366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.115500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b77692-f913-4bfb-a364-2c4140c0f9f3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.122140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.122881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.123277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.123550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] PATH_INFO: `/v3/projects/8d3fab3adf0c425680c91af8afcc516f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.124828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d3fab3adf0c425680c91af8afcc516f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.126836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.137319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3751/7520] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:30 2022] DELETE /identity/v3/users/c9b44775e0f945a9b0c7884ab9536858/application_credentials/2adf94b5130d45e69ccca79a6ea5d37b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:31.138547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.140566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.186669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d3fab3adf0c425680c91af8afcc516f', 'request_id': 'req-0961b325-09ff-4777-836d-4f93df9ce840'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.193865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d3fab3adf0c425680c91af8afcc516f', 'request_id': 'req-0961b325-09ff-4777-836d-4f93df9ce840'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.203599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d3fab3adf0c425680c91af8afcc516f', 'request_id': 'req-0961b325-09ff-4777-836d-4f93df9ce840'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.212578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:31.213017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb13b1f-a606-4a53-acb3-18b976a7875a tempest-ApplicationCredentialsV3AdminTest-973608471 tempest-ApplicationCredentialsV3AdminTest-973608471-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.221800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] The token cache is being invalidate because project 8d3fab3adf0c425680c91af8afcc516f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:31.222166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.223026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0961b325-09ff-4777-836d-4f93df9ce840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.230445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3770/7521] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:31 2022] DELETE /identity/v3/projects/8d3fab3adf0c425680c91af8afcc516f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:31.232765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50312f-1245-4e7e-8791-a79484f70a70 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.233433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50312f-1245-4e7e-8791-a79484f70a70 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.234532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50312f-1245-4e7e-8791-a79484f70a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.261537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7c50312f-1245-4e7e-8791-a79484f70a70 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:31.262919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7c50312f-1245-4e7e-8791-a79484f70a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.277089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c476453cc1644a1993e0a91ff23b64a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f3afd57d23d4e31a687eba14e520af5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.277928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.278209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.278486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] PATH_INFO: `/v3/roles/e64322b39293468388dd9cd320f96634` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.279921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e64322b39293468388dd9cd320f96634)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.281607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.327276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Invalidating the token cache because role e64322b39293468388dd9cd320f96634 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:31.327750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.328499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.330754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3752/7522] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:31 2022] DELETE /identity/v3/roles/e64322b39293468388dd9cd320f96634 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:31.336315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.337673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.339396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3771/7523] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:31.347357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaadb1-eae9-458b-813e-41aa1a042acd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.347643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaadb1-eae9-458b-813e-41aa1a042acd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.347921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaadb1-eae9-458b-813e-41aa1a042acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.358222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:31.371981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-08eaadb1-eae9-458b-813e-41aa1a042acd None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:31.373538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-08eaadb1-eae9-458b-813e-41aa1a042acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.414198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36078f3a-33fe-41f6-9fa4-7ab38668dfae tempest-ApplicationCredentialsV3AdminTest-1313165824 tempest-ApplicationCredentialsV3AdminTest-1313165824-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.420013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.420657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.420940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.421223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] PATH_INFO: `/v3/users/c9b44775e0f945a9b0c7884ab9536858` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.422395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b44775e0f945a9b0c7884ab9536858)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.424460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.438976 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3772/7524] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:31.443166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eaadb1-eae9-458b-813e-41aa1a042acd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.453956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eaadb1-eae9-458b-813e-41aa1a042acd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.459491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.460321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.460570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.460855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.461749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:31.462143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.463984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.481064 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3773/7525] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:31.482702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.494325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b44775e0f945a9b0c7884ab9536858'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.495575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189150-b7c5-4560-84e2-8f45d1c1191d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.502624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.507903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.508237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.509672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.510752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.513178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.516270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b44775e0f945a9b0c7884ab9536858'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.526276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b44775e0f945a9b0c7884ab9536858'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.527684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b44775e0f945a9b0c7884ab9536858'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.528347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-997ba285-ed5b-4d01-a5ef-02c869b3cfed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b44775e0f945a9b0c7884ab9536858'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:31.543822 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3753/7526] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:31 2022] DELETE /identity/v3/users/c9b44775e0f945a9b0c7884ab9536858 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:31.549766 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3774/7527] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:31 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:31.549930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.549930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.549930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.551108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.552925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.569182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:31.570443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.630401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d339b7c-37ef-4699-b55b-ee4273a02b7e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.637187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.637839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.638227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.638501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.639198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.641581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.645212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3754/7528] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:31.651708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.653107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.653537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:31.669428 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3775/7529] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:31 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:31.673413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.691252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77774936-caef-4a99-8615-9a113eeb1add admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.698817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.699828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.700117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.700385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.701034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.702667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.708003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:31.715204 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3776/7530] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:31.719024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.733744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcc7e5-32b9-47e9-b5aa-daf795be8e12 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.741545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c4aa6-464b-4fc2-ab0b-c0f913e5b073 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.746008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.746736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.747186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.747627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.760067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65c9df-e3fa-4ffd-98c8-5c0eecb63af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.760756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65c9df-e3fa-4ffd-98c8-5c0eecb63af6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.760900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65c9df-e3fa-4ffd-98c8-5c0eecb63af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.761111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65c9df-e3fa-4ffd-98c8-5c0eecb63af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.761873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65c9df-e3fa-4ffd-98c8-5c0eecb63af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.763423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.763548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65c9df-e3fa-4ffd-98c8-5c0eecb63af6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.765432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.776580 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3755/7531] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:31.781522 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3777/7532] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:31.784693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:31.795222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dff4d-97c8-4b44-897f-c551e31ebd42 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:31.800135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:31.800806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.801089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.801366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] PATH_INFO: `/v3/projects/8a4cb7f9fdb349b9be017068291a5c23/users/3203ea74578c474dab0e959b4c3cc76b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.820794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4cb7f9fdb349b9be017068291a5c23, user_id=3203ea74578c474dab0e959b4c3cc76b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:31.823986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:31.841637 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3756/7533] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:31 2022] PUT /identity/v3/projects/8a4cb7f9fdb349b9be017068291a5c23/users/3203ea74578c474dab0e959b4c3cc76b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:31.852222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a00864-fb7c-4cb4-929b-3f9bbaea5b1d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:31.852509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a00864-fb7c-4cb4-929b-3f9bbaea5b1d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:31.853153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a00864-fb7c-4cb4-929b-3f9bbaea5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:31.887813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a5a00864-fb7c-4cb4-929b-3f9bbaea5b1d None None] MFA Rules not processed for user `3203ea74578c474dab0e959b4c3cc76b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:31.889045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a5a00864-fb7c-4cb4-929b-3f9bbaea5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:31.958114 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3778/7534] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:31.965507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.049339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1a325-29f1-43c0-bc6e-2882eee70f11 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.057403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.058335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.058617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.058898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.059531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.061487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.098976 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3757/7535] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:31 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:32.101962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a00864-fb7c-4cb4-929b-3f9bbaea5b1d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.113605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a00864-fb7c-4cb4-929b-3f9bbaea5b1d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.174162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.174782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.175055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.176113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.176777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.179036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.209003 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3779/7536] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:32 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:32.214584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.223329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.225349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:32.226085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892237e8-c4d1-4fc5-b1f6-4cd21c564467 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.230824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.231547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.231900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.232241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.232958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.234693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.245144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b31c-1def-437d-a653-2f4e9b76b409 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.246801 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3758/7537] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:32.250945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.251106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.251692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.251984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.252259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] PATH_INFO: `/v3/projects/56926512f50b40669ecdbcf1e952eea6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.253430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=56926512f50b40669ecdbcf1e952eea6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.255405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.263332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d4365-ec29-4d04-9163-afd1308a60fb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.272726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.273575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.273974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.274348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] PATH_INFO: `/v3/projects/7cb5947bc0be4a2db8d18701756b5f3c/users/b840456466464658b73495e9861845f8/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.291368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb5947bc0be4a2db8d18701756b5f3c, user_id=b840456466464658b73495e9861845f8, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.294445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.310458 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3759/7538] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:32 2022] PUT /identity/v3/projects/7cb5947bc0be4a2db8d18701756b5f3c/users/b840456466464658b73495e9861845f8/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:32.314300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.315834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=4,keystone:SELECT=93,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:32.319689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56926512f50b40669ecdbcf1e952eea6', 'request_id': 'req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.330314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56926512f50b40669ecdbcf1e952eea6', 'request_id': 'req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.340657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56926512f50b40669ecdbcf1e952eea6', 'request_id': 'req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.354072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] The token cache is being invalidate because project 56926512f50b40669ecdbcf1e952eea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:32.354481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.355219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.360413 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3780/7539] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:32 2022] DELETE /identity/v3/projects/56926512f50b40669ecdbcf1e952eea6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:32.361591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.362888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:32.390858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a910475-5353-4bb4-b004-3376b0862d1c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.396896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.397538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.397823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.398096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.398747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.400370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.403600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:32.414545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3760/7540] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:32.418207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.420724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:32.438607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8d514-6b14-4ad6-b954-d5c456c45a6a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.440415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064eb83-adb4-4cc6-bc7c-f5a1201606ae admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.443738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081fc9e-1ff5-4210-b109-93271ed3533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.444461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081fc9e-1ff5-4210-b109-93271ed3533b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.444747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081fc9e-1ff5-4210-b109-93271ed3533b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.445033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081fc9e-1ff5-4210-b109-93271ed3533b admin admin] PATH_INFO: `/v3/projects/7cb5947bc0be4a2db8d18701756b5f3c/users/b840456466464658b73495e9861845f8/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.445802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.446518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.446859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.447014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081fc9e-1ff5-4210-b109-93271ed3533b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb5947bc0be4a2db8d18701756b5f3c, user_id=b840456466464658b73495e9861845f8, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.447246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] PATH_INFO: `/v3/users/1c476453cc1644a1993e0a91ff23b64a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.448454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c476453cc1644a1993e0a91ff23b64a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.449700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081fc9e-1ff5-4210-b109-93271ed3533b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.450566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.462185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3761/7541] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:32 2022] PUT /identity/v3/projects/7cb5947bc0be4a2db8d18701756b5f3c/users/b840456466464658b73495e9861845f8/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:32.473294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021518fe-78c2-4c5a-9a1d-efc37835a0ca None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.473604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021518fe-78c2-4c5a-9a1d-efc37835a0ca None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.473891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021518fe-78c2-4c5a-9a1d-efc37835a0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.507426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-021518fe-78c2-4c5a-9a1d-efc37835a0ca None None] MFA Rules not processed for user `b840456466464658b73495e9861845f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:32.508723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c476453cc1644a1993e0a91ff23b64a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.508866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-021518fe-78c2-4c5a-9a1d-efc37835a0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:32.509453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c476453cc1644a1993e0a91ff23b64a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.518610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c476453cc1644a1993e0a91ff23b64a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.519984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c476453cc1644a1993e0a91ff23b64a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.536175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c476453cc1644a1993e0a91ff23b64a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:32.553498 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3781/7542] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:32 2022] DELETE /identity/v3/users/1c476453cc1644a1993e0a91ff23b64a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:32.590878 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3762/7543] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:32.594639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.663507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:32.683739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752069b-9a42-433d-8c26-335af98bd39e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.703398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.704115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.704465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.704798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.705871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.707554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.746911 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3782/7544] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:32 2022] POST /identity/v3/projects => generated 359 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:32.747869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021518fe-78c2-4c5a-9a1d-efc37835a0ca None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.760146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021518fe-78c2-4c5a-9a1d-efc37835a0ca None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.849006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.849670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.849993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.850268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.850934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.853274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.891309 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3763/7545] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:32 2022] POST /identity/v3/users => generated 416 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:32.895577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.897880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.898017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:32.914407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d84932-7e88-4811-b436-ac154d5aa6b8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.920365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.922565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.923822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.924205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/users/7a9274b301284ef29c625a13c0dad7b8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.942004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_user(user_id=7a9274b301284ef29c625a13c0dad7b8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.944389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:32.946511 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3764/7546] 10.222.0.26 () {58 vars in 1221 bytes} [Sat Sep 17 21:20:32 2022] GET /identity/v3/users/7a9274b301284ef29c625a13c0dad7b8 => generated 416 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:32.950510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:32.969873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41567a-c19f-4b10-90d6-d99a63aedb11 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:32.975714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:32.977065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:32.977282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:32.977595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/users/7a9274b301284ef29c625a13c0dad7b8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:32.978824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_user(user_id=7a9274b301284ef29c625a13c0dad7b8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:32.980833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.002629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99e8ec-d2e0-48b1-a457-a3c135c5c234 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.009658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.010367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.010627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.010889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] PATH_INFO: `/v3/projects/2f3afd57d23d4e31a687eba14e520af5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.012094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3afd57d23d4e31a687eba14e520af5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.014438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.026587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:33.035471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a9274b301284ef29c625a13c0dad7b8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.050537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a9274b301284ef29c625a13c0dad7b8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.062982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a9274b301284ef29c625a13c0dad7b8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.065292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a9274b301284ef29c625a13c0dad7b8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.066568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a9274b301284ef29c625a13c0dad7b8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.073977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3afd57d23d4e31a687eba14e520af5', 'request_id': 'req-a5066502-1474-440e-ad82-90a02e075e1a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.082272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3afd57d23d4e31a687eba14e520af5', 'request_id': 'req-a5066502-1474-440e-ad82-90a02e075e1a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.089075 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3765/7547] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:32 2022] DELETE /identity/v3/users/7a9274b301284ef29c625a13c0dad7b8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:33.092312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3afd57d23d4e31a687eba14e520af5', 'request_id': 'req-a5066502-1474-440e-ad82-90a02e075e1a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.093541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.095453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:33.104849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] The token cache is being invalidate because project 2f3afd57d23d4e31a687eba14e520af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:33.105138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.105939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a5066502-1474-440e-ad82-90a02e075e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.108263 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3783/7548] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:32 2022] DELETE /identity/v3/projects/2f3afd57d23d4e31a687eba14e520af5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:33.113314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df55538-57ce-425a-a633-cb8a9eb0263d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.113674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df55538-57ce-425a-a633-cb8a9eb0263d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.114019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df55538-57ce-425a-a633-cb8a9eb0263d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.136073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3df55538-57ce-425a-a633-cb8a9eb0263d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:33.137529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3df55538-57ce-425a-a633-cb8a9eb0263d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:33.173364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0accd69-7575-468a-91c6-640af9fcb767 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.184586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.185785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.186478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.186817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/ce35c27821554451b42e6d4cb47841ea` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.188289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ce35c27821554451b42e6d4cb47841ea)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.190366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.201137 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3784/7549] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:33.208485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4e3f3-7b64-482b-8b0c-62429a2ff07a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.208763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4e3f3-7b64-482b-8b0c-62429a2ff07a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.209044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4e3f3-7b64-482b-8b0c-62429a2ff07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.232518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8cb4e3f3-7b64-482b-8b0c-62429a2ff07a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:33.234025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8cb4e3f3-7b64-482b-8b0c-62429a2ff07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:33.238494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:33.248271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce35c27821554451b42e6d4cb47841ea', 'request_id': 'req-129209e1-f84f-4afd-99b1-de8a7a83cb7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.257788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce35c27821554451b42e6d4cb47841ea', 'request_id': 'req-129209e1-f84f-4afd-99b1-de8a7a83cb7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.266515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce35c27821554451b42e6d4cb47841ea', 'request_id': 'req-129209e1-f84f-4afd-99b1-de8a7a83cb7d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.280771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project ce35c27821554451b42e6d4cb47841ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:33.281254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.282244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.285540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3766/7550] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:33 2022] DELETE /identity/v3/projects/ce35c27821554451b42e6d4cb47841ea => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:33.288821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.290955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:33.311588 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3785/7551] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:33.315732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4e3f3-7b64-482b-8b0c-62429a2ff07a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.325167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4e3f3-7b64-482b-8b0c-62429a2ff07a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.330940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.331092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.331306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.331574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.332226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:33.332527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.334795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.346228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:33.348140 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3786/7552] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:33.353706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.367057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8086b6-0238-4435-ae74-bd797ea3c2a8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.370642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129209e1-f84f-4afd-99b1-de8a7a83cb7d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.372289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.372899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.373172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.373599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.374259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.376177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.381432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.382761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.383045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.383319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.383972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.386319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.407966 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3767/7553] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:33 2022] POST /identity/v3/projects => generated 348 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:33.410738 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3787/7554] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:33 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:33.412100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.414826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.486531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b29f5d-20c7-4b7a-8b31-0f0153bb01d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.487175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba24ae-81a7-4ba4-b11a-376a3cf89b24 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.491881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.493500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.493793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.494068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.494745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.496873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.522445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3788/7555] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:33 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:33.526475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.538247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4704e-cb91-42dc-a810-3a61ddfd6520 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.542995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.543647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.543875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.544148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.544920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.546562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.557499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3789/7556] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:33.559331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.560054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.560531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.561003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.561142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.561857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:33.562163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.564182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.571744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b17c62-1bd3-4f8a-8ff1-44ae72ae7d92 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.576451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.577117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.577453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.577813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.578502 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3768/7557] 10.222.0.26 () {58 vars in 1160 bytes} [Sat Sep 17 21:20:33 2022] GET /identity/v3/projects?is_domain=True => generated 1020 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:33.578662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.580380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.582406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.592400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3790/7558] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:33.595943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19cdf-5ca6-4a84-967d-eb338c96c1b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.597261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.601722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.602458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.602847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.603103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.603945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.606076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.613741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e85d5-fb4f-42fa-ab90-35cb55c8d99a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.619953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a354060-d507-4fe8-8e63-294635ca3553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.620647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354060-d507-4fe8-8e63-294635ca3553 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.624340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354060-d507-4fe8-8e63-294635ca3553 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.624500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3769/7559] 10.222.0.26 () {58 vars in 1128 bytes} [Sat Sep 17 21:20:33 2022] GET /identity/v3/domains => generated 829 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:33.624626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354060-d507-4fe8-8e63-294635ca3553 admin admin] PATH_INFO: `/v3/projects/aa45eb07c2c548dda3a2ea32379a7cc7/users/da0b8af7a5364d3385e761fc2b4117fb/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.625439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.637553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4289e13-952e-45db-a01e-74ebcd75d4b9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.639924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a354060-d507-4fe8-8e63-294635ca3553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa45eb07c2c548dda3a2ea32379a7cc7, user_id=da0b8af7a5364d3385e761fc2b4117fb, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.643057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a354060-d507-4fe8-8e63-294635ca3553 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.643848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.645091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.645547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.645932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/5db255d5fce24beebea7bd6f0a34e021` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.647420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:update_project(project_id=5db255d5fce24beebea7bd6f0a34e021)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.648867 ncdv-master-1 devstack@keystone.service[99166]: WARNING py.warnings [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:20:33.648867 ncdv-master-1 devstack@keystone.service[99166]: warnings.warn(msg) Sep 17 21:20:33.648867 ncdv-master-1 devstack@keystone.service[99166]: Sep 17 21:20:33.650244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.660302 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3791/7560] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:33 2022] PUT /identity/v3/projects/aa45eb07c2c548dda3a2ea32379a7cc7/users/da0b8af7a5364d3385e761fc2b4117fb/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:33.671153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f7ce5-2481-40c2-95f6-03503bee9db3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.671153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f7ce5-2481-40c2-95f6-03503bee9db3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.671868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f7ce5-2481-40c2-95f6-03503bee9db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.678922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.679802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.704703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fd8f7ce5-2481-40c2-95f6-03503bee9db3 None None] MFA Rules not processed for user `da0b8af7a5364d3385e761fc2b4117fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:33.706024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd8f7ce5-2481-40c2-95f6-03503bee9db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:33.712128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:33.716128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.716877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.719904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3770/7561] 10.222.0.26 () {60 vars in 1252 bytes} [Sat Sep 17 21:20:33 2022] PATCH /identity/v3/projects/5db255d5fce24beebea7bd6f0a34e021 => generated 362 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:33.724008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.725540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:33.776281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:33.789888 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3792/7562] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:33.796941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f7ce5-2481-40c2-95f6-03503bee9db3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.809982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f7ce5-2481-40c2-95f6-03503bee9db3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.812538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9d4f0-6787-477c-8c86-1641200c17ab tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.815527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.816319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.816607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.816887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.818157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:create_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.818612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:33.819457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:33.819728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:33.820001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/5db255d5fce24beebea7bd6f0a34e021` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:33.820318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.833627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5db255d5fce24beebea7bd6f0a34e021)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:33.835432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:33.872817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.887406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.923312 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3793/7563] 10.222.0.26 () {60 vars in 1314 bytes} [Sat Sep 17 21:20:33 2022] POST /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 789 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:33.926441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:33.941086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021', 'request_id': 'req-8663e35c-e439-49cf-b25e-191c54d66265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.944645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def98b-1b44-4556-8088-f36522aac420 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:33.952206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021', 'request_id': 'req-8663e35c-e439-49cf-b25e-191c54d66265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.959753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021', 'request_id': 'req-8663e35c-e439-49cf-b25e-191c54d66265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.975043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 5db255d5fce24beebea7bd6f0a34e021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:33.975607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.976383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.986998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021', 'request_id': 'req-8663e35c-e439-49cf-b25e-191c54d66265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.991422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021', 'request_id': 'req-8663e35c-e439-49cf-b25e-191c54d66265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.992237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '5db255d5fce24beebea7bd6f0a34e021', 'request_id': 'req-8663e35c-e439-49cf-b25e-191c54d66265'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:33.999473 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3771/7564] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:33 2022] DELETE /identity/v3/projects/5db255d5fce24beebea7bd6f0a34e021 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:34.005075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.007222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:34.010794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7865c-11dc-4953-978b-ed3b31b3930b tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.011454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7865c-11dc-4953-978b-ed3b31b3930b tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.011738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7865c-11dc-4953-978b-ed3b31b3930b tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.012071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7865c-11dc-4953-978b-ed3b31b3930b tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/f24fb816251a490682210ed314cd0ea1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.021973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7865c-11dc-4953-978b-ed3b31b3930b tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:get_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=f24fb816251a490682210ed314cd0ea1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.023895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7865c-11dc-4953-978b-ed3b31b3930b tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.028579 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3794/7565] 10.222.0.26 () {58 vars in 1392 bytes} [Sat Sep 17 21:20:33 2022] GET /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/f24fb816251a490682210ed314cd0ea1 => generated 689 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:34.032635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.033022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.033425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.054835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] MFA Rules not processed for user `da0b8af7a5364d3385e761fc2b4117fb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:34.057126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:34.094306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663e35c-e439-49cf-b25e-191c54d66265 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.105271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:34.108301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.109637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.110178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.110612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.111837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.114308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.153599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:34.173803 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3772/7566] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:34 2022] POST /identity/v3/projects => generated 359 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:34.175914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.229636 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3795/7567] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:34 2022] POST /identity/v3/auth/tokens => generated 3425 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 406 bytes (2 switches on core 0) Sep 17 21:20:34.232346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.235193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:34.260203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74356b4b-031c-43ce-b87e-460e4b1b9c39 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.265507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.266178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.266464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.266763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/785b1106fdb14f9b9a9d7db4a190eacc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.268016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=785b1106fdb14f9b9a9d7db4a190eacc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.270283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.272633 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3773/7568] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:34 2022] GET /identity/v3/projects/785b1106fdb14f9b9a9d7db4a190eacc => generated 359 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:34.276632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.287497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3624d9-a9c7-4dea-86ba-c2558caa4e53 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.292483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.293239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.293635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.293986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/785b1106fdb14f9b9a9d7db4a190eacc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.295241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=785b1106fdb14f9b9a9d7db4a190eacc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.297679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.310647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6efbf5-c0d2-4028-8988-84595bc025f7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.315953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.317053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.317562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.318016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/f24fb816251a490682210ed314cd0ea1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.319423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=f24fb816251a490682210ed314cd0ea1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.321133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.345596 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3796/7569] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:34 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/f24fb816251a490682210ed314cd0ea1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:34.350866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.354550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785b1106fdb14f9b9a9d7db4a190eacc', 'request_id': 'req-439a0e29-2b6a-4739-91a9-d90c54cf3951'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.363351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95829-a0c7-4e64-8d31-541e7deddf75 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.365046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785b1106fdb14f9b9a9d7db4a190eacc', 'request_id': 'req-439a0e29-2b6a-4739-91a9-d90c54cf3951'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.372659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785b1106fdb14f9b9a9d7db4a190eacc', 'request_id': 'req-439a0e29-2b6a-4739-91a9-d90c54cf3951'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.390125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 785b1106fdb14f9b9a9d7db4a190eacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:34.390563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.391430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.393927 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3774/7570] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:34 2022] DELETE /identity/v3/projects/785b1106fdb14f9b9a9d7db4a190eacc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:34.403020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.406063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:34.430471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.431122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.431403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.431670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.432338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:create_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.433520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.471682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:34.486329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a0e29-2b6a-4739-91a9-d90c54cf3951 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.493770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.494598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.495001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.495402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.496149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.498421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.526092 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3797/7571] 10.222.0.26 () {60 vars in 1315 bytes} [Sat Sep 17 21:20:34 2022] POST /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 921 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:34.530606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.532397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:34.536786 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3775/7572] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:34 2022] POST /identity/v3/projects => generated 360 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:34.541400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.544292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885ebc-04a1-4722-9a2f-60de7bf62276 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.555600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:34.609435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.610120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.610556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.610829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/3132c6ae5d504805a4b5c2ce0e5e5db4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.617576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8236ac-8c2e-4cd5-bc30-74d71a91094b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.622899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.623639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.624018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.624424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/0a02a49aec19453da696b58a4863b72a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.624925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=3132c6ae5d504805a4b5c2ce0e5e5db4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.625810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=0a02a49aec19453da696b58a4863b72a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.626609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.628113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.630235 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3776/7573] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:34 2022] GET /identity/v3/projects/0a02a49aec19453da696b58a4863b72a => generated 360 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:34.634201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.645518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cf930-23af-43ec-9bc3-c72f554ab4aa tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.650656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.651318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.651601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.651966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/0a02a49aec19453da696b58a4863b72a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.653041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0a02a49aec19453da696b58a4863b72a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.655119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.657052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3798/7574] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:34 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/3132c6ae5d504805a4b5c2ce0e5e5db4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:34.662142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.671985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56f872-de66-496d-b336-69f637502caf tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.679289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.679996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.680289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.680568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.681251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:create_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.682453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.727943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a02a49aec19453da696b58a4863b72a', 'request_id': 'req-115ffafe-8af7-45a2-9379-84a89e13ff61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.739651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a02a49aec19453da696b58a4863b72a', 'request_id': 'req-115ffafe-8af7-45a2-9379-84a89e13ff61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.750877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a02a49aec19453da696b58a4863b72a', 'request_id': 'req-115ffafe-8af7-45a2-9379-84a89e13ff61'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.767490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 0a02a49aec19453da696b58a4863b72a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:34.767923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.770543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:34.772738 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3777/7575] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:34 2022] DELETE /identity/v3/projects/0a02a49aec19453da696b58a4863b72a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:34.776385 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3799/7576] 10.222.0.26 () {60 vars in 1315 bytes} [Sat Sep 17 21:20:34 2022] POST /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 813 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:34.777606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.778679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.780419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:34.780694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:34.792934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db5366-dd1d-4d83-acea-24045196b806 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.826078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:34.851524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.852203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.852489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.852773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/940bd0781295418abf4736e82f5b961c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.858355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ffafe-8af7-45a2-9379-84a89e13ff61 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.862186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=940bd0781295418abf4736e82f5b961c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.863753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.869399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.871089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.871558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.872740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.874435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.876410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.890891 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3800/7577] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:34 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/940bd0781295418abf4736e82f5b961c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:34.895618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.906354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7de8c1-bb94-4294-9f4c-8e66f55eefbb tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:34.911680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:34.912471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:34.912830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:34.913238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:34.913941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:create_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:34.915779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:34.928192 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3778/7578] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:34 2022] POST /identity/v3/projects => generated 348 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:34.930935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:34.963026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:35.009864 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3801/7579] 10.222.0.26 () {60 vars in 1314 bytes} [Sat Sep 17 21:20:34 2022] POST /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 789 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:35.011161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.023474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b86cf9-1675-44db-8d2c-6aee197303f3 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.027861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeae5c9-fa49-41c0-a713-029e5ada1494 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.040887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.041589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.041872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.042145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/d38f8cd8c5194a06a7c6750ca3c7989c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.044546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=d38f8cd8c5194a06a7c6750ca3c7989c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.046724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.050229 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3779/7580] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:34 2022] GET /identity/v3/projects/d38f8cd8c5194a06a7c6750ca3c7989c => generated 348 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:35.054345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.065717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1289-21fe-4760-9039-37b3d1ed0701 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.072359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.073171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.074145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.074145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/d38f8cd8c5194a06a7c6750ca3c7989c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.075404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d38f8cd8c5194a06a7c6750ca3c7989c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.077498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.106553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.108159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.108560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.110115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.111247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:create_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.112766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.170852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd38f8cd8c5194a06a7c6750ca3c7989c', 'request_id': 'req-704bb5c7-64d6-4dc7-85f8-942234abb1e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.181718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd38f8cd8c5194a06a7c6750ca3c7989c', 'request_id': 'req-704bb5c7-64d6-4dc7-85f8-942234abb1e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.192437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd38f8cd8c5194a06a7c6750ca3c7989c', 'request_id': 'req-704bb5c7-64d6-4dc7-85f8-942234abb1e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.211511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project d38f8cd8c5194a06a7c6750ca3c7989c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:35.211912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.212768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.214834 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3780/7581] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:35 2022] DELETE /identity/v3/projects/d38f8cd8c5194a06a7c6750ca3c7989c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:35.220414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.222204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:35.231554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:35.247910 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3802/7582] 10.222.0.26 () {60 vars in 1314 bytes} [Sat Sep 17 21:20:35 2022] POST /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 789 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:35.248846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.253884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:35.272792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22005054-d9bd-4eeb-b541-d9471a0b89be tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.312403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bb5c7-64d6-4dc7-85f8-942234abb1e6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.318798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.319849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.320055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.320558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.321394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.323575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.349101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.354351 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3781/7583] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:20:35 2022] POST /identity/v3/domains => generated 276 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:35.354472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.354472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.354472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.354472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:list_application_credentials(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.355477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.357077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.379117 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3803/7584] 10.222.0.26 () {58 vars in 1293 bytes} [Sat Sep 17 21:20:35 2022] GET /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 1502 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:35.380357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.395980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b16733-dd7a-4a15-bde2-27eeb6f7eae4 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.413130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:35.443093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9e30-f70d-49ae-927b-75d00c7b1111 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.448900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.449853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.450386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.450664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.451619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.453654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.475199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.476489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.476986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.477546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/10183c493b8e413abcf73cd8844209c4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.489699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=10183c493b8e413abcf73cd8844209c4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.492427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.508605 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3782/7585] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:35 2022] POST /identity/v3/projects => generated 404 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:35.509815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.521361 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3804/7586] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:35 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/10183c493b8e413abcf73cd8844209c4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:35.526916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.544695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6baaa7-c88a-4dd4-bfc8-4134925446f9 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.614631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7718b0-8f5e-46d4-a4fe-379cd20d56ff tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.622004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.622605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.623036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.623185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/c6812d878c324d3db769c465eba04546` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.624888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=c6812d878c324d3db769c465eba04546)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.627588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.632586 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3783/7587] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:35 2022] GET /identity/v3/projects/c6812d878c324d3db769c465eba04546 => generated 404 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:35.632754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.632754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.632754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.633089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/c9b8a948b42d405b9d5f58530232c587` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.634215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.646250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=c9b8a948b42d405b9d5f58530232c587)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.647315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3241-b1f2-4b65-98cb-04678b40c3f9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.648168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.654428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.655644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.656139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.656519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/c6812d878c324d3db769c465eba04546` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.658778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c6812d878c324d3db769c465eba04546)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.659585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:35.660663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.678303 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3805/7588] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:35 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/c9b8a948b42d405b9d5f58530232c587 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:35.694710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.712671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad28f28-626b-464c-ada8-95d6246866d2 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.727193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6812d878c324d3db769c465eba04546', 'request_id': 'req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.738875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6812d878c324d3db769c465eba04546', 'request_id': 'req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.746361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6812d878c324d3db769c465eba04546', 'request_id': 'req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.760010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project c6812d878c324d3db769c465eba04546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:35.760307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.761001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.763319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3784/7589] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:35 2022] DELETE /identity/v3/projects/c6812d878c324d3db769c465eba04546 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:35.768097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.769734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:35.820165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.820165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.820165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.822949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.822949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:create_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.822949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.846044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:35.880886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96d1a2-f853-4c49-b9ec-9e3d7d8a544e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.892797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:35.894579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:35.895020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:35.895616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains/7a3f8a59a1f54f5ab28d46db71fc7646` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:35.897221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=7a3f8a59a1f54f5ab28d46db71fc7646)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:35.899235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:35.922121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.922892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.938498 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3806/7590] 10.222.0.26 () {60 vars in 1314 bytes} [Sat Sep 17 21:20:35 2022] POST /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 788 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:35.939812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.942202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:35.962503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb002ebc-a556-46af-9b89-967865c27ae1 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:35.972083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.973799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:35.980000 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3785/7591] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:20:35 2022] PATCH /identity/v3/domains/7a3f8a59a1f54f5ab28d46db71fc7646 => generated 277 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:35.984480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:35.986970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:36.064633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.065885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.066353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.066859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.067723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:create_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.069414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.083750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e028da2-0190-4b1c-a54f-e22a31795786 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.088730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.089312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.092196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.092507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains/7a3f8a59a1f54f5ab28d46db71fc7646` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.107329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=7a3f8a59a1f54f5ab28d46db71fc7646)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.108946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.145486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.161649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:36.173709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.197558 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3807/7592] 10.222.0.26 () {60 vars in 1314 bytes} [Sat Sep 17 21:20:35 2022] POST /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials => generated 789 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:36.198680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.203982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:36.229878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89069db5-a124-4128-bc66-e7e4ac9c1caa tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.239653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.240348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.240681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.240990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.241752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Adding query filter params (name=tempest-application_credential-2135644925) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:36.242000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:list_application_credentials(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.244044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.261424 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3808/7593] 10.222.0.26 () {58 vars in 1386 bytes} [Sat Sep 17 21:20:36 2022] GET /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials?name=tempest-application_credential-2135644925 => generated 887 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:36.264256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.267888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646', 'request_id': 'req-430ac62c-a297-4a5f-864c-486349652ff2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.278470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5090e0-bec0-4e18-8e32-4a504553c921 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.280855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646', 'request_id': 'req-430ac62c-a297-4a5f-864c-486349652ff2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.289134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646', 'request_id': 'req-430ac62c-a297-4a5f-864c-486349652ff2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.305708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 7a3f8a59a1f54f5ab28d46db71fc7646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:36.306736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.307103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.311598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:UPDATE=1,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:36.317836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646', 'request_id': 'req-430ac62c-a297-4a5f-864c-486349652ff2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.330087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646', 'request_id': 'req-430ac62c-a297-4a5f-864c-486349652ff2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.331324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '7a3f8a59a1f54f5ab28d46db71fc7646', 'request_id': 'req-430ac62c-a297-4a5f-864c-486349652ff2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.340002 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3786/7594] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:35 2022] DELETE /identity/v3/domains/7a3f8a59a1f54f5ab28d46db71fc7646 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:36.344660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.346598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:36.366351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.367020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.367299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.367571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/5a114a6dcef643b69d420d43f6cb7f25` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.378260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=5a114a6dcef643b69d420d43f6cb7f25)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.379840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.408726 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3809/7595] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:36 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/5a114a6dcef643b69d420d43f6cb7f25 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:36.410064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.414310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:36.428579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672be702-c7aa-4055-846f-58f6bb421347 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.437317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da0b8af7a5364d3385e761fc2b4117fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.438162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.438544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.438917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/16bf2c8ee66241319a1850fa7bb30564` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.445784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac62c-a297-4a5f-864c-486349652ff2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.451237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.451949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.452349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.452675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.453285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.454162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorizing `identity:delete_application_credential(user_id=da0b8af7a5364d3385e761fc2b4117fb, application_credential_id=16bf2c8ee66241319a1850fa7bb30564)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.455189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.455836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.477950 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3787/7596] 10.222.0.26 () {60 vars in 1149 bytes} [Sat Sep 17 21:20:36 2022] POST /identity/v3/domains => generated 277 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:36.481513 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3810/7597] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:20:36 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb/application_credentials/16bf2c8ee66241319a1850fa7bb30564 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:36.481689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.487663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.489541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:36.580349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07114fa4-f498-4022-af03-ef280c23c88f tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.584710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf7d9b-6192-4583-918a-4dadb1843d01 tempest-ApplicationCredentialsV3Test-339252967 tempest-ApplicationCredentialsV3Test-339252967-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.585918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.586538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.586983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.587265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.596520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.604671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.643372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=2,keystone:SELECT=94,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:36.675966 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3788/7598] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:36 2022] POST /identity/v3/projects => generated 408 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:36.680290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.685268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.695718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.695859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.696087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] PATH_INFO: `/v3/users/da0b8af7a5364d3385e761fc2b4117fb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.697527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da0b8af7a5364d3385e761fc2b4117fb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.700768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.759823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:36.766054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d937d2-31e0-4f31-8374-805395004e56 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.771907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da0b8af7a5364d3385e761fc2b4117fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.772911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da0b8af7a5364d3385e761fc2b4117fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.774840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:36.775609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.775978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.776703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.777440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:36.779992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:36.783385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da0b8af7a5364d3385e761fc2b4117fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.784685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da0b8af7a5364d3385e761fc2b4117fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.817920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bfd161aa-1988-4e9c-9002-7b7676f20e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da0b8af7a5364d3385e761fc2b4117fb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:36.832079 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3789/7599] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:36 2022] POST /identity/v3/projects => generated 404 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:36.834720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.836742 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3811/7600] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:36 2022] DELETE /identity/v3/users/da0b8af7a5364d3385e761fc2b4117fb => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:36.837110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:36.841175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:36.841698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:36.842732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:36.866479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:36.868269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:36.924159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8468bcf-f4fa-421d-a9cf-d1e01c56a5f2 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:36.945648 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3812/7601] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:36.956838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:36.958534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:37.002246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:37.003052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.003427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.003892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/8af4c5aa99784101b4d36086e6efc76e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:37.006315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8af4c5aa99784101b4d36086e6efc76e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:37.009027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:37.032923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852527bf-924e-4e5a-a64d-20ad84d273b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:37.037672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:37.038460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.038748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.039030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:37.045419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:37.051348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:37.053247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:37.067290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3813/7602] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:37.089360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af4c5aa99784101b4d36086e6efc76e', 'request_id': 'req-e30f4908-1357-4679-8835-7bde7d2b4317'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.100860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af4c5aa99784101b4d36086e6efc76e', 'request_id': 'req-e30f4908-1357-4679-8835-7bde7d2b4317'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.113747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af4c5aa99784101b4d36086e6efc76e', 'request_id': 'req-e30f4908-1357-4679-8835-7bde7d2b4317'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.134684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 8af4c5aa99784101b4d36086e6efc76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:37.135378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.136362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.142598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:37.144900 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3790/7603] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:36 2022] DELETE /identity/v3/projects/8af4c5aa99784101b4d36086e6efc76e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:37.157975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2920e-eeff-4445-b648-5b2cb07468c0 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:37.261718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:37.262675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.263835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.264420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/b100fd62ed6b4ba8a0ac74babfac8987` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:37.266336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b100fd62ed6b4ba8a0ac74babfac8987)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:37.270386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:37.366082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b100fd62ed6b4ba8a0ac74babfac8987', 'request_id': 'req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.375702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b100fd62ed6b4ba8a0ac74babfac8987', 'request_id': 'req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.389503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b100fd62ed6b4ba8a0ac74babfac8987', 'request_id': 'req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.406279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project b100fd62ed6b4ba8a0ac74babfac8987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:37.406758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.408098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.410242 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3814/7604] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:37 2022] DELETE /identity/v3/projects/b100fd62ed6b4ba8a0ac74babfac8987 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:37.414418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:37.415899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:37.429588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f4908-1357-4679-8835-7bde7d2b4317 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:37.439744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:37.467921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:37.481615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:37.533863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:37.535146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.535418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.535685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains/ff6903dfd3d64d9b8851288794d36120` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:37.536994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=ff6903dfd3d64d9b8851288794d36120)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:37.539189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:37.572697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.575345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.591353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b21e5f-6d7a-4d69-b83a-6ff4505a029d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:37.614943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.616262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.620129 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3791/7605] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:20:37 2022] PATCH /identity/v3/domains/ff6903dfd3d64d9b8851288794d36120 => generated 278 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:37.624110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:37.626245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:37.631772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:37.708497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:37.710022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.710590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.710873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] PATH_INFO: `/v3/projects/aa45eb07c2c548dda3a2ea32379a7cc7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:37.712169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa45eb07c2c548dda3a2ea32379a7cc7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:37.714182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:37.728626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1b923-3553-45e1-9fb2-5bbd3d07b3ed tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:37.736177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:37.737050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.737695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.738248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains/ff6903dfd3d64d9b8851288794d36120` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:37.753836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=ff6903dfd3d64d9b8851288794d36120)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:37.756635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:37.778883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'request_id': 'req-dc77e797-7877-408d-bf16-f82dc0816969'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.789724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'request_id': 'req-dc77e797-7877-408d-bf16-f82dc0816969'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.798314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.805091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa45eb07c2c548dda3a2ea32379a7cc7', 'request_id': 'req-dc77e797-7877-408d-bf16-f82dc0816969'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.814644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.822167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] The token cache is being invalidate because project aa45eb07c2c548dda3a2ea32379a7cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:37.822467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.823225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dc77e797-7877-408d-bf16-f82dc0816969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.825416 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3815/7606] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:37 2022] DELETE /identity/v3/projects/aa45eb07c2c548dda3a2ea32379a7cc7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:37.831545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b75cee-ce43-42f5-96d9-2701a0378847 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.832270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b75cee-ce43-42f5-96d9-2701a0378847 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.832562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b75cee-ce43-42f5-96d9-2701a0378847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:37.867488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c8b75cee-ce43-42f5-96d9-2701a0378847 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:37.870133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8b75cee-ce43-42f5-96d9-2701a0378847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:37.872772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:37.877026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120', 'request_id': 'req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.894705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120', 'request_id': 'req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.903851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120', 'request_id': 'req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.918854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project ff6903dfd3d64d9b8851288794d36120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:37.919613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.920480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.931476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120', 'request_id': 'req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.935385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120', 'request_id': 'req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.936333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'ff6903dfd3d64d9b8851288794d36120', 'request_id': 'req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:37.947501 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3792/7607] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:37 2022] DELETE /identity/v3/domains/ff6903dfd3d64d9b8851288794d36120 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:37.953263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:37.956102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:37.962023 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3816/7608] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:37.971686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e94471-c182-47e6-b819-1f30a29d9489 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:37.972197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e94471-c182-47e6-b819-1f30a29d9489 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:37.972642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e94471-c182-47e6-b819-1f30a29d9489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.012258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d0e94471-c182-47e6-b819-1f30a29d9489 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:38.015573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d0e94471-c182-47e6-b819-1f30a29d9489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:38.034007 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3817/7609] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:38.038152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e94471-c182-47e6-b819-1f30a29d9489 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.051771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e94471-c182-47e6-b819-1f30a29d9489 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.060941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555fa36-62c0-4e8e-8e80-cd20b9fa2d42 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.066601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.067334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.067684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.067952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.068672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:38.068985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.070135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.070756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.071391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.072387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.072977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.074079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.078480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.084775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3818/7610] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:38.089901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.105020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992fc9-e5fd-4c98-a3b8-6ab228f195f8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.112002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.112801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.113185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.113911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.114879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.117322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.130317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3793/7611] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:37 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:38.134612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.157969 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3819/7612] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:38 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:38.159626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.205455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:38.233109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c3a7f-413a-46cf-9617-ad72e19217b3 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.268469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f824b4-8770-4651-abee-edf18cfbbb3b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.275377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.275747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.276262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.276632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.277431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.279659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.315279 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3820/7613] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:38 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:38.317035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.330721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440b85b-a753-4297-b0ff-b1bc73daf753 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.336262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.337190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.337782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.338116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.339016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.340269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:38.341030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.357076 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3821/7614] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:38.360899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.369726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.370394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.370671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.370990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/4e23397861044745b9e5a26be11b4f75` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.372660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=4e23397861044745b9e5a26be11b4f75)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.374456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5ff38-d6b6-4003-9564-6867a05c6c79 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.378408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.379691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.380603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.381068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.381425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.382205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.383734 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3794/7615] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:38 2022] GET /identity/v3/projects/4e23397861044745b9e5a26be11b4f75 => generated 377 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:38.384046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.388441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.402395 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3822/7616] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:38.403509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.415776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e0fc6-c4f4-4029-8421-844b33940136 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.421300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f79445-96c6-4bd3-92bd-4b30826f5d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.421741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09a46-e091-49c6-a69d-73277249cba9 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.422353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f79445-96c6-4bd3-92bd-4b30826f5d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.422746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f79445-96c6-4bd3-92bd-4b30826f5d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.423116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f79445-96c6-4bd3-92bd-4b30826f5d9f admin admin] PATH_INFO: `/v3/projects/b40ce4f6ce1c476a9c06cae6948c9e3f/users/11616677cbd7403a92b1d61121324b66/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.440602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.441237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.441543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.441819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.442219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f79445-96c6-4bd3-92bd-4b30826f5d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40ce4f6ce1c476a9c06cae6948c9e3f, user_id=11616677cbd7403a92b1d61121324b66, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.442477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.444648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.445660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f79445-96c6-4bd3-92bd-4b30826f5d9f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.469618 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3823/7617] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:38 2022] PUT /identity/v3/projects/b40ce4f6ce1c476a9c06cae6948c9e3f/users/11616677cbd7403a92b1d61121324b66/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:38.480244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.480536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.480814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.495656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3795/7618] 10.222.0.26 () {58 vars in 1131 bytes} [Sat Sep 17 21:20:38 2022] GET /identity/v3/projects => generated 6548 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:38.500444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.514449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1835d-31d2-4e4e-8412-c2739aedb278 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.517205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] MFA Rules not processed for user `11616677cbd7403a92b1d61121324b66`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:38.518895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:38.611874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.613034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.613034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.613284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/4e23397861044745b9e5a26be11b4f75` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.614333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4e23397861044745b9e5a26be11b4f75)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.616329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.630640 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3824/7619] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:38.643783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.645205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:38.702543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e23397861044745b9e5a26be11b4f75', 'request_id': 'req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:38.715770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e23397861044745b9e5a26be11b4f75', 'request_id': 'req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:38.724625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e23397861044745b9e5a26be11b4f75', 'request_id': 'req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:38.748193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1e4ac-ef83-40bd-82a9-5be431b41199 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.749729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 4e23397861044745b9e5a26be11b4f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:38.750031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:38.751288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:38.756123 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3796/7620] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:38 2022] DELETE /identity/v3/projects/4e23397861044745b9e5a26be11b4f75 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:38.761113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.761857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:38.764095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:38.833994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.835053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.835453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.835796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.838534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:38.870468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf1e5-fe2f-4c4b-a7de-b79d1472aa97 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.875271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.875895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:38.876174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:38.876442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:38.877265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.878998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.885639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:38.917357 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3797/7621] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:38 2022] POST /identity/v3/projects => generated 348 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:38.921895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:38.922611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:38.924122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:38.935800 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3825/7622] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:38.990725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2af624-8af3-4a81-afde-2ccc7dbf2f55 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:38.998528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:38.999376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.000150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.000427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/2755a2c431d444118257eb1f730f2258` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.002262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:update_project(project_id=2755a2c431d444118257eb1f730f2258)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.004419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.040336 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3798/7623] 10.222.0.26 () {60 vars in 1252 bytes} [Sat Sep 17 21:20:38 2022] PATCH /identity/v3/projects/2755a2c431d444118257eb1f730f2258 => generated 362 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:39.044615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.055763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42719-e1b3-4405-9cee-d92fdac62a37 service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.063173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8720b61-e0ed-4c3d-97e3-45cb5fa57b6e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.063952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8720b61-e0ed-4c3d-97e3-45cb5fa57b6e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.064241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8720b61-e0ed-4c3d-97e3-45cb5fa57b6e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.064531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8720b61-e0ed-4c3d-97e3-45cb5fa57b6e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/2755a2c431d444118257eb1f730f2258` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.076965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8720b61-e0ed-4c3d-97e3-45cb5fa57b6e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=2755a2c431d444118257eb1f730f2258)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.079168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8720b61-e0ed-4c3d-97e3-45cb5fa57b6e tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.081143 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3826/7624] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:39 2022] GET /identity/v3/projects/2755a2c431d444118257eb1f730f2258 => generated 349 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:39.085384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.098254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.098665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.099042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.100204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7a4c1-d500-4ff6-b71d-3cc6a79133a6 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.105003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.105692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.105999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.106398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/2755a2c431d444118257eb1f730f2258` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.107555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2755a2c431d444118257eb1f730f2258)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.109558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.122166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:39.123688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:39.162216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2755a2c431d444118257eb1f730f2258', 'request_id': 'req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.172730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2755a2c431d444118257eb1f730f2258', 'request_id': 'req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.179919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2755a2c431d444118257eb1f730f2258', 'request_id': 'req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.193096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 2755a2c431d444118257eb1f730f2258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:39.197572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.197776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.199749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3799/7625] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:39 2022] DELETE /identity/v3/projects/2755a2c431d444118257eb1f730f2258 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:39.203566 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3827/7626] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:39.204833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.207365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:39.269203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:UPDATE=1,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:39.269380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.270876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:39.291258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e48b0-ce52-4c3c-9b41-10a44a31ad0d tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.297495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.298204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.298589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.298932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.299670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.302431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.333125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:39.347762 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3800/7627] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:39 2022] POST /identity/v3/projects => generated 360 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:39.351802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.372852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a7acb-b40b-4cd9-aabd-e96361633f09 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.430649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b56674-4236-49da-9968-0a2ed6ed6356 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.436222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.437039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.437394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.437816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/72c7803d7d634a94831406a5f33dcc34` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.439063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:update_project(project_id=72c7803d7d634a94831406a5f33dcc34)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.441260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.441374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.442056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.442395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.442729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.444776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:39.501866 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3801/7628] 10.222.0.26 () {60 vars in 1252 bytes} [Sat Sep 17 21:20:39 2022] PATCH /identity/v3/projects/72c7803d7d634a94831406a5f33dcc34 => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:39.503083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.514104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af97c3-5178-45a4-a6b9-247731a24db5 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.516819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.518626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.519170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.519813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.520097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.520376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/72c7803d7d634a94831406a5f33dcc34` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.528339 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3828/7629] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:20:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:20:39.535549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=72c7803d7d634a94831406a5f33dcc34)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.537769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.542397 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3802/7630] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:39 2022] GET /identity/v3/projects/72c7803d7d634a94831406a5f33dcc34 => generated 359 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:39.543738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.558100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8862f-368a-475a-b0eb-04b04539a358 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.563332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.564053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.564395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.564736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/72c7803d7d634a94831406a5f33dcc34` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.565992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=72c7803d7d634a94831406a5f33dcc34)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.568925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.646092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c7803d7d634a94831406a5f33dcc34', 'request_id': 'req-9d7ed69c-9a4b-4852-a06d-1284d03c817b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.656098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c7803d7d634a94831406a5f33dcc34', 'request_id': 'req-9d7ed69c-9a4b-4852-a06d-1284d03c817b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.664506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c7803d7d634a94831406a5f33dcc34', 'request_id': 'req-9d7ed69c-9a4b-4852-a06d-1284d03c817b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.694866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project 72c7803d7d634a94831406a5f33dcc34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:39.695088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.696006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:39.698133 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3829/7631] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:39 2022] DELETE /identity/v3/projects/72c7803d7d634a94831406a5f33dcc34 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:39.703100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.707312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:39.723275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843976-ab3e-4661-9f38-7a9276261e26 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.818378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946d9a4-5751-42d6-aa95-9ad7dac63b44 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.819227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d9a4-5751-42d6-aa95-9ad7dac63b44 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.819610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d9a4-5751-42d6-aa95-9ad7dac63b44 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.820060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d9a4-5751-42d6-aa95-9ad7dac63b44 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.820800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946d9a4-5751-42d6-aa95-9ad7dac63b44 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.823000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946d9a4-5751-42d6-aa95-9ad7dac63b44 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.865934 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3803/7632] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:39 2022] POST /identity/v3/projects => generated 354 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:39.867133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:39.874644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:39.878821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed69c-9a4b-4852-a06d-1284d03c817b tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:39.953554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:39.954218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.954521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.954797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/e5e10ef880f14be58818a7e3d68477c5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.956034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:update_project(project_id=e5e10ef880f14be58818a7e3d68477c5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:39.957115 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:20:39.957115 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:20:39.957115 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:20:39.958478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:39.971911 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:39.972209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:39.972458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:39.997030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:39.998631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.001507 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3830/7633] 10.222.0.26 () {60 vars in 1252 bytes} [Sat Sep 17 21:20:39 2022] PATCH /identity/v3/projects/e5e10ef880f14be58818a7e3d68477c5 => generated 368 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:40.004020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:40.005043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.016440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4992-90e3-455d-9fd2-198dfeab95e4 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.021473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.022262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.022633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.023002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/e5e10ef880f14be58818a7e3d68477c5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.040164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:get_project(project_id=e5e10ef880f14be58818a7e3d68477c5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:40.042502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:40.044771 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3831/7634] 10.222.0.26 () {58 vars in 1230 bytes} [Sat Sep 17 21:20:40 2022] GET /identity/v3/projects/e5e10ef880f14be58818a7e3d68477c5 => generated 355 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:40.048746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.068822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebecda-d8d1-4fcb-83f5-1bfab703ea32 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.072990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3804/7635] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:40.073582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.074307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.074597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.074875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/projects/e5e10ef880f14be58818a7e3d68477c5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.076019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e5e10ef880f14be58818a7e3d68477c5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:40.078111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:40.085437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.087061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.133901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e10ef880f14be58818a7e3d68477c5', 'request_id': 'req-69539ac9-8cb6-4984-85f3-174eea67f777'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.142781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e10ef880f14be58818a7e3d68477c5', 'request_id': 'req-69539ac9-8cb6-4984-85f3-174eea67f777'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.156603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e10ef880f14be58818a7e3d68477c5', 'request_id': 'req-69539ac9-8cb6-4984-85f3-174eea67f777'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.174099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] The token cache is being invalidate because project e5e10ef880f14be58818a7e3d68477c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:40.174403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.175131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.177105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3832/7636] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:40 2022] DELETE /identity/v3/projects/e5e10ef880f14be58818a7e3d68477c5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:40.181996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcba21-a96d-4009-b14f-71e1d4094593 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.182304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.183806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.258201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.258929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.259211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.259484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.261345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.261818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:40.279159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69539ac9-8cb6-4984-85f3-174eea67f777 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.286178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.286861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.287406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.288693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains/ff6903dfd3d64d9b8851288794d36120` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.298182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Could not find domain: ff6903dfd3d64d9b8851288794d36120.: keystone.exception.DomainNotFound: Could not find domain: ff6903dfd3d64d9b8851288794d36120. Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ff6903dfd3d64d9b8851288794d36120. Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:40.303642 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:20:40.304937 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: ff6903dfd3d64d9b8851288794d36120. Sep 17 21:20:40.306626 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:40.308213 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3833/7637] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:20:40 2022] PATCH /identity/v3/domains/ff6903dfd3d64d9b8851288794d36120 => generated 112 bytes in 123 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:40.310031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.321144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda42e1c-0803-453d-805f-0655161ca809 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.327950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200ac26-8f85-47ee-853c-7527172e55a8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b840456466464658b73495e9861845f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cb5947bc0be4a2db8d18701756b5f3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.328939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200ac26-8f85-47ee-853c-7527172e55a8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.329334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200ac26-8f85-47ee-853c-7527172e55a8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.329652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200ac26-8f85-47ee-853c-7527172e55a8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] PATH_INFO: `/v3/domains/7a3f8a59a1f54f5ab28d46db71fc7646` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.330153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:40.332034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:40.336054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:40.337750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200ac26-8f85-47ee-853c-7527172e55a8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 21:20:40.343032 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3805/7638] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-6200ac26-8f85-47ee-853c-7527172e55a8 tempest-ProjectsTestJSON-1377739379 tempest-ProjectsTestJSON-1377739379-project-admin] Could not find domain: 7a3f8a59a1f54f5ab28d46db71fc7646.: keystone.exception.DomainNotFound: Could not find domain: 7a3f8a59a1f54f5ab28d46db71fc7646. Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 7a3f8a59a1f54f5ab28d46db71fc7646. Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:40.344104 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:20:40.345211 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application with Lock( Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._enter() Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.creator() Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application created_value = creator( Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 7a3f8a59a1f54f5ab28d46db71fc7646. Sep 17 21:20:40.347946 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:40.349489 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3834/7639] 10.222.0.26 () {60 vars in 1249 bytes} [Sat Sep 17 21:20:40 2022] PATCH /identity/v3/domains/7a3f8a59a1f54f5ab28d46db71fc7646 => generated 112 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:40.353536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.355298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.438444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d3aae-c72d-4c6d-abe8-dbbf80536e98 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.444206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.445725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.446301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.447089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] PATH_INFO: `/v3/users/3203ea74578c474dab0e959b4c3cc76b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.448721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3203ea74578c474dab0e959b4c3cc76b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:40.451133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:40.518693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3203ea74578c474dab0e959b4c3cc76b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.537218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3203ea74578c474dab0e959b4c3cc76b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.551233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3203ea74578c474dab0e959b4c3cc76b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.552143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3203ea74578c474dab0e959b4c3cc76b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.552923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3203ea74578c474dab0e959b4c3cc76b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:40.570270 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3806/7640] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:40 2022] DELETE /identity/v3/users/3203ea74578c474dab0e959b4c3cc76b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:40.571929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3d82a-6f6e-498d-941f-53655faa58ae None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.572304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3d82a-6f6e-498d-941f-53655faa58ae None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.572659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3d82a-6f6e-498d-941f-53655faa58ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.597992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-edf3d82a-6f6e-498d-941f-53655faa58ae None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:40.599680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-edf3d82a-6f6e-498d-941f-53655faa58ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.699747 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3835/7641] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:40.709576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.711062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.779832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.780119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.780391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.789224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a828ec-0955-4371-ae21-4fc5bed62221 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.793093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.793960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.794303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.794648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:40.804876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:40.805096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:40.806181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.806294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:40.817423 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3807/7642] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:40.871275 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3836/7643] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:40.885692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:40.887603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:40.914719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:40.976834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff883a5-f79d-4095-a117-6d4ffbc8439a service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:40.982254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2cd16-1015-41eb-a593-ba13c79b3b1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:40.982957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cd16-1015-41eb-a593-ba13c79b3b1f service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:40.983232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cd16-1015-41eb-a593-ba13c79b3b1f service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:40.983502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cd16-1015-41eb-a593-ba13c79b3b1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:41.006663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2cd16-1015-41eb-a593-ba13c79b3b1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:41.008313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2cd16-1015-41eb-a593-ba13c79b3b1f service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:41.019063 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3808/7644] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:20:41.132064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:41.134081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:41.170537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbdbdd-ce52-4c7c-b4b5-09e333c0ecd8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:41.177000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:41.177801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:41.178121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:41.178503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] PATH_INFO: `/v3/projects/8a4cb7f9fdb349b9be017068291a5c23` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:41.179633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4cb7f9fdb349b9be017068291a5c23)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:41.181652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:41.205817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:41.206049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:41.206517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:41.234868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:41.236352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:41.242675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4cb7f9fdb349b9be017068291a5c23', 'request_id': 'req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.251195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4cb7f9fdb349b9be017068291a5c23', 'request_id': 'req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.260474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4cb7f9fdb349b9be017068291a5c23', 'request_id': 'req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.273116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] The token cache is being invalidate because project 8a4cb7f9fdb349b9be017068291a5c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:41.273400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.274071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.275818 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3837/7645] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:41 2022] DELETE /identity/v3/projects/8a4cb7f9fdb349b9be017068291a5c23 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:41.279448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:41.281008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:41.298383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:41.304054 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3809/7646] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:41.317145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:41.318709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:41.366407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47b3b9-fcf9-4021-8e5c-6df2988bf8d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:41.373180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:41.374501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:41.375235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:41.375649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] PATH_INFO: `/v3/users/b840456466464658b73495e9861845f8` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:41.376997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b840456466464658b73495e9861845f8)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:41.388654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:41.408194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36360477-2cb5-4048-8c17-18f991784c02 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:41.414007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:41.414920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:41.415293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:41.415958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:41.426344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:41.428433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:41.441596 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3810/7647] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:20:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:20:41.466357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b840456466464658b73495e9861845f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.467281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b840456466464658b73495e9861845f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.480398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b840456466464658b73495e9861845f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.482280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b840456466464658b73495e9861845f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.504910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13c7fc77-3948-4576-85af-1bda6f5da1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b840456466464658b73495e9861845f8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:41.521597 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3838/7648] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:41 2022] DELETE /identity/v3/users/b840456466464658b73495e9861845f8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:41.848140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:41.861027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92308d6-c45e-48b2-9d1b-88e24b2998e5 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:41.888007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:41.888471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:41.888738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:41.925702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:41.944483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:41.945129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:41.945857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:41.946024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:41.946696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:41.946696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] PATH_INFO: `/v3/projects/7cb5947bc0be4a2db8d18701756b5f3c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:41.947952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb5947bc0be4a2db8d18701756b5f3c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:41.949940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.024095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb5947bc0be4a2db8d18701756b5f3c', 'request_id': 'req-ee4f4437-6f0f-4762-84ca-395400e5c02d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:42.033728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb5947bc0be4a2db8d18701756b5f3c', 'request_id': 'req-ee4f4437-6f0f-4762-84ca-395400e5c02d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:42.035946 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3839/7649] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:42.043985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb5947bc0be4a2db8d18701756b5f3c', 'request_id': 'req-ee4f4437-6f0f-4762-84ca-395400e5c02d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:42.051195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.052682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:42.062625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] The token cache is being invalidate because project 7cb5947bc0be4a2db8d18701756b5f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:42.063162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:42.064103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ee4f4437-6f0f-4762-84ca-395400e5c02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:42.066034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3811/7650] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:41 2022] DELETE /identity/v3/projects/7cb5947bc0be4a2db8d18701756b5f3c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:42.072071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d59f-0e61-4da4-afcd-f12acf9a679f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.073180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d59f-0e61-4da4-afcd-f12acf9a679f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.073655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d59f-0e61-4da4-afcd-f12acf9a679f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.077536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:42.100000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0ad9d59f-0e61-4da4-afcd-f12acf9a679f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:42.101328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ad9d59f-0e61-4da4-afcd-f12acf9a679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:42.145246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02944fc6-0e90-46a1-8306-51feebd6c7b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.174078 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3812/7651] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:42.177216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043592c8-61b8-4128-9432-168522ad1b65 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.177614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043592c8-61b8-4128-9432-168522ad1b65 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.178005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043592c8-61b8-4128-9432-168522ad1b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.199917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-043592c8-61b8-4128-9432-168522ad1b65 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:42.201213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-043592c8-61b8-4128-9432-168522ad1b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:42.216126 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3813/7652] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:42.217269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043592c8-61b8-4128-9432-168522ad1b65 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.218841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.219536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.219838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.220305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.222904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:42.228254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043592c8-61b8-4128-9432-168522ad1b65 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.232945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.233811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.234100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.234370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.235028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:42.235336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.237109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.250081 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3814/7653] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:42.254903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.265073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335d4f-9c4d-4d3d-a744-62c3a6251b00 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.270752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.271360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.271705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.271987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.272560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.275586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.288408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.290514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.307684 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3840/7654] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:20:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:20:42.313685 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3815/7655] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:42.317348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.332476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a49b0-2bea-4549-a591-b7222ae44a87 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.392866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:42.403654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.404371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.404599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.404939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.406232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.408194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.445161 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3841/7656] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/users => generated 285 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:42.448972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.465930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd32dbc-53b2-4434-ad57-3b37d96e6b9f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.471266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.472061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.472490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.472936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.473777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.475623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.488373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3816/7657] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:42.492730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.507142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1902b-ad1a-42ea-aa17-46cb751b7e4a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.512235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf41d3-b118-406b-bb5c-e58a4841eaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.513043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf41d3-b118-406b-bb5c-e58a4841eaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.514168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf41d3-b118-406b-bb5c-e58a4841eaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.514168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf41d3-b118-406b-bb5c-e58a4841eaf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.515287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf41d3-b118-406b-bb5c-e58a4841eaf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.516969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf41d3-b118-406b-bb5c-e58a4841eaf7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.533368 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3842/7658] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:42.533988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.546393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164de5f3-e0cb-4f71-a93d-f779cbc6941b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.547471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.547937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.548398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.551120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53f5a5-de9a-4890-b9bb-65e0547bc924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.551768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53f5a5-de9a-4890-b9bb-65e0547bc924 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.552048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53f5a5-de9a-4890-b9bb-65e0547bc924 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.552324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53f5a5-de9a-4890-b9bb-65e0547bc924 admin admin] PATH_INFO: `/v3/projects/23f41e28c7ee40429bb41120fec3db79/users/cb68671c40134a90bfb149f064c03906/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.569862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53f5a5-de9a-4890-b9bb-65e0547bc924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f41e28c7ee40429bb41120fec3db79, user_id=cb68671c40134a90bfb149f064c03906, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.571363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:42.572564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53f5a5-de9a-4890-b9bb-65e0547bc924 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.572739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:42.583707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:42.600940 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3817/7659] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:42 2022] PUT /identity/v3/projects/23f41e28c7ee40429bb41120fec3db79/users/cb68671c40134a90bfb149f064c03906/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:42.602486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c1fe-d790-4b8c-876b-6902ee8e17c2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.602891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c1fe-d790-4b8c-876b-6902ee8e17c2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.603234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c1fe-d790-4b8c-876b-6902ee8e17c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.634694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9518c1fe-d790-4b8c-876b-6902ee8e17c2 None None] MFA Rules not processed for user `cb68671c40134a90bfb149f064c03906`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:42.636262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9518c1fe-d790-4b8c-876b-6902ee8e17c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:42.640901 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3843/7660] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:42.653851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.655451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:42.731094 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3818/7661] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:42.738358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518c1fe-d790-4b8c-876b-6902ee8e17c2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.754292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf6df3-ba9b-4d67-9350-aec3b2f67e72 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.826502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.827132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.827406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.827744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.827888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518c1fe-d790-4b8c-876b-6902ee8e17c2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.832798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.833534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.834028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.834364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.835064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.837122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.862279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:42.870479 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3819/7662] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:42.874445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:42.920626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.922467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:42.932292 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3844/7663] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:20:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:20:42.965535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1d66c-cd03-46d2-a4f1-c5b67d98b295 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:42.972195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:42.973015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:42.973443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:42.973971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:42.974834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:42.976992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.005626 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3820/7664] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:42 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:43.010761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.022298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa009-b33e-4705-a8bf-3cf45491d27d service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.027560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.028288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.028614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.028949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.029679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.031622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.047929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3845/7665] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:43.048151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.052990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:43.059096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a367303-566c-4d3e-91a5-b4767a3f7da5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.066801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.067498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.067780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.068215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] PATH_INFO: `/v3/projects/e41796951b9f48cf98f29c2f03491a35/users/0fbd741eb9584de7b6a4644413992b48/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.084233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41796951b9f48cf98f29c2f03491a35, user_id=0fbd741eb9584de7b6a4644413992b48, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.087050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.094634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.096694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:43.105888 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3821/7666] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:43 2022] PUT /identity/v3/projects/e41796951b9f48cf98f29c2f03491a35/users/0fbd741eb9584de7b6a4644413992b48/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:43.110669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.114684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5afa83-3a5c-45b8-8ddb-c46b3d7dea26 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.188270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ab22c-7bbd-4a2e-87c3-c0ad8364b7db admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.194480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.195857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.196296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.196789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.197658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.199462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.200789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.201557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.201836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.202102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] PATH_INFO: `/v3/users/11616677cbd7403a92b1d61121324b66` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.203235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] RBAC: Authorizing `identity:delete_user(user_id=11616677cbd7403a92b1d61121324b66)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.205282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.216781 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3822/7667] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:43.218318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.230880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dc578-271a-4f92-819b-1754d8460874 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.236845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5186b-4bdd-408b-9a94-aa3d9182a4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.237865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5186b-4bdd-408b-9a94-aa3d9182a4b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.238325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5186b-4bdd-408b-9a94-aa3d9182a4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.238787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5186b-4bdd-408b-9a94-aa3d9182a4b7 admin admin] PATH_INFO: `/v3/projects/e41796951b9f48cf98f29c2f03491a35/users/0fbd741eb9584de7b6a4644413992b48/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.241406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5186b-4bdd-408b-9a94-aa3d9182a4b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41796951b9f48cf98f29c2f03491a35, user_id=0fbd741eb9584de7b6a4644413992b48, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.246157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5186b-4bdd-408b-9a94-aa3d9182a4b7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.258455 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3823/7668] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:43 2022] PUT /identity/v3/projects/e41796951b9f48cf98f29c2f03491a35/users/0fbd741eb9584de7b6a4644413992b48/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:43.269143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11616677cbd7403a92b1d61121324b66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:43.270079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11616677cbd7403a92b1d61121324b66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:43.270515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f06187-8f98-4b34-9126-cac73d51b327 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.270768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f06187-8f98-4b34-9126-cac73d51b327 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.271070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f06187-8f98-4b34-9126-cac73d51b327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.287367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11616677cbd7403a92b1d61121324b66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:43.288095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11616677cbd7403a92b1d61121324b66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:43.305949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-75f06187-8f98-4b34-9126-cac73d51b327 None None] MFA Rules not processed for user `0fbd741eb9584de7b6a4644413992b48`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:43.306842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab98729d-d781-42a7-b220-f569b6fcdcae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11616677cbd7403a92b1d61121324b66'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:43.307439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-75f06187-8f98-4b34-9126-cac73d51b327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:43.322833 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3846/7669] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:43 2022] DELETE /identity/v3/users/11616677cbd7403a92b1d61121324b66 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:43.327681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.327972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.328249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.348382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:43.349692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:43.387857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:43.415252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3824/7670] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:43.416134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f06187-8f98-4b34-9126-cac73d51b327 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.425657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f06187-8f98-4b34-9126-cac73d51b327 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.449114 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3847/7671] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:43.460611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.462724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:43.483725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:43.506010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.507079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.507543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.507979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.509129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.511731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.538129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b18e9-8d2c-4546-b65a-be7f28d67ed4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.542398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.543130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.543506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.543896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.548184 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3825/7672] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:43 2022] POST /identity/v3/projects => generated 329 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:43.553104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.559643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.561243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.574550 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3848/7673] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:43.628838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71811a-3821-4cd1-872a-3382328c401a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.633652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.634273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.634544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.634814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.635459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.637399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.665932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3826/7674] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:43 2022] POST /identity/v3/users => generated 416 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:43.672992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.700776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87629a-3530-48d7-8965-89692644b2cb service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.707299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.708278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.708706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.709243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.710143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.712045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.729746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3849/7675] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:43 2022] POST /identity/v3/roles => generated 243 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:43.734852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.747696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee58f05-5edc-4d1e-8330-79ed9c11abe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.753704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.754724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.755195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.755564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.756886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.759072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.784384 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3827/7676] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:43 2022] POST /identity/v3/roles => generated 247 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:43.785731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.796413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439dbb3-8cb3-4158-b2b1-80bca2c23228 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.801488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.802272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.802720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.803298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/26cb98cff726402090184ea0b3992e48/users/f6b5108d6bed4eb3bf930ec2a5f05101/roles/8a0c1ceb790649058857b84c2e7c16df` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.819904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=26cb98cff726402090184ea0b3992e48, user_id=f6b5108d6bed4eb3bf930ec2a5f05101, role_id=8a0c1ceb790649058857b84c2e7c16df)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.822828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.840466 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3850/7677] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:43 2022] PUT /identity/v3/projects/26cb98cff726402090184ea0b3992e48/users/f6b5108d6bed4eb3bf930ec2a5f05101/roles/8a0c1ceb790649058857b84c2e7c16df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:43.845296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.859271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e467b-e310-48b8-a596-1f39b6f0c25f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:43.944303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5280-f83a-40c9-8265-7a8c35cefeb9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:43.945214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5280-f83a-40c9-8265-7a8c35cefeb9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:43.946172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5280-f83a-40c9-8265-7a8c35cefeb9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:43.947218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5280-f83a-40c9-8265-7a8c35cefeb9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/26cb98cff726402090184ea0b3992e48/users/f6b5108d6bed4eb3bf930ec2a5f05101/roles/8e78c0bd0ba844e5999750c01db205f5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:43.948970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5280-f83a-40c9-8265-7a8c35cefeb9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=26cb98cff726402090184ea0b3992e48, user_id=f6b5108d6bed4eb3bf930ec2a5f05101, role_id=8e78c0bd0ba844e5999750c01db205f5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:43.951764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5280-f83a-40c9-8265-7a8c35cefeb9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:43.966167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3828/7678] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:43 2022] PUT /identity/v3/projects/26cb98cff726402090184ea0b3992e48/users/f6b5108d6bed4eb3bf930ec2a5f05101/roles/8e78c0bd0ba844e5999750c01db205f5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:43.970217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:43.986764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4334dc-20fe-48d7-bf09-04e515cf940c tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.047684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd4613-4276-4315-b7cb-0aefa9b39076 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.048350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4613-4276-4315-b7cb-0aefa9b39076 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.048628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4613-4276-4315-b7cb-0aefa9b39076 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.048902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4613-4276-4315-b7cb-0aefa9b39076 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.049845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd4613-4276-4315-b7cb-0aefa9b39076 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.051575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd4613-4276-4315-b7cb-0aefa9b39076 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.081402 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3851/7679] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:43 2022] GET /identity/v3/users => generated 8053 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:44.086787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.087037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.087543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.127974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] MFA Rules not processed for user `f6b5108d6bed4eb3bf930ec2a5f05101`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:44.129195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.169639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:44.215155 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3829/7680] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:44.225644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74d81b-d4ac-40c6-8248-60c8a95ab80f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.225879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74d81b-d4ac-40c6-8248-60c8a95ab80f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.226184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74d81b-d4ac-40c6-8248-60c8a95ab80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.253981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8f74d81b-d4ac-40c6-8248-60c8a95ab80f None None] MFA Rules not processed for user `f6b5108d6bed4eb3bf930ec2a5f05101`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:44.253981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8f74d81b-d4ac-40c6-8248-60c8a95ab80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.254841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.256245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.273328 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3852/7681] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:44.277847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74d81b-d4ac-40c6-8248-60c8a95ab80f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.289123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74d81b-d4ac-40c6-8248-60c8a95ab80f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.294359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6b5108d6bed4eb3bf930ec2a5f05101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26cb98cff726402090184ea0b3992e48', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1790654597', 'tempest-DelegatedRole-230464161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.294932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.295686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.295686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.296349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] RBAC: Authorizing `identity:create_trust()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.297777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.332309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:44.355414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03123ab0-8f11-45ea-88e0-03f5a2c2f39a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.360163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.360866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.361212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.361585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] PATH_INFO: `/v3/projects/b40ce4f6ce1c476a9c06cae6948c9e3f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.362784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40ce4f6ce1c476a9c06cae6948c9e3f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.364846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.404172 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3853/7682] 10.222.0.26 () {60 vars in 1174 bytes} [Sat Sep 17 21:20:44 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:44.407031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.423518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff5c6d-c754-4df4-9d67-b6ef6135d7a3 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.430857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40ce4f6ce1c476a9c06cae6948c9e3f', 'request_id': 'req-d328bfd1-39c1-42e6-bf48-490cecbc894e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.440557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40ce4f6ce1c476a9c06cae6948c9e3f', 'request_id': 'req-d328bfd1-39c1-42e6-bf48-490cecbc894e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.447537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40ce4f6ce1c476a9c06cae6948c9e3f', 'request_id': 'req-d328bfd1-39c1-42e6-bf48-490cecbc894e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.463043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] The token cache is being invalidate because project b40ce4f6ce1c476a9c06cae6948c9e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:44.463233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.464048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d328bfd1-39c1-42e6-bf48-490cecbc894e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.469781 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3830/7683] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:44 2022] DELETE /identity/v3/projects/b40ce4f6ce1c476a9c06cae6948c9e3f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:44.471006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf150-964e-4acc-897b-87fc324d89c9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.471290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf150-964e-4acc-897b-87fc324d89c9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.471563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aaf150-964e-4acc-897b-87fc324d89c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.496095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-26aaf150-964e-4acc-897b-87fc324d89c9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:44.497598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-26aaf150-964e-4acc-897b-87fc324d89c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.497795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.498428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.498708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.498981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.499889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.500946 ncdv-master-1 devstack@keystone.service[99167]: WARNING py.warnings [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:20:44.500946 ncdv-master-1 devstack@keystone.service[99167]: warnings.warn(msg) Sep 17 21:20:44.500946 ncdv-master-1 devstack@keystone.service[99167]: Sep 17 21:20:44.501972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.511103 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3854/7684] 10.222.0.26 () {58 vars in 1155 bytes} [Sat Sep 17 21:20:44 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:44.516538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.518904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.550038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:44.565772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3831/7685] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:44.574726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65b450-199a-40e2-8e1a-53de3ea3edf4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.575224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65b450-199a-40e2-8e1a-53de3ea3edf4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.575503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65b450-199a-40e2-8e1a-53de3ea3edf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.589505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad1c8c-6752-40b2-9b29-bd414ab256e1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.593813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6b5108d6bed4eb3bf930ec2a5f05101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26cb98cff726402090184ea0b3992e48', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1790654597', 'tempest-DelegatedRole-230464161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.594493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.594737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.595006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] PATH_INFO: `/v3/OS-TRUST/trusts/20bfadbc4f9d4c9bbbb0b206b8494bc3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.596715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2d65b450-199a-40e2-8e1a-53de3ea3edf4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:44.597950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d65b450-199a-40e2-8e1a-53de3ea3edf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.604088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] RBAC: Authorizing `identity:delete_trust(trust_id=20bfadbc4f9d4c9bbbb0b206b8494bc3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.605603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.606865 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3832/7686] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:44.610856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65b450-199a-40e2-8e1a-53de3ea3edf4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.620542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65b450-199a-40e2-8e1a-53de3ea3edf4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.626239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.626887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.627199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.627590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.628221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:44.628536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.630114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '20bfadbc4f9d4c9bbbb0b206b8494bc3', 'request_id': 'req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.630373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.639434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '20bfadbc4f9d4c9bbbb0b206b8494bc3', 'request_id': 'req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.639863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '20bfadbc4f9d4c9bbbb0b206b8494bc3', 'request_id': 'req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.646753 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3833/7687] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:44.649498 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3855/7688] 10.222.0.26 () {58 vars in 1254 bytes} [Sat Sep 17 21:20:44 2022] DELETE /identity/v3/OS-TRUST/trusts/20bfadbc4f9d4c9bbbb0b206b8494bc3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:44.650739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.651181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.652661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.653806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.666144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401774-a9cd-4e91-9289-f01fe37b05db admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.674208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4ee40-e4f5-4a79-a9dc-61295ef85f86 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.675461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.676185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.676541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.676882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.677622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.679684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.680998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6b5108d6bed4eb3bf930ec2a5f05101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26cb98cff726402090184ea0b3992e48', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1790654597', 'tempest-DelegatedRole-230464161'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.681747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.682089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.682484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] PATH_INFO: `/v3/OS-TRUST/trusts/20bfadbc4f9d4c9bbbb0b206b8494bc3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.710347 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3834/7689] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:44 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:44.714562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Could not find trust: 20bfadbc4f9d4c9bbbb0b206b8494bc3.: keystone.exception.TrustNotFound: Could not find trust: 20bfadbc4f9d4c9bbbb0b206b8494bc3. Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 17 21:20:44.721423 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 20bfadbc4f9d4c9bbbb0b206b8494bc3. Sep 17 21:20:44.722645 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:44.723541 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3856/7690] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:44 2022] GET /identity/v3/OS-TRUST/trusts/20bfadbc4f9d4c9bbbb0b206b8494bc3 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:44.727343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.729087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.748488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68c6f9-94b4-49d6-be1e-22f8f6578935 tempest-TrustsV3TestJSON-1426310757 tempest-user-1692582627] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.763013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:44.797350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971e870-55df-4d59-a41d-ec180f526ada admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.798685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.799239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.799518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.799924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.800594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.802692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.820968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.821888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.822170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.822442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/8e78c0bd0ba844e5999750c01db205f5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.823873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8e78c0bd0ba844e5999750c01db205f5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.825595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.828014 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3835/7691] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:44 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:44.832365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.843089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd87532-172a-4e36-a976-392b61a46b94 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.848332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.849067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.849569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.849799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.850449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.852071 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.863263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3836/7692] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:44 2022] GET /identity/v3/roles => generated 1892 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:44.867699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.872749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role 8e78c0bd0ba844e5999750c01db205f5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:44.873042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.873743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:44.876052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3857/7693] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:44 2022] DELETE /identity/v3/roles/8e78c0bd0ba844e5999750c01db205f5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:44.880097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.881563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.881694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8ceb1-51d4-4573-b2e1-118e45167e7c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.950652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.951480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.951850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.952179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.952884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.954657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.955479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa017c7-47da-41e0-a6d7-0bf87b1c69c9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.960696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.961341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.961655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.962394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/8a0c1ceb790649058857b84c2e7c16df` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:44.963313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8a0c1ceb790649058857b84c2e7c16df)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:44.964915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:44.966556 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3837/7694] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:44 2022] GET /identity/v3/roles => generated 1654 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:44.970515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:44.972330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:44.985226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc488c-b3b8-4fff-9c4c-5f1c37fd6957 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:44.990711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee8f88-448b-4c61-830d-9241b909cad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:44.991374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee8f88-448b-4c61-830d-9241b909cad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:44.991861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee8f88-448b-4c61-830d-9241b909cad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:44.992051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee8f88-448b-4c61-830d-9241b909cad5 admin admin] PATH_INFO: `/v3/projects/d839cc8be70e44ae9fa4ee75b2d1d407/users/0f64f10896b34f18a178fdb9e21df9f5/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.010705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee8f88-448b-4c61-830d-9241b909cad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d839cc8be70e44ae9fa4ee75b2d1d407, user_id=0f64f10896b34f18a178fdb9e21df9f5, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.013308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee8f88-448b-4c61-830d-9241b909cad5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.021806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role 8a0c1ceb790649058857b84c2e7c16df has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:45.022674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.023798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.026545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3858/7695] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:44 2022] DELETE /identity/v3/roles/8a0c1ceb790649058857b84c2e7c16df => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:45.029157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3838/7696] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:44 2022] PUT /identity/v3/projects/d839cc8be70e44ae9fa4ee75b2d1d407/users/0f64f10896b34f18a178fdb9e21df9f5/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:45.030868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.032459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.040666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.041157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.041660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.051046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:45.074640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] MFA Rules not processed for user `0f64f10896b34f18a178fdb9e21df9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:45.076032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.089821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:45.109709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e81b1f-d6ec-4e81-aaf4-158b19917f3e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.115672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.116435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.116721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.116997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users/f6b5108d6bed4eb3bf930ec2a5f05101` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.118851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f6b5108d6bed4eb3bf930ec2a5f05101)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.120811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.159425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3839/7697] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:45.160932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.162485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.176103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b5108d6bed4eb3bf930ec2a5f05101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.176775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b5108d6bed4eb3bf930ec2a5f05101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.186623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b5108d6bed4eb3bf930ec2a5f05101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.187934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b5108d6bed4eb3bf930ec2a5f05101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.206950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b5108d6bed4eb3bf930ec2a5f05101'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.230465 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3859/7698] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:45 2022] DELETE /identity/v3/users/f6b5108d6bed4eb3bf930ec2a5f05101 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:45.234773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.236203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.254794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0b9b6-31be-406f-8b8e-643b97e49831 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.317089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.317655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.317933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.318204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.318909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.320750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e16516-2dd3-439a-bb09-9c4d011baea8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.320884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.325619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.326329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.326665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.327004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/26cb98cff726402090184ea0b3992e48` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.328217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_project(project_id=26cb98cff726402090184ea0b3992e48)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.330280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.355634 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3840/7699] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:45.358627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.360465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.393505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:45.396227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26cb98cff726402090184ea0b3992e48', 'request_id': 'req-b5ec34a4-30f2-4634-9779-c66c4fb34371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.405314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26cb98cff726402090184ea0b3992e48', 'request_id': 'req-b5ec34a4-30f2-4634-9779-c66c4fb34371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.415682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26cb98cff726402090184ea0b3992e48', 'request_id': 'req-b5ec34a4-30f2-4634-9779-c66c4fb34371'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.422179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:45.434250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2246dcb-71ff-4038-ba43-2d7318986713 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.434784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] The token cache is being invalidate because project 26cb98cff726402090184ea0b3992e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:45.435489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.436659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:45.439020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3860/7700] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:45 2022] DELETE /identity/v3/projects/26cb98cff726402090184ea0b3992e48 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:45.445348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.446932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.505536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.507345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.508317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.508597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.509304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.511316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.520948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec34a4-30f2-4634-9779-c66c4fb34371 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.526584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.527408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.527667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.528027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.528783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.530888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.541088 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3841/7701] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:45.542506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.544767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.558174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8c34b-6c9f-41d6-b449-0b4a3b0bd269 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.561347 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3861/7702] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/projects => generated 329 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:45.565496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.567042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.567793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.568107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.568485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.569145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.570813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.582085 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3842/7703] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:45.585848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.597712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73056572-69ae-4950-b5ff-8e17e0fe162d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.643051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96d735-e209-4a35-a0f8-60d00cf88ce7 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.648705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.649486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.649778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.650051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.650816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.652902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.683756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.683995 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3862/7704] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/users => generated 413 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:45.684814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.685299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.685696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.686558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.687888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.688509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.706709 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3843/7705] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:45.707944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.712321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffcc07-108a-4f7c-bb6c-f37a1df0fc92 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.718777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d096b3-4e34-4739-a064-1a680c7a628e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.719826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.720925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.721354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.721963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.723004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.723778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e34071-7659-4d85-b221-4ab8027df79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.724436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e34071-7659-4d85-b221-4ab8027df79d admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.724751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e34071-7659-4d85-b221-4ab8027df79d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.725029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e34071-7659-4d85-b221-4ab8027df79d admin admin] PATH_INFO: `/v3/projects/29154538c0374a67a0e730db3ceb5b25/users/03c5d28ef7594471b8fca4f63ec3cf32/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.725141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.742118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e34071-7659-4d85-b221-4ab8027df79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29154538c0374a67a0e730db3ceb5b25, user_id=03c5d28ef7594471b8fca4f63ec3cf32, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.745211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e34071-7659-4d85-b221-4ab8027df79d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.746052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3863/7706] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/roles => generated 243 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:45.750313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.763141 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3844/7707] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:45 2022] PUT /identity/v3/projects/29154538c0374a67a0e730db3ceb5b25/users/03c5d28ef7594471b8fca4f63ec3cf32/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:45.773712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39640a-656f-4272-a361-8b5b484a16f5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.777386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.777779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.778349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.819888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] MFA Rules not processed for user `03c5d28ef7594471b8fca4f63ec3cf32`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:45.821069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.834944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:45.835139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:45.844454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.845234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.846485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.846954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.848020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.849798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.871610 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3864/7708] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/roles => generated 247 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:45.872940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.884599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48509f13-7f77-407b-b9dc-512061b27549 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:45.890010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:45.890740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:45.891878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:45.891878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/c578d118a9804b5abe6a89b459f80171/users/157333abd6394c17b1a68d27053d4d98/roles/adbcec2c74a9424eb0c4d7526ef83f2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:45.908161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3845/7709] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:45.909129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c578d118a9804b5abe6a89b459f80171, user_id=157333abd6394c17b1a68d27053d4d98, role_id=adbcec2c74a9424eb0c4d7526ef83f2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:45.911841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:45.922132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.923941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:45.929136 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3865/7710] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:45 2022] PUT /identity/v3/projects/c578d118a9804b5abe6a89b459f80171/users/157333abd6394c17b1a68d27053d4d98/roles/adbcec2c74a9424eb0c4d7526ef83f2e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:45.932939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:45.993780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78a496-bf18-4bb2-96c3-9e37c2f453fc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.009069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d2d71-88ba-45e1-82c9-7f7c68078420 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.017416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.018100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.018399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.018723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/c578d118a9804b5abe6a89b459f80171/users/157333abd6394c17b1a68d27053d4d98/roles/9081ea3f31be4286a071e9820aab81d0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.022111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c578d118a9804b5abe6a89b459f80171, user_id=157333abd6394c17b1a68d27053d4d98, role_id=9081ea3f31be4286a071e9820aab81d0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.025208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.047201 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3866/7711] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:45 2022] PUT /identity/v3/projects/c578d118a9804b5abe6a89b459f80171/users/157333abd6394c17b1a68d27053d4d98/roles/9081ea3f31be4286a071e9820aab81d0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:46.052404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.061169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a69112-c104-4ddb-82c2-a014bfbd1ffa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.061862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a69112-c104-4ddb-82c2-a014bfbd1ffa service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.062138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a69112-c104-4ddb-82c2-a014bfbd1ffa service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.062574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a69112-c104-4ddb-82c2-a014bfbd1ffa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.064763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-88a69112-c104-4ddb-82c2-a014bfbd1ffa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.139795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a69112-c104-4ddb-82c2-a014bfbd1ffa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.140495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1277336-822a-4f1a-b8e2-60cdd0b789da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.142227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a69112-c104-4ddb-82c2-a014bfbd1ffa service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.145564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.146184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.146656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.146958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.147627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.150496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.152695 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3846/7712] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:46.186158 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3867/7713] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:46 2022] GET /identity/v3/users => generated 8616 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:46.192700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba473-db4d-499b-acca-5031b22442ab None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.192700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba473-db4d-499b-acca-5031b22442ab None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.192700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba473-db4d-499b-acca-5031b22442ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.237453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bb9ba473-db4d-499b-acca-5031b22442ab None None] MFA Rules not processed for user `157333abd6394c17b1a68d27053d4d98`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:46.238736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bb9ba473-db4d-499b-acca-5031b22442ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.246993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:46.300376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.315745 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3847/7714] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:46 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:46.326504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816cf1c-41d0-4d07-8e98-25fe1323c093 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.326857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816cf1c-41d0-4d07-8e98-25fe1323c093 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.327497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816cf1c-41d0-4d07-8e98-25fe1323c093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.347288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a816cf1c-41d0-4d07-8e98-25fe1323c093 None None] MFA Rules not processed for user `157333abd6394c17b1a68d27053d4d98`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:46.348415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a816cf1c-41d0-4d07-8e98-25fe1323c093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.359246 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3848/7715] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:46 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:46.361007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816cf1c-41d0-4d07-8e98-25fe1323c093 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.370491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816cf1c-41d0-4d07-8e98-25fe1323c093 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.376214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '157333abd6394c17b1a68d27053d4d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c578d118a9804b5abe6a89b459f80171', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1287226637', 'tempest-DelegatedRole-463831331'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.376214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.376214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.376572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.376572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: Authorizing `identity:create_trust()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.377922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.378129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece375a4-bee8-42ae-9c72-a808185b365b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.381933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.382546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.382823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.383120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.394051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:46.453309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.454916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.464748 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3868/7716] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:46.480081 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3849/7717] 10.222.0.26 () {60 vars in 1174 bytes} [Sat Sep 17 21:20:46 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:46.484159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.497167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d90d09-646e-48dd-80cc-75debbb0210b service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.502406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '157333abd6394c17b1a68d27053d4d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c578d118a9804b5abe6a89b459f80171', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1287226637', 'tempest-DelegatedRole-463831331'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.503182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.503668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.503777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.504669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.506208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.513088 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3869/7718] 10.222.0.26 () {58 vars in 1252 bytes} [Sat Sep 17 21:20:46 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=157333abd6394c17b1a68d27053d4d98 => generated 611 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:46.517277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.528599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace768-c835-47dd-9a2c-7beff4b991de tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.533443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '157333abd6394c17b1a68d27053d4d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c578d118a9804b5abe6a89b459f80171', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1287226637', 'tempest-DelegatedRole-463831331'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.534230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.534550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.534856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] PATH_INFO: `/v3/OS-TRUST/trusts/4f6c01a2362945aeae87f24e3dd726ca` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.545922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: Authorizing `identity:delete_trust(trust_id=4f6c01a2362945aeae87f24e3dd726ca)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.547477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.574396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '4f6c01a2362945aeae87f24e3dd726ca', 'request_id': 'req-7f9ed4b5-8527-4610-a053-1e52a9ac2868'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:46.585852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '4f6c01a2362945aeae87f24e3dd726ca', 'request_id': 'req-7f9ed4b5-8527-4610-a053-1e52a9ac2868'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:46.588136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '4f6c01a2362945aeae87f24e3dd726ca', 'request_id': 'req-7f9ed4b5-8527-4610-a053-1e52a9ac2868'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:46.599379 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3850/7719] 10.222.0.26 () {58 vars in 1254 bytes} [Sat Sep 17 21:20:46 2022] DELETE /identity/v3/OS-TRUST/trusts/4f6c01a2362945aeae87f24e3dd726ca => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:46.603400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.605168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.618886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccc23d-c425-4157-8e41-6f6a1d8ccbaf tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.623301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '157333abd6394c17b1a68d27053d4d98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c578d118a9804b5abe6a89b459f80171', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1287226637', 'tempest-DelegatedRole-463831331'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.623974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.624249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.624515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] PATH_INFO: `/v3/OS-TRUST/trusts/4f6c01a2362945aeae87f24e3dd726ca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Could not find trust: 4f6c01a2362945aeae87f24e3dd726ca.: keystone.exception.TrustNotFound: Could not find trust: 4f6c01a2362945aeae87f24e3dd726ca. Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 17 21:20:46.635710 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 4f6c01a2362945aeae87f24e3dd726ca. Sep 17 21:20:46.637101 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:46.639047 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3870/7720] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:46 2022] GET /identity/v3/OS-TRUST/trusts/4f6c01a2362945aeae87f24e3dd726ca => generated 111 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:20:46.643440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.644360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.665004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ed4b5-8527-4610-a053-1e52a9ac2868 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.671329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.672082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.672380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.672769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/9081ea3f31be4286a071e9820aab81d0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.674415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=9081ea3f31be4286a071e9820aab81d0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.678653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.737174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role 9081ea3f31be4286a071e9820aab81d0 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:46.737526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:46.738212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:46.741721 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3851/7721] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:46 2022] DELETE /identity/v3/roles/9081ea3f31be4286a071e9820aab81d0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:46.746202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.748044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.771224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821578b9-9787-4eb9-b8bf-3456d1caa187 tempest-TrustsV3TestJSON-1624594868 tempest-user-190834460] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.846924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:46.847514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:46.847798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:46.848074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/adbcec2c74a9424eb0c4d7526ef83f2e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:46.850293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=adbcec2c74a9424eb0c4d7526ef83f2e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:46.851683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:46.889452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.891392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.902911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role adbcec2c74a9424eb0c4d7526ef83f2e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:46.903185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:46.905605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:46.906783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:46.908991 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3871/7722] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:46 2022] DELETE /identity/v3/roles/adbcec2c74a9424eb0c4d7526ef83f2e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:46.913110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:46.915111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:46.970956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabb393-cb6d-40cd-bf42-3d6d872a80aa tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:46.995611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc475b6-e273-4525-8415-d9f34bcbb661 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.001167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.001846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.002138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.002421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users/157333abd6394c17b1a68d27053d4d98` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.003571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_user(user_id=157333abd6394c17b1a68d27053d4d98)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.006214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.008315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:47.042784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.043589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.044129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.044555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] PATH_INFO: `/v3/users/0f64f10896b34f18a178fdb9e21df9f5` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.045954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f64f10896b34f18a178fdb9e21df9f5)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.048019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.061994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '157333abd6394c17b1a68d27053d4d98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.062697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '157333abd6394c17b1a68d27053d4d98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.071617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '157333abd6394c17b1a68d27053d4d98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.073584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '157333abd6394c17b1a68d27053d4d98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.087276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '157333abd6394c17b1a68d27053d4d98'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.102691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3872/7723] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:46 2022] DELETE /identity/v3/users/157333abd6394c17b1a68d27053d4d98 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:47.103536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f64f10896b34f18a178fdb9e21df9f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.106522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.108018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:47.130367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f64f10896b34f18a178fdb9e21df9f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.140862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f64f10896b34f18a178fdb9e21df9f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.141517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f64f10896b34f18a178fdb9e21df9f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.142185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802d9f45-798f-46d0-935f-1cfbe06a3071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f64f10896b34f18a178fdb9e21df9f5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.158442 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3852/7724] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:46 2022] DELETE /identity/v3/users/0f64f10896b34f18a178fdb9e21df9f5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:47.164309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.164610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.164899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.178719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a51306-d469-4203-a9b8-92d6336869ce tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.186446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:47.187671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:47.222855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:47.255837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.256537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.256809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.257072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/c578d118a9804b5abe6a89b459f80171` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.258648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c578d118a9804b5abe6a89b459f80171)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.260662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.280446 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3853/7725] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:47.290369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.294983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:47.317608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:47.348026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c578d118a9804b5abe6a89b459f80171', 'request_id': 'req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.357082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c578d118a9804b5abe6a89b459f80171', 'request_id': 'req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.377324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c578d118a9804b5abe6a89b459f80171', 'request_id': 'req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.392957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3e14-6d71-4933-8e83-9dc4b4a539b4 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.394449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] The token cache is being invalidate because project c578d118a9804b5abe6a89b459f80171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:47.394747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.395406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:47.397506 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3873/7726] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:47 2022] DELETE /identity/v3/projects/c578d118a9804b5abe6a89b459f80171 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:47.402631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.404172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:47.462151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.462785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.463066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.463338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.465124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:47.483857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88ebc1-57a9-49a0-b89f-ee8378772ae0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.490562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.491265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.491544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.491817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.492446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.495517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.529656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3874/7727] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:47 2022] POST /identity/v3/projects => generated 329 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:47.531216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.535589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.537341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.548451 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3854/7728] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:47.614039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b02bc-ef1c-4ea6-ad2b-e1f054e80a15 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.620843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.621618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.621814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.622156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.622926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.625254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.658307 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3875/7729] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:47 2022] POST /identity/v3/users => generated 413 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:47.659625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.674064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac8450-8302-4b2e-be52-3cbac325a2f8 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.683084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.683741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.684016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.684317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.685019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.686728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.714103 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3855/7730] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:47 2022] POST /identity/v3/roles => generated 243 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:47.718024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.744644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef5b5b-eff7-4ef2-a351-ebbd3df10679 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.752435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.753225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.753636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.754009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.754703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.756940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.775770 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3876/7731] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:47 2022] POST /identity/v3/roles => generated 247 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:47.780201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.791718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff214c-60b9-4a68-8b3d-ef6d206611e9 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.796661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.797391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.797777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.798063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/af450a22aa134c6cbae317318e2f1ad4/users/862ed61131b94da98bf94e95836cdb24/roles/8d0201f76ba141cd8f8d240632170358` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.817628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=af450a22aa134c6cbae317318e2f1ad4, user_id=862ed61131b94da98bf94e95836cdb24, role_id=8d0201f76ba141cd8f8d240632170358)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.820444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.838094 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3856/7732] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:47 2022] PUT /identity/v3/projects/af450a22aa134c6cbae317318e2f1ad4/users/862ed61131b94da98bf94e95836cdb24/roles/8d0201f76ba141cd8f8d240632170358 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:47.842062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.853290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106595c4-81bc-423f-96ed-927ed0cdf17b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:47.865822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=5,keystone:UPDATE=1,keystone:INSERT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:47.880269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:47.882575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:47.901122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:47.948518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:47.949386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:47.949808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:47.950083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/af450a22aa134c6cbae317318e2f1ad4/users/862ed61131b94da98bf94e95836cdb24/roles/2e2958bdf8bd460c9e400651b1839b1c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:47.952323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=af450a22aa134c6cbae317318e2f1ad4, user_id=862ed61131b94da98bf94e95836cdb24, role_id=2e2958bdf8bd460c9e400651b1839b1c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:47.955440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:47.971450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3877/7733] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:47 2022] PUT /identity/v3/projects/af450a22aa134c6cbae317318e2f1ad4/users/862ed61131b94da98bf94e95836cdb24/roles/2e2958bdf8bd460c9e400651b1839b1c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:47.975636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.004803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ec03e-2052-48cf-88b1-79b3a0156af8 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.048499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2878af8-5700-4f87-b30c-d77f50963304 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.053521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295e80e-6e02-4ff8-9eaa-fc9a59d3e020 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.054244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e80e-6e02-4ff8-9eaa-fc9a59d3e020 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.054527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e80e-6e02-4ff8-9eaa-fc9a59d3e020 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.054797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295e80e-6e02-4ff8-9eaa-fc9a59d3e020 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.055459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295e80e-6e02-4ff8-9eaa-fc9a59d3e020 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.057312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295e80e-6e02-4ff8-9eaa-fc9a59d3e020 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.070149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.071098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.071558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.071895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] PATH_INFO: `/v3/projects/d839cc8be70e44ae9fa4ee75b2d1d407` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.073822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d839cc8be70e44ae9fa4ee75b2d1d407)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.075405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.091820 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3878/7734] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:47 2022] GET /identity/v3/users => generated 8333 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:48.095857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86237fa-5104-4003-b412-158211ba39e1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.096141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86237fa-5104-4003-b412-158211ba39e1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.096411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86237fa-5104-4003-b412-158211ba39e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.128986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd839cc8be70e44ae9fa4ee75b2d1d407', 'request_id': 'req-bd61a777-1d91-4096-88b4-0af8d0f8913b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.138840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd839cc8be70e44ae9fa4ee75b2d1d407', 'request_id': 'req-bd61a777-1d91-4096-88b4-0af8d0f8913b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.141046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f86237fa-5104-4003-b412-158211ba39e1 None None] MFA Rules not processed for user `862ed61131b94da98bf94e95836cdb24`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:48.142176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f86237fa-5104-4003-b412-158211ba39e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:48.145777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd839cc8be70e44ae9fa4ee75b2d1d407', 'request_id': 'req-bd61a777-1d91-4096-88b4-0af8d0f8913b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.157745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] The token cache is being invalidate because project d839cc8be70e44ae9fa4ee75b2d1d407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:48.158126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.158856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.160672 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3857/7735] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:47 2022] DELETE /identity/v3/projects/d839cc8be70e44ae9fa4ee75b2d1d407 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:48.164589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.166051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:48.213491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3879/7736] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:48 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:48.221035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:48.253858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61a777-1d91-4096-88b4-0af8d0f8913b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.261282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.262132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.262485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.263610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] PATH_INFO: `/v3/users/03c5d28ef7594471b8fca4f63ec3cf32` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.266343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c5d28ef7594471b8fca4f63ec3cf32)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.269521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.309886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f006a-0a41-4491-b9d4-f055053254bd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.310200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f006a-0a41-4491-b9d4-f055053254bd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.310443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f006a-0a41-4491-b9d4-f055053254bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.332484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d25f006a-0a41-4491-b9d4-f055053254bd None None] MFA Rules not processed for user `862ed61131b94da98bf94e95836cdb24`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:48.334321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d25f006a-0a41-4491-b9d4-f055053254bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:48.338196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c5d28ef7594471b8fca4f63ec3cf32'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.343013 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3880/7737] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:48 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:48.348032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f006a-0a41-4491-b9d4-f055053254bd None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.353652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c5d28ef7594471b8fca4f63ec3cf32'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.357612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f006a-0a41-4491-b9d4-f055053254bd None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.362442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.363141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.364249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.364376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c5d28ef7594471b8fca4f63ec3cf32'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.364480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.364592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c5d28ef7594471b8fca4f63ec3cf32'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.365038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c5d28ef7594471b8fca4f63ec3cf32'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.365418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:create_trust()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.366738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.384031 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3858/7738] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:48 2022] DELETE /identity/v3/users/03c5d28ef7594471b8fca4f63ec3cf32 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:48.392637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:48.457152 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3881/7739] 10.222.0.26 () {60 vars in 1174 bytes} [Sat Sep 17 21:20:48 2022] POST /identity/v3/OS-TRUST/trusts => generated 858 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:48.462080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.464020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:48.548824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caa4ff-df66-4169-a8c0-44ef891d0f31 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.553120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.554338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.554620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.554894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.562982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:get_trust(trust_id=369baa9370a84b6399a283468f9c294a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.564446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.575459 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3859/7740] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:48 2022] GET /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a => generated 858 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:48.576764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.586376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9ac04-a01b-4763-9f11-927559faab60 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.590428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.591077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.591351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.591618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.598935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=369baa9370a84b6399a283468f9c294a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.600507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.608262 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3882/7741] 10.222.0.26 () {58 vars in 1269 bytes} [Sat Sep 17 21:20:48 2022] GET /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles => generated 370 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:48.612491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.624199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc023c3c-ead3-45f0-b658-c65ff849959b tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.628778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.629467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.629776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.630048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/8d0201f76ba141cd8f8d240632170358` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.636832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:get_role_for_trust(trust_id=369baa9370a84b6399a283468f9c294a, role_id=8d0201f76ba141cd8f8d240632170358)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.638311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.645694 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3860/7742] 10.222.0.26 () {58 vars in 1368 bytes} [Sat Sep 17 21:20:48 2022] GET /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/8d0201f76ba141cd8f8d240632170358 => generated 243 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:20:48.649573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.659348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3882b57-57b3-4a00-9e5e-3099d7cfd7ea tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.663632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.664502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.664635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.664810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/8d0201f76ba141cd8f8d240632170358` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.671125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:get_role_for_trust(trust_id=369baa9370a84b6399a283468f9c294a, role_id=8d0201f76ba141cd8f8d240632170358)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.672946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.679618 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3883/7743] 10.222.0.26 () {58 vars in 1369 bytes} [Sat Sep 17 21:20:48 2022] HEAD /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/8d0201f76ba141cd8f8d240632170358 => generated 0 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:48.683988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.694119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc9032-688c-4503-814d-770a6ba94e63 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.698662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.699306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.699594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.699859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/2e2958bdf8bd460c9e400651b1839b1c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.706734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:get_role_for_trust(trust_id=369baa9370a84b6399a283468f9c294a, role_id=2e2958bdf8bd460c9e400651b1839b1c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.709271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Could not find role: 2e2958bdf8bd460c9e400651b1839b1c.: keystone.exception.RoleNotFound: Could not find role: 2e2958bdf8bd460c9e400651b1839b1c. Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 21:20:48.720192 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2e2958bdf8bd460c9e400651b1839b1c. Sep 17 21:20:48.721675 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:48.722285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3861/7744] 10.222.0.26 () {58 vars in 1368 bytes} [Sat Sep 17 21:20:48 2022] GET /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/2e2958bdf8bd460c9e400651b1839b1c => generated 110 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:48.726849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.736569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038e995-0307-478c-8fcc-6bbdafb6b12a tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.740876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.741571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.741835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.742104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/2e2958bdf8bd460c9e400651b1839b1c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.748494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:get_role_for_trust(trust_id=369baa9370a84b6399a283468f9c294a, role_id=2e2958bdf8bd460c9e400651b1839b1c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.750138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Could not find role: 2e2958bdf8bd460c9e400651b1839b1c.: keystone.exception.RoleNotFound: Could not find role: 2e2958bdf8bd460c9e400651b1839b1c. Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 21:20:48.761893 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2e2958bdf8bd460c9e400651b1839b1c. Sep 17 21:20:48.763286 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:48.763286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3884/7745] 10.222.0.26 () {58 vars in 1369 bytes} [Sat Sep 17 21:20:48 2022] HEAD /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a/roles/2e2958bdf8bd460c9e400651b1839b1c => generated 0 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 17 21:20:48.767899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.779302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96d98a-caa0-4f4e-bdeb-04ccfb34e590 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.783771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.784549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.784843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.785095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.791453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorizing `identity:delete_trust(trust_id=369baa9370a84b6399a283468f9c294a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.792925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:48.799945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:48.813771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '369baa9370a84b6399a283468f9c294a', 'request_id': 'req-1129884b-5cc8-4674-ae42-b0bfed994f74'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.822135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '369baa9370a84b6399a283468f9c294a', 'request_id': 'req-1129884b-5cc8-4674-ae42-b0bfed994f74'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.822877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '369baa9370a84b6399a283468f9c294a', 'request_id': 'req-1129884b-5cc8-4674-ae42-b0bfed994f74'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:48.830113 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3862/7746] 10.222.0.26 () {58 vars in 1254 bytes} [Sat Sep 17 21:20:48 2022] DELETE /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:48.834916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.837575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:48.852445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d0917-8e03-4b08-bfb5-3040faaa1bb1 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.856830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862ed61131b94da98bf94e95836cdb24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af450a22aa134c6cbae317318e2f1ad4', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1018675175', 'tempest-DelegatedRole-533138260'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.857494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.857780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.858211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] PATH_INFO: `/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Could not find trust: 369baa9370a84b6399a283468f9c294a.: keystone.exception.TrustNotFound: Could not find trust: 369baa9370a84b6399a283468f9c294a. Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 17 21:20:48.867016 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 369baa9370a84b6399a283468f9c294a. Sep 17 21:20:48.868588 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:48.871860 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3885/7747] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:48 2022] GET /identity/v3/OS-TRUST/trusts/369baa9370a84b6399a283468f9c294a => generated 111 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:48.872786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.874280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:48.900659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:48.902992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:48.974654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129884b-5cc8-4674-ae42-b0bfed994f74 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:48.979498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:48.980140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:48.980420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:48.980822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/2e2958bdf8bd460c9e400651b1839b1c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:48.982642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2e2958bdf8bd460c9e400651b1839b1c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:48.984655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.014475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe01a7-23b9-44db-a672-7f22fa396ed9 tempest-TrustsV3TestJSON-1254140196 tempest-user-375873048] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.019512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.020851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.021330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.021509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] PATH_INFO: `/v3/projects/29154538c0374a67a0e730db3ceb5b25` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.022758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29154538c0374a67a0e730db3ceb5b25)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.024832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.045408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role 2e2958bdf8bd460c9e400651b1839b1c has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:49.046086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.047178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.050519 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3863/7748] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:48 2022] DELETE /identity/v3/roles/2e2958bdf8bd460c9e400651b1839b1c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:49.055548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.057024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.088066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29154538c0374a67a0e730db3ceb5b25', 'request_id': 'req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.091618 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:49.097871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29154538c0374a67a0e730db3ceb5b25', 'request_id': 'req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.110358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29154538c0374a67a0e730db3ceb5b25', 'request_id': 'req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.124406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] The token cache is being invalidate because project 29154538c0374a67a0e730db3ceb5b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:49.124899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.125680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ca7d2d2e-7ff6-42c4-ae7c-db221caad6f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.131854 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3886/7749] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:48 2022] DELETE /identity/v3/projects/29154538c0374a67a0e730db3ceb5b25 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:49.132457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240cf39-6f3e-492b-bd2f-b8f518c90eca tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.134534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ade40-79c3-48aa-94db-f00d418bd852 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.134826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ade40-79c3-48aa-94db-f00d418bd852 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.135098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ade40-79c3-48aa-94db-f00d418bd852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.159820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c86ade40-79c3-48aa-94db-f00d418bd852 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:49.162174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c86ade40-79c3-48aa-94db-f00d418bd852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.191591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:49.207633 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.208380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.208964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.209979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/8d0201f76ba141cd8f8d240632170358` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.211689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8d0201f76ba141cd8f8d240632170358)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.214959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.234321 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3887/7750] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:49.245512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11d47b-50a7-4cfb-a5d4-78e401e4e5f3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.245797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11d47b-50a7-4cfb-a5d4-78e401e4e5f3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.246139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11d47b-50a7-4cfb-a5d4-78e401e4e5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.268898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1f11d47b-50a7-4cfb-a5d4-78e401e4e5f3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:49.270236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f11d47b-50a7-4cfb-a5d4-78e401e4e5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.275750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role 8d0201f76ba141cd8f8d240632170358 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:49.276130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.277697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.281121 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3864/7751] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:49 2022] DELETE /identity/v3/roles/8d0201f76ba141cd8f8d240632170358 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:49.282682 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3888/7752] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:49.285436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.286345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11d47b-50a7-4cfb-a5d4-78e401e4e5f3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.292956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.297296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11d47b-50a7-4cfb-a5d4-78e401e4e5f3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.370651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40053e7e-5aca-4064-a9a0-b9331b5451c4 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.372621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.373277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.373585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.373860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.374545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:49.374850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.375711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.376413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.376517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.376761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.377106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users/862ed61131b94da98bf94e95836cdb24` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.378353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_user(user_id=862ed61131b94da98bf94e95836cdb24)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.380353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.391364 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3889/7753] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:49.395637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.397499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.411479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528ad3a-bdf2-401c-9cc9-2dbf2bd559dd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.416628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.417446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.417973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.418099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.418961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.421063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.451602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '862ed61131b94da98bf94e95836cdb24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.464744 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3890/7754] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:49 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:49.468602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.471528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '862ed61131b94da98bf94e95836cdb24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.484879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '862ed61131b94da98bf94e95836cdb24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.485755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '862ed61131b94da98bf94e95836cdb24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.487401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '862ed61131b94da98bf94e95836cdb24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.509299 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3865/7755] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:49 2022] DELETE /identity/v3/users/862ed61131b94da98bf94e95836cdb24 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:49.511008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.512910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.531656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:49.561488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c819d3-6038-4d97-b76c-700a349a9652 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.575689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:49.602384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb8b93-50ad-4c43-b8e4-b01cfeecc78d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.608110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.608759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.609041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.609316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/af450a22aa134c6cbae317318e2f1ad4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.610491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_project(project_id=af450a22aa134c6cbae317318e2f1ad4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.612462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.632889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.633584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.633861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.634129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.635267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.639667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.665597 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3891/7756] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:49 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:49.668152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af450a22aa134c6cbae317318e2f1ad4', 'request_id': 'req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.670649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.673565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.680549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af450a22aa134c6cbae317318e2f1ad4', 'request_id': 'req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.688123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca86386-6078-43bb-bf03-d58639cceb1c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.694445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af450a22aa134c6cbae317318e2f1ad4', 'request_id': 'req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.709207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] The token cache is being invalidate because project af450a22aa134c6cbae317318e2f1ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:49.710137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.711284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:49.713323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3866/7757] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:49 2022] DELETE /identity/v3/projects/af450a22aa134c6cbae317318e2f1ad4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:49.718894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.720415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.762010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.763094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.763409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.763656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.764363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.765969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.777244 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3892/7758] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:49.780986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.783084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.796455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c251a-61bb-4205-b69a-1ad609d91a02 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.801381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.802075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.802411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.802779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.803754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.805573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.810436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b9e8b-c357-483b-b9ab-40ceaeffa100 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.815726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.816479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.816824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.817113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.817627 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3893/7759] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:49.817864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.819910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.821999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.832981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a5604-68d2-48c3-aa90-49520b84ca74 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.838751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f2cc1-f1fa-407a-99bf-7f24b30fa481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:49.839497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f2cc1-f1fa-407a-99bf-7f24b30fa481 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.839796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f2cc1-f1fa-407a-99bf-7f24b30fa481 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.840098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f2cc1-f1fa-407a-99bf-7f24b30fa481 admin admin] PATH_INFO: `/v3/projects/2007b313194d4a2c8a266e8ca407bd34/users/b4d3effbde7243bf859a8bb58dc6924d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.853209 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3867/7760] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:49 2022] POST /identity/v3/projects => generated 328 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:49.854373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:49.857743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f2cc1-f1fa-407a-99bf-7f24b30fa481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2007b313194d4a2c8a266e8ca407bd34, user_id=b4d3effbde7243bf859a8bb58dc6924d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:49.860600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f2cc1-f1fa-407a-99bf-7f24b30fa481 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:49.878871 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3894/7761] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:49 2022] PUT /identity/v3/projects/2007b313194d4a2c8a266e8ca407bd34/users/b4d3effbde7243bf859a8bb58dc6924d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:49.891739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d373df-c2f8-446f-8e47-6872f98dae60 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:49.892058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d373df-c2f8-446f-8e47-6872f98dae60 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:49.892536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d373df-c2f8-446f-8e47-6872f98dae60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:49.913327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:49.924667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-66d373df-c2f8-446f-8e47-6872f98dae60 None None] MFA Rules not processed for user `b4d3effbde7243bf859a8bb58dc6924d`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:49.925867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-66d373df-c2f8-446f-8e47-6872f98dae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:49.928387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41800cf2-988c-4e2c-9816-1a87a1cad818 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:49.967984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:50.000803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.001424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.001732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.002003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.002653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.004577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.014822 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3895/7762] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:50.015884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d373df-c2f8-446f-8e47-6872f98dae60 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.028631 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3868/7763] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:49 2022] POST /identity/v3/users => generated 413 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:50.032751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.045756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36269b0-32c5-441d-b655-a0bfb2cd3965 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.051044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.051659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.051940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.053270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.053990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.055612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.072348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3869/7764] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/roles => generated 244 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:50.076422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.088936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa501af7-4d56-4d8a-b707-867502d463a3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.095385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.096219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.096505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.096680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d373df-c2f8-446f-8e47-6872f98dae60 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.096864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.097528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.099471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.101723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.102412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.102754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.103104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.103795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.105869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.129188 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3870/7765] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/roles => generated 246 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:50.130220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.139274 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3896/7766] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:50.143442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.144999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f9f39-0974-4fbe-8ba7-e25daf45e51f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.209828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.210419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.210702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.210979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/6a6bb41bf0494a66a2b6013528d577bd/users/8b0e0ce85ee24484a20d718df737dfcb/roles/e8b7182cf86d406fbbdbb289be119d49` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.219718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c57d06-a5c5-46ee-9508-3e7d95796169 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.225042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.225694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.225971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.226245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.226885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.228953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.229118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=6a6bb41bf0494a66a2b6013528d577bd, user_id=8b0e0ce85ee24484a20d718df737dfcb, role_id=e8b7182cf86d406fbbdbb289be119d49)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.231835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.246843 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3871/7767] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:50 2022] PUT /identity/v3/projects/6a6bb41bf0494a66a2b6013528d577bd/users/8b0e0ce85ee24484a20d718df737dfcb/roles/e8b7182cf86d406fbbdbb289be119d49 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:50.251112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.257840 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3897/7768] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:50.260960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.273731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68af35-062b-46a0-bd85-6ddc1d3a56f3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.302754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:50.326829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:50.327728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be58a94-f12a-41a5-902e-6ecb65ff7c3f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.333610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.335143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.335494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.336401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/6a6bb41bf0494a66a2b6013528d577bd/users/8b0e0ce85ee24484a20d718df737dfcb/roles/a67226da58734389aacc8cb4cf9ab8a1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.339834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.340544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.340888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.341347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.341447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=6a6bb41bf0494a66a2b6013528d577bd, user_id=8b0e0ce85ee24484a20d718df737dfcb, role_id=a67226da58734389aacc8cb4cf9ab8a1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.342457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.343847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.344228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.361175 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3898/7769] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:50 2022] GET /identity/v3/roles => generated 1892 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:50.361352 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3872/7770] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:50 2022] PUT /identity/v3/projects/6a6bb41bf0494a66a2b6013528d577bd/users/8b0e0ce85ee24484a20d718df737dfcb/roles/a67226da58734389aacc8cb4cf9ab8a1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:50.362745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.363686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.374245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a30a-c9a9-4e5b-9d9d-2f14ce211d22 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.448498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdaa1f-4a5b-4a33-8f96-cd5d68acd53e tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.454786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af2b69-56bd-469d-b86f-bc603a62a4bc tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.455480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af2b69-56bd-469d-b86f-bc603a62a4bc tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.455826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af2b69-56bd-469d-b86f-bc603a62a4bc tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.456171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af2b69-56bd-469d-b86f-bc603a62a4bc tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.456913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af2b69-56bd-469d-b86f-bc603a62a4bc tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.457838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.459122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af2b69-56bd-469d-b86f-bc603a62a4bc tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.459268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.459495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.459694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] PATH_INFO: `/v3/projects/152496f26d16469a9df94d5abe65ef6b/users/a157ec614b95412a820ead30ee3d8691/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.476265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=152496f26d16469a9df94d5abe65ef6b, user_id=a157ec614b95412a820ead30ee3d8691, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.479076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.498845 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3899/7771] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:50 2022] PUT /identity/v3/projects/152496f26d16469a9df94d5abe65ef6b/users/a157ec614b95412a820ead30ee3d8691/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:50.499923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.506333 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3873/7772] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:50 2022] GET /identity/v3/users => generated 8607 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:50.511500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d8f76-f825-4b11-a92b-618df47566c6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.511891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d8f76-f825-4b11-a92b-618df47566c6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.512290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d8f76-f825-4b11-a92b-618df47566c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.551555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b09d8f76-f825-4b11-a92b-618df47566c6 None None] MFA Rules not processed for user `8b0e0ce85ee24484a20d718df737dfcb`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:50.552818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b09d8f76-f825-4b11-a92b-618df47566c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:50.569210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4602ed2-25a6-4db0-9a3a-89d30a5a87dd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.574782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.575554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.575942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.576291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.577033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.578784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.593193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3900/7773] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:50 2022] GET /identity/v3/roles => generated 1892 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:50.594417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.605482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d96c5-9a6a-4ca8-8e7f-3457465cc329 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.613534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f8fab-9fca-4c75-8e52-a81bae748d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.614273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f8fab-9fca-4c75-8e52-a81bae748d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.614706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f8fab-9fca-4c75-8e52-a81bae748d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.615004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f8fab-9fca-4c75-8e52-a81bae748d48 admin admin] PATH_INFO: `/v3/projects/152496f26d16469a9df94d5abe65ef6b/users/a157ec614b95412a820ead30ee3d8691/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.617235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f8fab-9fca-4c75-8e52-a81bae748d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=152496f26d16469a9df94d5abe65ef6b, user_id=a157ec614b95412a820ead30ee3d8691, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.621029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f8fab-9fca-4c75-8e52-a81bae748d48 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.634469 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3874/7774] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:50.637828 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3901/7775] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:50 2022] PUT /identity/v3/projects/152496f26d16469a9df94d5abe65ef6b/users/a157ec614b95412a820ead30ee3d8691/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:50.644330 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dac6f1-ec6b-41fd-9c9b-37a24f90b71a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.644826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dac6f1-ec6b-41fd-9c9b-37a24f90b71a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.644961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dac6f1-ec6b-41fd-9c9b-37a24f90b71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.649400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.649722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.649981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.671189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-76dac6f1-ec6b-41fd-9c9b-37a24f90b71a None None] MFA Rules not processed for user `8b0e0ce85ee24484a20d718df737dfcb`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:50.672430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76dac6f1-ec6b-41fd-9c9b-37a24f90b71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:50.682056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] MFA Rules not processed for user `a157ec614b95412a820ead30ee3d8691`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:50.683220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:50.727350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:50.749354 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3875/7776] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:20:50.753020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dac6f1-ec6b-41fd-9c9b-37a24f90b71a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.759315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:50.763398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3902/7777] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:50.764296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dac6f1-ec6b-41fd-9c9b-37a24f90b71a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.770501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b0e0ce85ee24484a20d718df737dfcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a6bb41bf0494a66a2b6013528d577bd', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1051127837', 'tempest-NotDelegatedRole-546670521'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.771180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.771479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.771785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.772476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] RBAC: Authorizing `identity:create_trust()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.773691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.778382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.780238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:50.846645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf31eac-811f-49ca-ba6b-cf6a01462c86 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.850489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.851128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.851416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.851674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Sep 17 21:20:50.856451 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise ValueError(str(e)) Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Sep 17 21:20:50.858280 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Sep 17 21:20:50.859616 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Sep 17 21:20:50.859616 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Sep 17 21:20:50.859616 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Sep 17 21:20:50.859616 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Sep 17 21:20:50.859616 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 17 21:20:50.859616 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:50.859616 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3876/7778] 10.222.0.26 () {60 vars in 1174 bytes} [Sat Sep 17 21:20:50 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 107 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 17 21:20:50.862139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.863711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.864638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:50.875179 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3903/7779] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:50.933677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130935d-d589-42b9-87a4-5fedfe3f0fc7 tempest-TrustsV3TestJSON-912200408 tempest-user-172491890] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:50.938823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:50.939456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:50.939721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:50.939992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/a67226da58734389aacc8cb4cf9ab8a1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:50.941413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a67226da58734389aacc8cb4cf9ab8a1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:50.943194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:50.983797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role a67226da58734389aacc8cb4cf9ab8a1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:50.984190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:50.984859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:50.986952 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3877/7780] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:50 2022] DELETE /identity/v3/roles/a67226da58734389aacc8cb4cf9ab8a1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:50.991520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.003996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a0dd4-6e8c-4b00-958d-9ba8d8482c33 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.069695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.070648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.070937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.071215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/e8b7182cf86d406fbbdbb289be119d49` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.072665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=e8b7182cf86d406fbbdbb289be119d49)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.074399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.117705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role e8b7182cf86d406fbbdbb289be119d49 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:51.118758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.119382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.121497 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3904/7781] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:50 2022] DELETE /identity/v3/roles/e8b7182cf86d406fbbdbb289be119d49 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:51.125753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.127429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:51.141831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46dcd2-f5f6-496f-8ac1-cebb282a8cff tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.203618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.204375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.204517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.204813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users/8b0e0ce85ee24484a20d718df737dfcb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.206075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_user(user_id=8b0e0ce85ee24484a20d718df737dfcb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.208101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.221349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:51.260083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0e0ce85ee24484a20d718df737dfcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.273311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0e0ce85ee24484a20d718df737dfcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.281542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0e0ce85ee24484a20d718df737dfcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.282308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0e0ce85ee24484a20d718df737dfcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.282957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0e0ce85ee24484a20d718df737dfcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.296388 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3878/7782] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:51 2022] DELETE /identity/v3/users/8b0e0ce85ee24484a20d718df737dfcb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:51.300704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.302174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:51.316675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3fc52-f26b-4f8b-993a-7379137f241b tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.361191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:51.377925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.378602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.378878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.379150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/6a6bb41bf0494a66a2b6013528d577bd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.380302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6a6bb41bf0494a66a2b6013528d577bd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.382332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.434334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6bb41bf0494a66a2b6013528d577bd', 'request_id': 'req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.440693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6bb41bf0494a66a2b6013528d577bd', 'request_id': 'req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.450636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6bb41bf0494a66a2b6013528d577bd', 'request_id': 'req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.462842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] The token cache is being invalidate because project 6a6bb41bf0494a66a2b6013528d577bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:51.463149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.463806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:51.465371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3905/7783] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:51 2022] DELETE /identity/v3/projects/6a6bb41bf0494a66a2b6013528d577bd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:51.472987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.474563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:51.489215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799b513-5e0f-4038-bacb-d3acf490d95a tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.554063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.554576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.554850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.555119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.555759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.557759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.589208 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3879/7784] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:51 2022] POST /identity/v3/projects => generated 329 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:51.593375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.594867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:51.610010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8dc69-5cef-4e98-bb97-18cc5c0239c1 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.670215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.670848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.671154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.671799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.672218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.674256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.706127 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3906/7785] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:51 2022] POST /identity/v3/users => generated 416 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:51.710639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.722988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09149cc-b8bd-4bd0-9495-6536c7c3726f tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.728081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.729251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.729564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.729834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.730518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.732165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.750342 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3880/7786] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:51 2022] POST /identity/v3/roles => generated 243 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:51.754606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.769634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554008-be28-4aad-859d-d0b24dc714da tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.775585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.776293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.776653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.776992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.777887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.779575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.797516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3907/7787] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:51 2022] POST /identity/v3/roles => generated 246 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:51.801800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.814216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fdf68-b9da-46d9-8829-d226b87ec7fd tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.819346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.820548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.821006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.821438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/3ea67bef7c054c0bb5c6462fe80b31e7/users/1ec1753a90d74aa28824a416524c1880/roles/32cbe98393624a8ea28aff7f5706a709` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.840463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3ea67bef7c054c0bb5c6462fe80b31e7, user_id=1ec1753a90d74aa28824a416524c1880, role_id=32cbe98393624a8ea28aff7f5706a709)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.843327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.861145 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3881/7788] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:51 2022] PUT /identity/v3/projects/3ea67bef7c054c0bb5c6462fe80b31e7/users/1ec1753a90d74aa28824a416524c1880/roles/32cbe98393624a8ea28aff7f5706a709 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:51.865226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:51.878059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abbbd6-53f5-4f30-b625-d2d92d87a942 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:51.937153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:51.951243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb359c9-e33c-4fcd-8851-8b81a7f3648d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:51.952209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb359c9-e33c-4fcd-8851-8b81a7f3648d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:51.952979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb359c9-e33c-4fcd-8851-8b81a7f3648d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:51.953348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb359c9-e33c-4fcd-8851-8b81a7f3648d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/3ea67bef7c054c0bb5c6462fe80b31e7/users/1ec1753a90d74aa28824a416524c1880/roles/d1abce2c183c4e38841f24425d82c55d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:51.957282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb359c9-e33c-4fcd-8851-8b81a7f3648d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=3ea67bef7c054c0bb5c6462fe80b31e7, user_id=1ec1753a90d74aa28824a416524c1880, role_id=d1abce2c183c4e38841f24425d82c55d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:51.960647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb359c9-e33c-4fcd-8851-8b81a7f3648d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:51.979680 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3908/7789] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:51 2022] PUT /identity/v3/projects/3ea67bef7c054c0bb5c6462fe80b31e7/users/1ec1753a90d74aa28824a416524c1880/roles/d1abce2c183c4e38841f24425d82c55d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:51.984263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.008726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd15ec6-660c-4447-8d66-3e77f6d5dca5 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.069214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:52.112078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.113411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.113604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.114062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.114913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.120202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.159572 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3882/7790] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:51 2022] GET /identity/v3/users => generated 8610 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:52.165730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8368cf-6d6e-4f02-aefc-baebcd06a9e9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.166166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8368cf-6d6e-4f02-aefc-baebcd06a9e9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.166545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8368cf-6d6e-4f02-aefc-baebcd06a9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.167998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.170157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:52.232389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6e8368cf-6d6e-4f02-aefc-baebcd06a9e9 None None] MFA Rules not processed for user `1ec1753a90d74aa28824a416524c1880`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:52.233819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6e8368cf-6d6e-4f02-aefc-baebcd06a9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:52.276770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfaec4-7e4b-471f-96a0-3ab9c6a71df2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.281793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.282414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.282687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.282952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.285290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:52.340514 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3909/7791] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:52.351386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a6a6d-f0b7-4aa2-9c6f-f7dc40aeea4b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.351674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a6a6d-f0b7-4aa2-9c6f-f7dc40aeea4b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.352076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a6a6d-f0b7-4aa2-9c6f-f7dc40aeea4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.375363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c09a6a6d-f0b7-4aa2-9c6f-f7dc40aeea4b None None] MFA Rules not processed for user `1ec1753a90d74aa28824a416524c1880`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:52.377738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c09a6a6d-f0b7-4aa2-9c6f-f7dc40aeea4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:52.387289 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3910/7792] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:52.392533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a6a6d-f0b7-4aa2-9c6f-f7dc40aeea4b None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.394941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.397026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.402767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a6a6d-f0b7-4aa2-9c6f-f7dc40aeea4b None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.412733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.413372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.413718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.413988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.415666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:create_trust()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.418688 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3883/7793] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:20:52.418888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.515578 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3911/7794] 10.222.0.26 () {60 vars in 1174 bytes} [Sat Sep 17 21:20:52 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:20:52.519803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.532556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c12bc-5753-4273-8e5d-244644d087f4 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.541543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.542287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.542565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.542906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.550433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:get_trust(trust_id=815acee74d4f482aa3d85b8617ed80c4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.552015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.566400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.568044 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3884/7795] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:52 2022] GET /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4 => generated 833 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:52.579418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d19d3-0b56-4444-aa99-dc4adc189356 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.586596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.587350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.587693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.588211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.597090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=815acee74d4f482aa3d85b8617ed80c4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.599393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.619176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3912/7796] 10.222.0.26 () {58 vars in 1269 bytes} [Sat Sep 17 21:20:52 2022] GET /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles => generated 370 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:52.621429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.680733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139661a6-1ef4-4dee-a8b0-24da63f7238d tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.711169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.711869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.712137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.712405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/32cbe98393624a8ea28aff7f5706a709` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.722197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:get_role_for_trust(trust_id=815acee74d4f482aa3d85b8617ed80c4, role_id=32cbe98393624a8ea28aff7f5706a709)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.724340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.738971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.739115 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3885/7797] 10.222.0.26 () {58 vars in 1368 bytes} [Sat Sep 17 21:20:52 2022] GET /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/32cbe98393624a8ea28aff7f5706a709 => generated 243 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:52.754092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd79c9-af5d-458b-a65a-9c3617191f6b tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.767558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.768855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.775271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.776101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/32cbe98393624a8ea28aff7f5706a709` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.788594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:get_role_for_trust(trust_id=815acee74d4f482aa3d85b8617ed80c4, role_id=32cbe98393624a8ea28aff7f5706a709)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.797565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.829398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3913/7798] 10.222.0.26 () {58 vars in 1369 bytes} [Sat Sep 17 21:20:52 2022] HEAD /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/32cbe98393624a8ea28aff7f5706a709 => generated 0 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 17 21:20:52.833715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.849025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e14e1-722b-4ad2-8443-cd2da807b73e tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.863153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.864836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.865354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.865895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/d1abce2c183c4e38841f24425d82c55d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.873989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:get_role_for_trust(trust_id=815acee74d4f482aa3d85b8617ed80c4, role_id=d1abce2c183c4e38841f24425d82c55d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.875749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Could not find role: d1abce2c183c4e38841f24425d82c55d.: keystone.exception.RoleNotFound: Could not find role: d1abce2c183c4e38841f24425d82c55d. Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 21:20:52.887960 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: d1abce2c183c4e38841f24425d82c55d. Sep 17 21:20:52.889240 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:52.890024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3886/7799] 10.222.0.26 () {58 vars in 1368 bytes} [Sat Sep 17 21:20:52 2022] GET /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/d1abce2c183c4e38841f24425d82c55d => generated 110 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 17 21:20:52.893750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.906334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88cc8-bd81-4936-b6ba-d6e0f526d878 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.914569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.915345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.915723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.916051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/d1abce2c183c4e38841f24425d82c55d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.942711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:get_role_for_trust(trust_id=815acee74d4f482aa3d85b8617ed80c4, role_id=d1abce2c183c4e38841f24425d82c55d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:52.945682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:52.948808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Could not find role: d1abce2c183c4e38841f24425d82c55d.: keystone.exception.RoleNotFound: Could not find role: d1abce2c183c4e38841f24425d82c55d. Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 21:20:52.964386 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: d1abce2c183c4e38841f24425d82c55d. Sep 17 21:20:52.965717 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:52.971131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:52.971714 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3914/7800] 10.222.0.26 () {58 vars in 1369 bytes} [Sat Sep 17 21:20:52 2022] HEAD /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4/roles/d1abce2c183c4e38841f24425d82c55d => generated 0 bytes in 74 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 17 21:20:52.982575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7d8ce-5a43-424e-81fe-701f82245a62 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:52.987085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:52.987810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:52.988183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:52.988712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:52.997815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:53.006522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorizing `identity:delete_trust(trust_id=815acee74d4f482aa3d85b8617ed80c4)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:53.008745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:53.041630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '815acee74d4f482aa3d85b8617ed80c4', 'request_id': 'req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.051645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '815acee74d4f482aa3d85b8617ed80c4', 'request_id': 'req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.052533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '815acee74d4f482aa3d85b8617ed80c4', 'request_id': 'req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.059495 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3887/7801] 10.222.0.26 () {58 vars in 1254 bytes} [Sat Sep 17 21:20:52 2022] DELETE /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:53.063748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:53.066695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:53.085417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60de082-92d3-459d-b164-5887df1fce1a tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:53.090209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec1753a90d74aa28824a416524c1880', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ea67bef7c054c0bb5c6462fe80b31e7', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-726670204', 'tempest-NotDelegatedRole-764342526'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:53.090880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:53.091157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:53.091474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] PATH_INFO: `/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Could not find trust: 815acee74d4f482aa3d85b8617ed80c4.: keystone.exception.TrustNotFound: Could not find trust: 815acee74d4f482aa3d85b8617ed80c4. Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 17 21:20:53.101934 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 815acee74d4f482aa3d85b8617ed80c4. Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:53.103012 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3915/7802] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:53 2022] GET /identity/v3/OS-TRUST/trusts/815acee74d4f482aa3d85b8617ed80c4 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:53.107485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:53.108954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:53.142531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb2b92-e7b2-4a39-bc6d-9aace0f3a002 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:53.149582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:53.150240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:53.150512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:53.150788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/d1abce2c183c4e38841f24425d82c55d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:53.152431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d1abce2c183c4e38841f24425d82c55d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:53.154086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:53.210544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role d1abce2c183c4e38841f24425d82c55d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:53.210854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.211541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.214971 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3888/7803] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:53 2022] DELETE /identity/v3/roles/d1abce2c183c4e38841f24425d82c55d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:53.220441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:53.222875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:53.242144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400a5f8-a0a5-4ba8-a7dc-648bce255601 tempest-TrustsV3TestJSON-1911494201 tempest-user-2044238503] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:53.310733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:53.311652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:53.312126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:53.312406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/32cbe98393624a8ea28aff7f5706a709` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:53.313895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=32cbe98393624a8ea28aff7f5706a709)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:53.315787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:53.370885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role 32cbe98393624a8ea28aff7f5706a709 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:53.371219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.371968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.374275 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3916/7804] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:53 2022] DELETE /identity/v3/roles/32cbe98393624a8ea28aff7f5706a709 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:53.378441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:53.393014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b4bd3-bea9-4e1b-af5f-7f3c6a2d5997 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:53.499107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:53.500118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:53.500314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:53.500555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users/1ec1753a90d74aa28824a416524c1880` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:53.501930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1ec1753a90d74aa28824a416524c1880)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:53.504528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:53.577615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec1753a90d74aa28824a416524c1880'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.595982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec1753a90d74aa28824a416524c1880'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.610615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec1753a90d74aa28824a416524c1880'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.611306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec1753a90d74aa28824a416524c1880'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.612057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec1753a90d74aa28824a416524c1880'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.633306 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3889/7805] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:53 2022] DELETE /identity/v3/users/1ec1753a90d74aa28824a416524c1880 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:53.638363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:53.640029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:53.655923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec52de-b449-4b34-9ce8-6f453ba2f4ec tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:53.736702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:53.737669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:53.737950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:53.738269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/3ea67bef7c054c0bb5c6462fe80b31e7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:53.739622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3ea67bef7c054c0bb5c6462fe80b31e7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:53.741728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:53.819624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea67bef7c054c0bb5c6462fe80b31e7', 'request_id': 'req-74ae4bfa-2937-41ff-b498-1ed7b7466fea'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.829727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea67bef7c054c0bb5c6462fe80b31e7', 'request_id': 'req-74ae4bfa-2937-41ff-b498-1ed7b7466fea'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.832035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:UPDATE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:53.843449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea67bef7c054c0bb5c6462fe80b31e7', 'request_id': 'req-74ae4bfa-2937-41ff-b498-1ed7b7466fea'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.864411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] The token cache is being invalidate because project 3ea67bef7c054c0bb5c6462fe80b31e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:53.865034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.866641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:53.869767 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3917/7806] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:53 2022] DELETE /identity/v3/projects/3ea67bef7c054c0bb5c6462fe80b31e7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:53.874889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:53.876335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:53.892839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18ff6a-a6df-4ec7-beab-eb4debfacbe3 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:53.899347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=14 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:53.977799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:53.978976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:53.979213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:53.979589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:53.981730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:53.984044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.018761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3890/7807] 10.222.0.26 () {60 vars in 1152 bytes} [Sat Sep 17 21:20:53 2022] POST /identity/v3/projects => generated 329 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:54.023103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.025589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:54.042816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae4bfa-2937-41ff-b498-1ed7b7466fea tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.121086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.121805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.122091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.122469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.124643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.127209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.152018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3918/7808] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:54 2022] POST /identity/v3/users => generated 413 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:54.156190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.168045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19281b-8e87-4fb9-a676-f6a65a178b90 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.173264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.174550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.174835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.175195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.175926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.177639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.197418 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3891/7809] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:54 2022] POST /identity/v3/roles => generated 243 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:54.201753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.214629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424adf-454c-439d-82df-ec547a208992 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.220909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.221641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.221886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.222162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.222836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.224636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.242149 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3919/7810] 10.222.0.26 () {60 vars in 1143 bytes} [Sat Sep 17 21:20:54 2022] POST /identity/v3/roles => generated 246 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:54.250782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.268651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce01cf-9015-4fe7-86b4-b816f48f252d tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.275306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.276115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.276471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.276818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/724299b7c7aa4f6e8aa4913a17912c23/users/f8d741796b954eed80ce04d5921140ba/roles/c239897369424f3592c11cb58d57f2d6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.302776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=724299b7c7aa4f6e8aa4913a17912c23, user_id=f8d741796b954eed80ce04d5921140ba, role_id=c239897369424f3592c11cb58d57f2d6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.307528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.326533 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3892/7811] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:54 2022] PUT /identity/v3/projects/724299b7c7aa4f6e8aa4913a17912c23/users/f8d741796b954eed80ce04d5921140ba/roles/c239897369424f3592c11cb58d57f2d6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:54.331840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.346290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba5902-116f-4623-be9e-018b138a5cd2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.415434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a743c1-5933-4ec9-84ea-4e2350bbceae tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.416290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a743c1-5933-4ec9-84ea-4e2350bbceae tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.416599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a743c1-5933-4ec9-84ea-4e2350bbceae tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.416959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a743c1-5933-4ec9-84ea-4e2350bbceae tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/724299b7c7aa4f6e8aa4913a17912c23/users/f8d741796b954eed80ce04d5921140ba/roles/2bcc3bec2eab4c4481a2619fb5d545d1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.419476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a743c1-5933-4ec9-84ea-4e2350bbceae tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:create_grant(project_id=724299b7c7aa4f6e8aa4913a17912c23, user_id=f8d741796b954eed80ce04d5921140ba, role_id=2bcc3bec2eab4c4481a2619fb5d545d1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.423141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a743c1-5933-4ec9-84ea-4e2350bbceae tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.439507 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3920/7812] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:54 2022] PUT /identity/v3/projects/724299b7c7aa4f6e8aa4913a17912c23/users/f8d741796b954eed80ce04d5921140ba/roles/2bcc3bec2eab4c4481a2619fb5d545d1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:54.443771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.456571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16cfdc-a1e6-4e38-a975-c8e773d73d97 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.529072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020801c-908d-4b50-b2aa-c632b5170ff0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.529828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020801c-908d-4b50-b2aa-c632b5170ff0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.530174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020801c-908d-4b50-b2aa-c632b5170ff0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.530511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020801c-908d-4b50-b2aa-c632b5170ff0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.531382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020801c-908d-4b50-b2aa-c632b5170ff0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.533408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020801c-908d-4b50-b2aa-c632b5170ff0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.564247 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3893/7813] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:54 2022] GET /identity/v3/users => generated 8607 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:20:54.569845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd56d-f8f3-4669-a2bc-854190157c0e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.570211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd56d-f8f3-4669-a2bc-854190157c0e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.570557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd56d-f8f3-4669-a2bc-854190157c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.611094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a1ecd56d-f8f3-4669-a2bc-854190157c0e None None] MFA Rules not processed for user `f8d741796b954eed80ce04d5921140ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:54.612322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a1ecd56d-f8f3-4669-a2bc-854190157c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:54.650872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=6,keystone:DELETE=3,keystone:SELECT=91 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:54.691421 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3921/7814] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:54.702146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3dda-6ecc-4583-bd9b-4ded0e704286 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.702146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3dda-6ecc-4583-bd9b-4ded0e704286 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.702146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec3dda-6ecc-4583-bd9b-4ded0e704286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.722085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-58ec3dda-6ecc-4583-bd9b-4ded0e704286 None None] MFA Rules not processed for user `f8d741796b954eed80ce04d5921140ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:54.723309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-58ec3dda-6ecc-4583-bd9b-4ded0e704286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:54.731096 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3894/7815] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:54.735432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecd56d-f8f3-4669-a2bc-854190157c0e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.745133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecd56d-f8f3-4669-a2bc-854190157c0e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.749483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.750097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.750378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.750647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.751375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:create_trust()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.752590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.836709 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3922/7816] 10.222.0.26 () {60 vars in 1174 bytes} [Sat Sep 17 21:20:54 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:54.840947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec3dda-6ecc-4583-bd9b-4ded0e704286 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.850758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec3dda-6ecc-4583-bd9b-4ded0e704286 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.856157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.856829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.857158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.857644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.865968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:get_trust(trust_id=cab69d867bf4465fa73d9d3de4ab691b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.867664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.881048 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3895/7817] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:54 2022] GET /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b => generated 834 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:54.882936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.895727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020b2d-1b38-491e-8bd1-294cb3c18d8d tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.902921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.904329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.905173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.905656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.914515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=cab69d867bf4465fa73d9d3de4ab691b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.916450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.928295 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3923/7818] 10.222.0.26 () {58 vars in 1269 bytes} [Sat Sep 17 21:20:54 2022] GET /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles => generated 370 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:54.932394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.944900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1a762-92ca-4860-a597-94a8c3abf1da tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.949705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.950466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.950830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.951471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/c239897369424f3592c11cb58d57f2d6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.958924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:get_role_for_trust(trust_id=cab69d867bf4465fa73d9d3de4ab691b, role_id=c239897369424f3592c11cb58d57f2d6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.960503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:54.968967 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3896/7819] 10.222.0.26 () {58 vars in 1368 bytes} [Sat Sep 17 21:20:54 2022] GET /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/c239897369424f3592c11cb58d57f2d6 => generated 243 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:54.972938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:54.983053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae77ec-7cff-4a2b-bc73-ffbab75e13e1 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:54.987952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:54.988630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:54.988901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:54.989174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/c239897369424f3592c11cb58d57f2d6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:54.996656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:get_role_for_trust(trust_id=cab69d867bf4465fa73d9d3de4ab691b, role_id=c239897369424f3592c11cb58d57f2d6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:54.998109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.006891 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3924/7820] 10.222.0.26 () {58 vars in 1369 bytes} [Sat Sep 17 21:20:54 2022] HEAD /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/c239897369424f3592c11cb58d57f2d6 => generated 0 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 17 21:20:55.011646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.022024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbaf16-bcac-4293-b05c-ecbf3c5440f8 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.026888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.027611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.027923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.028220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/2bcc3bec2eab4c4481a2619fb5d545d1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.033825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:55.036645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:get_role_for_trust(trust_id=cab69d867bf4465fa73d9d3de4ab691b, role_id=2bcc3bec2eab4c4481a2619fb5d545d1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.039030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Could not find role: 2bcc3bec2eab4c4481a2619fb5d545d1.: keystone.exception.RoleNotFound: Could not find role: 2bcc3bec2eab4c4481a2619fb5d545d1. Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 21:20:55.057944 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2bcc3bec2eab4c4481a2619fb5d545d1. Sep 17 21:20:55.059248 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:20:55.061617 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3897/7821] 10.222.0.26 () {58 vars in 1368 bytes} [Sat Sep 17 21:20:55 2022] GET /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/2bcc3bec2eab4c4481a2619fb5d545d1 => generated 110 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:55.063663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.074406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf6b1-f200-480f-8dbf-8da897069f66 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.078889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.079588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `HEAD` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.079869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.080142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/2bcc3bec2eab4c4481a2619fb5d545d1` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.087938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:get_role_for_trust(trust_id=cab69d867bf4465fa73d9d3de4ab691b, role_id=2bcc3bec2eab4c4481a2619fb5d545d1)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.089359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Could not find role: 2bcc3bec2eab4c4481a2619fb5d545d1.: keystone.exception.RoleNotFound: Could not find role: 2bcc3bec2eab4c4481a2619fb5d545d1. Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 21:20:55.100431 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2bcc3bec2eab4c4481a2619fb5d545d1. Sep 17 21:20:55.105252 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:55.105252 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3925/7822] 10.222.0.26 () {58 vars in 1369 bytes} [Sat Sep 17 21:20:55 2022] HEAD /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b/roles/2bcc3bec2eab4c4481a2619fb5d545d1 => generated 0 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:55.106226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.116509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484c4c9-7b4b-4e20-a03a-8995b1c2e967 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.122209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.122855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.123134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.123413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.130445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorizing `identity:delete_trust(trust_id=cab69d867bf4465fa73d9d3de4ab691b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.132418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.167595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'cab69d867bf4465fa73d9d3de4ab691b', 'request_id': 'req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.183492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'cab69d867bf4465fa73d9d3de4ab691b', 'request_id': 'req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.184177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'cab69d867bf4465fa73d9d3de4ab691b', 'request_id': 'req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.191669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3898/7823] 10.222.0.26 () {58 vars in 1254 bytes} [Sat Sep 17 21:20:55 2022] DELETE /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:55.195847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.197870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:55.219781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940cc8a-c06a-4424-8024-972fa6585675 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.226226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8d741796b954eed80ce04d5921140ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724299b7c7aa4f6e8aa4913a17912c23', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-452587629', 'tempest-NotDelegatedRole-858490022'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.227048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.227325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.227634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] PATH_INFO: `/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Could not find trust: cab69d867bf4465fa73d9d3de4ab691b.: keystone.exception.TrustNotFound: Could not find trust: cab69d867bf4465fa73d9d3de4ab691b. Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 17 21:20:55.240350 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: cab69d867bf4465fa73d9d3de4ab691b. Sep 17 21:20:55.241435 ncdv-master-1 devstack@keystone.service[99167]: ERROR keystone.server.flask.application Sep 17 21:20:55.242612 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3926/7824] 10.222.0.26 () {58 vars in 1251 bytes} [Sat Sep 17 21:20:55 2022] GET /identity/v3/OS-TRUST/trusts/cab69d867bf4465fa73d9d3de4ab691b => generated 111 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:20:55.246385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.248844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:55.269158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02ddb3-a2d8-48a3-95ee-a8de829b26e5 tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.275237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.275989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.276372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.277012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/2bcc3bec2eab4c4481a2619fb5d545d1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.281107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2bcc3bec2eab4c4481a2619fb5d545d1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.283528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.342066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role 2bcc3bec2eab4c4481a2619fb5d545d1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:55.342602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.344175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.350158 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3899/7825] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:55 2022] DELETE /identity/v3/roles/2bcc3bec2eab4c4481a2619fb5d545d1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:55.354137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.355974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:55.372550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebe813-9f95-4cc6-af7b-ef3c66dfee3e tempest-TrustsV3TestJSON-1721878413 tempest-user-494155755] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.459050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.459713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.459990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.460260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/roles/c239897369424f3592c11cb58d57f2d6` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.463174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c239897369424f3592c11cb58d57f2d6)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.464961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.477032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:55.485596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.488251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:55.532560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invalidating the token cache because role c239897369424f3592c11cb58d57f2d6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:55.533039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.534105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.537272 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3927/7826] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:55 2022] DELETE /identity/v3/roles/c239897369424f3592c11cb58d57f2d6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:55.541311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.543234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:55.586269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec924-548f-4945-9b07-72b21b12c5d0 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.677293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80bf90-306b-402d-a383-7369d4ac2266 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.683406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.684424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.684857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.685285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/users/f8d741796b954eed80ce04d5921140ba` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.696661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f8d741796b954eed80ce04d5921140ba)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.699232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.702369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.703128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.703665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.704030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.711407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:55.767415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d741796b954eed80ce04d5921140ba'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.768734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d741796b954eed80ce04d5921140ba'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.782132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d741796b954eed80ce04d5921140ba'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.782981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d741796b954eed80ce04d5921140ba'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.796051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:55.805564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d741796b954eed80ce04d5921140ba'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:55.829922 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3928/7827] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:55 2022] DELETE /identity/v3/users/f8d741796b954eed80ce04d5921140ba => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:55.835301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:55.838775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:55.857566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.859702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.881405 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3900/7828] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:55.905205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:55.924899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80242f-6391-4d8b-a4ee-dbeece7c90a2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:55.931190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbd741eb9584de7b6a4644413992b48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e41796951b9f48cf98f29c2f03491a35', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:55.932205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:55.932205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:55.932484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] PATH_INFO: `/v3/projects/724299b7c7aa4f6e8aa4913a17912c23` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:55.933755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorizing `identity:delete_project(project_id=724299b7c7aa4f6e8aa4913a17912c23)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:55.935751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:55.999320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724299b7c7aa4f6e8aa4913a17912c23', 'request_id': 'req-938285a8-be99-42a3-9e7c-3056e9b3cbc2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.008327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724299b7c7aa4f6e8aa4913a17912c23', 'request_id': 'req-938285a8-be99-42a3-9e7c-3056e9b3cbc2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.018881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724299b7c7aa4f6e8aa4913a17912c23', 'request_id': 'req-938285a8-be99-42a3-9e7c-3056e9b3cbc2'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.035429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] The token cache is being invalidate because project 724299b7c7aa4f6e8aa4913a17912c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:56.035778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.038156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-938285a8-be99-42a3-9e7c-3056e9b3cbc2 tempest-TrustsV3TestJSON-686711809 tempest-TrustsV3TestJSON-686711809-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.046918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3929/7829] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:55 2022] DELETE /identity/v3/projects/724299b7c7aa4f6e8aa4913a17912c23 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:56.047201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:56.049536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:56.138650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05919ca1-ec4f-4e8c-b45e-7945596c73af service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:56.206923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:56.207786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:56.208053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:56.208394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] PATH_INFO: `/v3/users/cb68671c40134a90bfb149f064c03906` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:56.210406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb68671c40134a90bfb149f064c03906)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:56.213427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:56.283344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb68671c40134a90bfb149f064c03906'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.297997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb68671c40134a90bfb149f064c03906'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.313156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb68671c40134a90bfb149f064c03906'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.314187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb68671c40134a90bfb149f064c03906'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.314940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb68671c40134a90bfb149f064c03906'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:56.326112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:56.345554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3901/7830] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:56 2022] DELETE /identity/v3/users/cb68671c40134a90bfb149f064c03906 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:20:56.363096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:56.368916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:56.368916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:56.402533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:56.404208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:56.487441 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3930/7831] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:56.500360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:56.502100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:56.616938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c77c64-dbff-4680-8b88-b468065dd929 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:56.622469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:56.623772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:56.623772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:56.624020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:56.635610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:56.637218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:56.649261 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3902/7832] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:57.248620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:57.250105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:57.270768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3865-5b77-42d3-8627-97ba94c0cb31 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:57.286426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:57.287222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:57.287503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:57.287772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] PATH_INFO: `/v3/projects/23f41e28c7ee40429bb41120fec3db79` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:57.290898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f41e28c7ee40429bb41120fec3db79)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:57.295814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:57.398160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f41e28c7ee40429bb41120fec3db79', 'request_id': 'req-004ef15f-3a73-49a3-aa31-2da7ebd26826'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.407298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f41e28c7ee40429bb41120fec3db79', 'request_id': 'req-004ef15f-3a73-49a3-aa31-2da7ebd26826'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.422994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f41e28c7ee40429bb41120fec3db79', 'request_id': 'req-004ef15f-3a73-49a3-aa31-2da7ebd26826'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.440823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:57.454643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] The token cache is being invalidate because project 23f41e28c7ee40429bb41120fec3db79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:57.455793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.459258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.461385 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3931/7833] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:57 2022] DELETE /identity/v3/projects/23f41e28c7ee40429bb41120fec3db79 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:57.467548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:57.486967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2552fe-f4fe-4b46-b919-0557ccc17eef service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:57.600855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:57.601684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:57.601869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:57.602400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] PATH_INFO: `/v3/users/0fbd741eb9584de7b6a4644413992b48` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:57.603887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbd741eb9584de7b6a4644413992b48)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:57.608599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:57.697615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbd741eb9584de7b6a4644413992b48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.754664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbd741eb9584de7b6a4644413992b48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.777646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbd741eb9584de7b6a4644413992b48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.778846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbd741eb9584de7b6a4644413992b48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.780227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-866a3668-cad0-4337-8c9a-238a74d36891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbd741eb9584de7b6a4644413992b48'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:57.813207 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3903/7834] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:20:57 2022] DELETE /identity/v3/users/0fbd741eb9584de7b6a4644413992b48 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:58.580550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:58.582355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:58.617196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ef15f-3a73-49a3-aa31-2da7ebd26826 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:58.701562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:58.701562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:58.701562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:58.702181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] PATH_INFO: `/v3/projects/e41796951b9f48cf98f29c2f03491a35` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:58.709433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41796951b9f48cf98f29c2f03491a35)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:58.709433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:58.820392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41796951b9f48cf98f29c2f03491a35', 'request_id': 'req-93516f1b-c2ef-4898-9adb-2e32cab40dbe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:58.827840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41796951b9f48cf98f29c2f03491a35', 'request_id': 'req-93516f1b-c2ef-4898-9adb-2e32cab40dbe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:58.846166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41796951b9f48cf98f29c2f03491a35', 'request_id': 'req-93516f1b-c2ef-4898-9adb-2e32cab40dbe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:58.868555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] The token cache is being invalidate because project e41796951b9f48cf98f29c2f03491a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:20:58.870373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:58.871244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-93516f1b-c2ef-4898-9adb-2e32cab40dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:20:58.880109 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3932/7835] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:20:58 2022] DELETE /identity/v3/projects/e41796951b9f48cf98f29c2f03491a35 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:58.883322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7a3b2-5c7e-42a5-8f51-f519cda6eaad None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:58.884784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7a3b2-5c7e-42a5-8f51-f519cda6eaad None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:58.885220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7a3b2-5c7e-42a5-8f51-f519cda6eaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:58.920858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ffd7a3b2-5c7e-42a5-8f51-f519cda6eaad None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:58.924174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ffd7a3b2-5c7e-42a5-8f51-f519cda6eaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:58.946485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:59.057216 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3904/7836] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:59.064910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85787-7e14-4703-9eca-beff99689c0a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.065769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85787-7e14-4703-9eca-beff99689c0a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.066130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85787-7e14-4703-9eca-beff99689c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.117005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-caf85787-7e14-4703-9eca-beff99689c0a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:59.118706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-caf85787-7e14-4703-9eca-beff99689c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:59.137339 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3933/7837] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:59.138341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7a3b2-5c7e-42a5-8f51-f519cda6eaad None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.151321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7a3b2-5c7e-42a5-8f51-f519cda6eaad None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:59.157447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:59.158333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.158681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.159163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.159951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:20:59.160296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:59.162105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:59.177756 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3905/7838] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:20:59.182588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85787-7e14-4703-9eca-beff99689c0a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.193697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85787-7e14-4703-9eca-beff99689c0a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:59.201769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:59.203546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.203895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.204783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.204906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:59.207494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:59.289656 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3934/7839] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:20:59 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:59.295143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.319188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f105183-bfb0-48cd-ab62-8d1f51b51258 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:59.411005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:59.412228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.412228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.412439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.413082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:59.415424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:59.453805 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3906/7840] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:20:59 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:59.462499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.489752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3671eef-585f-480a-8f7f-a192b9cd53e3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:59.503210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:59.504995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.506149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.506679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.507562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:59.509676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:59.527460 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3935/7841] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:59.530483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.553866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a8882-7bba-43cf-ab0b-3fbebfb969fb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:59.568466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20fa41-ad78-40d9-919a-91f23e573547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:59.569487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20fa41-ad78-40d9-919a-91f23e573547 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.570194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20fa41-ad78-40d9-919a-91f23e573547 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.570682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20fa41-ad78-40d9-919a-91f23e573547 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.571565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20fa41-ad78-40d9-919a-91f23e573547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:59.573448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20fa41-ad78-40d9-919a-91f23e573547 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:59.586873 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3907/7842] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:20:59.591788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.602761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31675047-edbe-4a6f-bacc-e6abb1d5513e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:59.607520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5445e-2dc0-4f71-b585-d54776a63149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:59.608384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5445e-2dc0-4f71-b585-d54776a63149 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.608654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5445e-2dc0-4f71-b585-d54776a63149 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.608921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5445e-2dc0-4f71-b585-d54776a63149 admin admin] PATH_INFO: `/v3/projects/c46c200677a54988835e2fc6316e74f3/users/a30a76c2645249c0b65ba7c9e84c7ae7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.633177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5445e-2dc0-4f71-b585-d54776a63149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46c200677a54988835e2fc6316e74f3, user_id=a30a76c2645249c0b65ba7c9e84c7ae7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:59.636366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5445e-2dc0-4f71-b585-d54776a63149 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:59.654345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f32c27-7384-48b8-af52-52bf18735223 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.654999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f32c27-7384-48b8-af52-52bf18735223 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.655449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f32c27-7384-48b8-af52-52bf18735223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.659521 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3936/7843] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:20:59 2022] PUT /identity/v3/projects/c46c200677a54988835e2fc6316e74f3/users/a30a76c2645249c0b65ba7c9e84c7ae7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:20:59.671311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.671747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.672081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.699654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-87f32c27-7384-48b8-af52-52bf18735223 None None] MFA Rules not processed for user `2fc6b11f473e4231adf4278688619f24`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:59.703665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87f32c27-7384-48b8-af52-52bf18735223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:59.708123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] MFA Rules not processed for user `a30a76c2645249c0b65ba7c9e84c7ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:20:59.709440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:59.722589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:59.819195 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3908/7844] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:59.827495 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3937/7845] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:20:59.832505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f32c27-7384-48b8-af52-52bf18735223 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.843825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f32c27-7384-48b8-af52-52bf18735223 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:59.848470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:59.849109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:59.849386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:59.849729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:59.850402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:create_application_credential(user_id=a30a76c2645249c0b65ba7c9e84c7ae7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:59.851581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:59.889810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:59.947193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:59.948868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:59.984378 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3909/7846] 10.222.0.26 () {60 vars in 1315 bytes} [Sat Sep 17 21:20:59 2022] POST /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials => generated 921 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:20:59.988614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.000668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e12e91-e953-4fb6-a5bc-06dafa1d78f3 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.005531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.006384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.006673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.006955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules/2a6dcb3e061e41419ca1c26d098c99d9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.008122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:delete_access_rule(user_id=a30a76c2645249c0b65ba7c9e84c7ae7, access_rule_id=2a6dcb3e061e41419ca1c26d098c99d9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.009812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.051761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc6f8a-6455-4012-b0fe-82be075a09bc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.060058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.060690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.060963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.061262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.064219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application session.delete(ref) Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application yield resource Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application next(self.gen) Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 21:21:00.066707 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise self.value Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application session.commit() Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.session.flush() Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 21:21:00.067888 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application compat.raise_( Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self._synchronize( Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise AssertionError( Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Sep 17 21:21:00.069652 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:21:00.071384 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3910/7847] 10.222.0.26 () {58 vars in 1362 bytes} [Sat Sep 17 21:20:59 2022] DELETE /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules/2a6dcb3e061e41419ca1c26d098c99d9 => generated 89 bytes in 80 msecs (HTTP/1.1 403) 5 headers in 183 bytes (2 switches on core 0) Sep 17 21:21:00.073093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.082080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567949ee-0959-4b2e-8fed-0c2ddf65e4a6 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.086842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.087654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.087937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.088213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials/77915d08af484e46997e0e2dc91499a9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.110693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.112320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.115591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a30a76c2645249c0b65ba7c9e84c7ae7, application_credential_id=77915d08af484e46997e0e2dc91499a9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.117124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.151950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3938/7848] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:21:00.157978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.159496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:00.197012 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3911/7849] 10.222.0.26 () {58 vars in 1395 bytes} [Sat Sep 17 21:21:00 2022] DELETE /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials/77915d08af484e46997e0e2dc91499a9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:00.202196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.225509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609166aa-cf65-4b02-9a61-6559ed00dc28 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.231013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.231672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.231933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.232397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.233759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:list_access_rules(user_id=a30a76c2645249c0b65ba7c9e84c7ae7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.235928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.248233 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3912/7850] 10.222.0.26 () {58 vars in 1260 bytes} [Sat Sep 17 21:21:00 2022] GET /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules => generated 369 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:00.251814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.288875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4a3d9-f30f-4831-9bf8-e29b21f6c30c tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.294516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.295195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.295469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.295749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules/2a6dcb3e061e41419ca1c26d098c99d9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.296985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:delete_access_rule(user_id=a30a76c2645249c0b65ba7c9e84c7ae7, access_rule_id=2a6dcb3e061e41419ca1c26d098c99d9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.298485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.335969 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3913/7851] 10.222.0.26 () {58 vars in 1362 bytes} [Sat Sep 17 21:21:00 2022] DELETE /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules/2a6dcb3e061e41419ca1c26d098c99d9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:00.341350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.360402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdc16e-ae48-400d-a5e8-1e4c08dcb1b2 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.369927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc61de-3bc5-4a2a-8228-99b0d34938ea service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.372021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.372694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.372975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.373249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.374842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:list_access_rules(user_id=a30a76c2645249c0b65ba7c9e84c7ae7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.376392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.386448 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3914/7852] 10.222.0.26 () {58 vars in 1260 bytes} [Sat Sep 17 21:21:00 2022] GET /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules => generated 158 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:21:00.387095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a157ec614b95412a820ead30ee3d8691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '152496f26d16469a9df94d5abe65ef6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.387732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.388011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.388281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.388940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:00.389243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.391269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.394035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.412434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45294b5-6216-4a88-90c8-b9a91ad67531 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.420193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3939/7853] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:00.424704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.431882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.432555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.432837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.433111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.434377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:create_application_credential(user_id=a30a76c2645249c0b65ba7c9e84c7ae7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.435005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.448227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea024b-abe1-481a-81d2-91b1ac2ecdcb tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.461378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a157ec614b95412a820ead30ee3d8691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '152496f26d16469a9df94d5abe65ef6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.462035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.462365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.462643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.463286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.476332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.577125 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3940/7854] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:00 2022] POST /identity/v3/projects => generated 348 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:00.578437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.617405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:00.660754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3915/7855] 10.222.0.26 () {60 vars in 1315 bytes} [Sat Sep 17 21:21:00 2022] POST /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials => generated 921 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:00.666146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.713360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d7b5-3a73-476f-8f35-23d1368ab764 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.737264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bfe0a-ba26-40b4-882e-4e0b132eab26 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.753920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a157ec614b95412a820ead30ee3d8691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '152496f26d16469a9df94d5abe65ef6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.754064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:INSERT=8,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:00.755061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.755311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.755592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.756362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:00.756707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.759356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.795198 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3941/7856] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:00.867177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.868564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.869683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.869974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.872800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:list_access_rules(user_id=a30a76c2645249c0b65ba7c9e84c7ae7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.875337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:00.894197 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3916/7857] 10.222.0.26 () {58 vars in 1260 bytes} [Sat Sep 17 21:21:00 2022] GET /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules => generated 369 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:21:00.896215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:00.908224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638aab31-3984-4063-ac62-65bbc8adc558 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:00.913540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:00.914470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:00.914981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:00.915458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:00.916359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:create_application_credential(user_id=a30a76c2645249c0b65ba7c9e84c7ae7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:00.917861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:01.112436 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3942/7858] 10.222.0.26 () {60 vars in 1315 bytes} [Sat Sep 17 21:21:00 2022] POST /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/application_credentials => generated 921 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:01.115127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:01.137878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a236f46-7d14-4dc5-9771-802231e4cc34 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:01.148880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a30a76c2645249c0b65ba7c9e84c7ae7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c46c200677a54988835e2fc6316e74f3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:01.150125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:01.150641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:01.151189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules/6fdedb22e71645fda4bb9b52f4a33420` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:01.152941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorizing `identity:get_access_rule(user_id=a30a76c2645249c0b65ba7c9e84c7ae7, access_rule_id=6fdedb22e71645fda4bb9b52f4a33420)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:01.158585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:01.182011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:01.185534 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3917/7859] 10.222.0.26 () {58 vars in 1359 bytes} [Sat Sep 17 21:21:01 2022] GET /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7/access_rules/6fdedb22e71645fda4bb9b52f4a33420 => generated 229 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:01.278421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb4c0e-bfe5-4038-9aad-ead89e459028 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:01.284984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:01.294853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:01.295222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:01.295659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] PATH_INFO: `/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:01.297731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30a76c2645249c0b65ba7c9e84c7ae7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:01.299992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:01.388846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:01.399169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30a76c2645249c0b65ba7c9e84c7ae7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.400477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30a76c2645249c0b65ba7c9e84c7ae7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.419376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30a76c2645249c0b65ba7c9e84c7ae7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.420357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30a76c2645249c0b65ba7c9e84c7ae7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.448903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:01.451686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:01.478003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e28ac-4f61-49fd-923f-675ca99c0b36 tempest-AccessRulesV3Test-1665458968 tempest-AccessRulesV3Test-1665458968-project] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:01.484471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a157ec614b95412a820ead30ee3d8691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '152496f26d16469a9df94d5abe65ef6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:01.485259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:01.485693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:01.486056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] PATH_INFO: `/v3/projects/d2718f9019e9499991052d3c51bdb91e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:01.500463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d2718f9019e9499991052d3c51bdb91e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:01.502629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:01.515367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-916e4e21-2ada-4d59-8530-88366675b5bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30a76c2645249c0b65ba7c9e84c7ae7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.544067 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3943/7860] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:01 2022] DELETE /identity/v3/users/a30a76c2645249c0b65ba7c9e84c7ae7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:01.548516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:01.548794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:01.549073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:01.582778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:01.584084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:01.680317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2718f9019e9499991052d3c51bdb91e', 'request_id': 'req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.705415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2718f9019e9499991052d3c51bdb91e', 'request_id': 'req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.718561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2718f9019e9499991052d3c51bdb91e', 'request_id': 'req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.730784 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3944/7861] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:01.741590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:01.744098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:01.755736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] The token cache is being invalidate because project d2718f9019e9499991052d3c51bdb91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:01.756246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.760109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:01.765744 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3918/7862] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:01 2022] DELETE /identity/v3/projects/d2718f9019e9499991052d3c51bdb91e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:01.778453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:01.783764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:01.896085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e13d2-f988-4121-a0a1-dcb4c596cfc2 tempest-RoutersAdminTest-1768177306 tempest-RoutersAdminTest-1768177306-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:01.901044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15723e36-51d8-40bb-b004-9d77d8ffb3d8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:01.901344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:01.902423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:01.903020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:01.903640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:01.905649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:01.913555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:01.915242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:01.915522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:01.915792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:01.924325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:01.986126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:02.016945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:02.019461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:02.019893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:02.033725 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3919/7863] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:02.043239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:02.044861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:02.057625 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3945/7864] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:03.162227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.164053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:03.199644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c9e5c-6d6a-481b-b204-6680e3053761 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.215592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.218824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.219134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.219419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] PATH_INFO: `/v3/projects/c46c200677a54988835e2fc6316e74f3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.221205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46c200677a54988835e2fc6316e74f3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.223612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.232058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.233675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:03.248936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e0278-a45c-4f6a-8b95-f95d848a6952 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.254333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.255232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.255736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.256200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] PATH_INFO: `/v3/users/90e226926c024e0da6e1bff6c92105b4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.258065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e226926c024e0da6e1bff6c92105b4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.260189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.288571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46c200677a54988835e2fc6316e74f3', 'request_id': 'req-29b772e5-be7c-42cd-b498-8c91eed12236'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.299921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46c200677a54988835e2fc6316e74f3', 'request_id': 'req-29b772e5-be7c-42cd-b498-8c91eed12236'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.308360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46c200677a54988835e2fc6316e74f3', 'request_id': 'req-29b772e5-be7c-42cd-b498-8c91eed12236'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.320775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] The token cache is being invalidate because project c46c200677a54988835e2fc6316e74f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:03.320986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e226926c024e0da6e1bff6c92105b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.321344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.321694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e226926c024e0da6e1bff6c92105b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.322557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-29b772e5-be7c-42cd-b498-8c91eed12236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.324171 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3920/7865] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:03 2022] DELETE /identity/v3/projects/c46c200677a54988835e2fc6316e74f3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:03.329141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e98570-a50e-4c2b-a22f-cd9e5557d962 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.329516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e98570-a50e-4c2b-a22f-cd9e5557d962 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.329885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e98570-a50e-4c2b-a22f-cd9e5557d962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.332981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e226926c024e0da6e1bff6c92105b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.333926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e226926c024e0da6e1bff6c92105b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.350819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-77e98570-a50e-4c2b-a22f-cd9e5557d962 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:03.351362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-624700eb-23d4-4110-aa26-d867fbf43f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e226926c024e0da6e1bff6c92105b4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:03.352035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-77e98570-a50e-4c2b-a22f-cd9e5557d962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:03.367316 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3946/7866] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:03 2022] DELETE /identity/v3/users/90e226926c024e0da6e1bff6c92105b4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:03.372001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.372278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.372815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.395155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:03.396745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:03.436170 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3921/7867] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:03.444046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e0c38-de27-49e6-9fed-80a9963d01a3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.445182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e0c38-de27-49e6-9fed-80a9963d01a3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.445485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e0c38-de27-49e6-9fed-80a9963d01a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.462846 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3947/7868] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:03.467796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-009e0c38-de27-49e6-9fed-80a9963d01a3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:03.469074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-009e0c38-de27-49e6-9fed-80a9963d01a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:03.473593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.475057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:03.479006 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3922/7869] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:03.483768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e0c38-de27-49e6-9fed-80a9963d01a3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.494270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e0c38-de27-49e6-9fed-80a9963d01a3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.499133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.499873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.500146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.500413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.501226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:03.501730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.503920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.517944 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3923/7870] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:03.522123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.535999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabf9c7-377e-49e9-8511-ca96a159df33 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.543466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.544307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.544556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.545027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.545741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.548535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.558295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deae885-e246-438c-b1e0-bf95243e7455 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.562720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.563443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.563781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.563978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.575680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.580344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.588772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:03.591739 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3924/7871] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:03 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:03.597499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3948/7872] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:03.598324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.650389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:03.708804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79e49f-518d-4377-8332-93d32c0b0027 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.715770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.716681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.717027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.717359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.718445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.720360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.752425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3925/7873] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:03 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:03.755227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.775708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77613876-2334-406b-bc96-22eea16ea64c service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.784456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.784456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.784456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.784456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.786130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.788549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.806813 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3949/7874] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:03.811117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.823489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e77d3-8aeb-404c-881f-4dd9328f3cbd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.828376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e6518-8aa3-411c-9c78-ca2a132cc05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.829160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e6518-8aa3-411c-9c78-ca2a132cc05d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.829489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e6518-8aa3-411c-9c78-ca2a132cc05d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.829765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e6518-8aa3-411c-9c78-ca2a132cc05d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.830451 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e6518-8aa3-411c-9c78-ca2a132cc05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.832213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e6518-8aa3-411c-9c78-ca2a132cc05d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.843662 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3926/7875] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:03.848364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:03.860033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c82e04-07fd-4658-9260-9e82bf77ac0d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:03.865059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48fbfe-1081-44a3-8af7-38e6235054e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:03.866196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48fbfe-1081-44a3-8af7-38e6235054e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.866196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48fbfe-1081-44a3-8af7-38e6235054e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.866731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48fbfe-1081-44a3-8af7-38e6235054e4 admin admin] PATH_INFO: `/v3/projects/4d2e5e0a89924e89ad187654825b3d1f/users/32c5ed7f9b7444189c6d162e078ab5ed/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.890453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48fbfe-1081-44a3-8af7-38e6235054e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2e5e0a89924e89ad187654825b3d1f, user_id=32c5ed7f9b7444189c6d162e078ab5ed, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:03.895012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48fbfe-1081-44a3-8af7-38e6235054e4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:03.920839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.921123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.921446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.936180 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3950/7876] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:03 2022] PUT /identity/v3/projects/4d2e5e0a89924e89ad187654825b3d1f/users/32c5ed7f9b7444189c6d162e078ab5ed/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:03.947587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05d4e6-1314-483b-b78e-c9956e80fe91 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:03.948122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05d4e6-1314-483b-b78e-c9956e80fe91 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:03.948801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05d4e6-1314-483b-b78e-c9956e80fe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:03.956550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] MFA Rules not processed for user `fd15b9d354eb44b0bfa83bcdcf5c1384`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:03.957891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:03.998444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ad05d4e6-1314-483b-b78e-c9956e80fe91 None None] MFA Rules not processed for user `32c5ed7f9b7444189c6d162e078ab5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:03.999733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ad05d4e6-1314-483b-b78e-c9956e80fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:04.075149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3927/7877] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:04.076289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:04.077614 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3951/7878] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:04.077923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:04.080847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05d4e6-1314-483b-b78e-c9956e80fe91 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:04.092775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05d4e6-1314-483b-b78e-c9956e80fe91 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:04.098915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a1617-29d0-4133-9662-f46b0c01a073 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32c5ed7f9b7444189c6d162e078ab5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d2e5e0a89924e89ad187654825b3d1f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:04.099671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a1617-29d0-4133-9662-f46b0c01a073 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.099953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a1617-29d0-4133-9662-f46b0c01a073 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.100225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a1617-29d0-4133-9662-f46b0c01a073 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] PATH_INFO: `/v3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.102580 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3952/7879] 10.222.0.26 () {58 vars in 1104 bytes} [Sat Sep 17 21:21:04 2022] GET /identity/v3 => generated 255 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:21:04.107171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a1617-29d0-4133-9662-f46b0c01a073 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:04.123053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a1617-29d0-4133-9662-f46b0c01a073 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:04.128979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d074df3-0c8f-49fb-9afe-b0db08c8dd20 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32c5ed7f9b7444189c6d162e078ab5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d2e5e0a89924e89ad187654825b3d1f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:04.129776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d074df3-0c8f-49fb-9afe-b0db08c8dd20 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.131392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d074df3-0c8f-49fb-9afe-b0db08c8dd20 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.131583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d074df3-0c8f-49fb-9afe-b0db08c8dd20 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] PATH_INFO: `/v3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.137098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3953/7880] 10.222.0.26 () {58 vars in 1104 bytes} [Sat Sep 17 21:21:04 2022] GET /identity/v3 => generated 255 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:04.140206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d074df3-0c8f-49fb-9afe-b0db08c8dd20 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:04.152004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d074df3-0c8f-49fb-9afe-b0db08c8dd20 tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:04.156880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7619b-4bbd-43ff-8c08-1b7f0b74552e tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32c5ed7f9b7444189c6d162e078ab5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d2e5e0a89924e89ad187654825b3d1f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:04.157497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7619b-4bbd-43ff-8c08-1b7f0b74552e tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.157792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7619b-4bbd-43ff-8c08-1b7f0b74552e tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.158064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7619b-4bbd-43ff-8c08-1b7f0b74552e tempest-TestApiDiscovery-1698372859 tempest-TestApiDiscovery-1698372859-project] PATH_INFO: `/v3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.161527 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3954/7881] 10.222.0.26 () {58 vars in 1104 bytes} [Sat Sep 17 21:21:04 2022] GET /identity/v3 => generated 255 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:04.165385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98221-3576-4d81-be13-9662558d08db None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.165725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98221-3576-4d81-be13-9662558d08db None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.165999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98221-3576-4d81-be13-9662558d08db None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.167407 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3955/7882] 10.222.0.26 () {56 vars in 895 bytes} [Sat Sep 17 21:21:04 2022] GET /identity/ => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 21:21:04.172551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15852ea9-17ba-4568-90f0-e393203faa02 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.172551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15852ea9-17ba-4568-90f0-e393203faa02 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.172793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15852ea9-17ba-4568-90f0-e393203faa02 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.174031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3956/7883] 10.222.0.26 () {56 vars in 895 bytes} [Sat Sep 17 21:21:04 2022] GET /identity/ => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:21:04.179024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15852ea9-17ba-4568-90f0-e393203faa02 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:04.202392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf4cfc-bd73-4ce6-a008-86a442238073 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:04.215160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:04.215894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.216175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.217222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] PATH_INFO: `/v3/projects/e356478356274aacbc41e14a05b7819c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.222257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e356478356274aacbc41e14a05b7819c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:04.224204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:04.278119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15852ea9-17ba-4568-90f0-e393203faa02 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:04.294849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:04.298204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e356478356274aacbc41e14a05b7819c', 'request_id': 'req-3b2285b3-3b5b-4401-a735-b7912b77d1f4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.322409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e356478356274aacbc41e14a05b7819c', 'request_id': 'req-3b2285b3-3b5b-4401-a735-b7912b77d1f4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.336232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e356478356274aacbc41e14a05b7819c', 'request_id': 'req-3b2285b3-3b5b-4401-a735-b7912b77d1f4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.355368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] The token cache is being invalidate because project e356478356274aacbc41e14a05b7819c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:04.355972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.356940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.358634 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3928/7884] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:04 2022] DELETE /identity/v3/projects/e356478356274aacbc41e14a05b7819c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:04.363771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:04.366138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:04.374436 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:04.397980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:04.398998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.399410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.399863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] PATH_INFO: `/v3/users/32c5ed7f9b7444189c6d162e078ab5ed` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.401162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c5ed7f9b7444189c6d162e078ab5ed)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:04.404609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:04.466668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2285b3-3b5b-4401-a735-b7912b77d1f4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:04.472707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:04.473432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.473742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.474019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] PATH_INFO: `/v3/users/5e4eb93d88bc4dd9acb4ed0809331011` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.475493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4eb93d88bc4dd9acb4ed0809331011)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:04.477594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:04.489100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c5ed7f9b7444189c6d162e078ab5ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.490268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c5ed7f9b7444189c6d162e078ab5ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.504919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c5ed7f9b7444189c6d162e078ab5ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.506373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c5ed7f9b7444189c6d162e078ab5ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.524878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9b724515-ff90-4268-a973-0b0433a876b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c5ed7f9b7444189c6d162e078ab5ed'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.548014 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3957/7885] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:04 2022] DELETE /identity/v3/users/32c5ed7f9b7444189c6d162e078ab5ed => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:04.552839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.553332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.554546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.571231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4eb93d88bc4dd9acb4ed0809331011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.582177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:04.583880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:04.587869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4eb93d88bc4dd9acb4ed0809331011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.605506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4eb93d88bc4dd9acb4ed0809331011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.606406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4eb93d88bc4dd9acb4ed0809331011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.607740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4eb93d88bc4dd9acb4ed0809331011'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:04.627642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3929/7886] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:04 2022] DELETE /identity/v3/users/5e4eb93d88bc4dd9acb4ed0809331011 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:04.663980 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3958/7887] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:04.675552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:04.677249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:04.760613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435c83-4921-42e5-9fc7-121ebfd08935 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:04.766120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c78be-a741-4cd7-8cf2-ebcbe6be514a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:04.766819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c78be-a741-4cd7-8cf2-ebcbe6be514a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:04.767157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c78be-a741-4cd7-8cf2-ebcbe6be514a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:04.767492 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c78be-a741-4cd7-8cf2-ebcbe6be514a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:04.783248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:04.847230 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c78be-a741-4cd7-8cf2-ebcbe6be514a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:04.849961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c78be-a741-4cd7-8cf2-ebcbe6be514a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:04.864750 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3930/7888] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:05.038959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:05.040801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:05.081803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9233f7-2eff-4cde-bdff-1ff0b170bc8a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:05.092900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:05.093793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.094001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.094234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] PATH_INFO: `/v3/projects/222c5b6a8ae94481aa9ca228914d9518` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.095405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=222c5b6a8ae94481aa9ca228914d9518)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:05.098102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:05.204014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222c5b6a8ae94481aa9ca228914d9518', 'request_id': 'req-ab66124a-e758-4e33-a80a-7acccd3f5012'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.220866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222c5b6a8ae94481aa9ca228914d9518', 'request_id': 'req-ab66124a-e758-4e33-a80a-7acccd3f5012'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.247768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222c5b6a8ae94481aa9ca228914d9518', 'request_id': 'req-ab66124a-e758-4e33-a80a-7acccd3f5012'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.251516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:05.273382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] The token cache is being invalidate because project 222c5b6a8ae94481aa9ca228914d9518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:05.273777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.274713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.280290 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3959/7889] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:05 2022] DELETE /identity/v3/projects/222c5b6a8ae94481aa9ca228914d9518 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:05.282813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bea2ea-012e-40b9-b1b9-1b447131e24e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.283283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bea2ea-012e-40b9-b1b9-1b447131e24e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.283469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bea2ea-012e-40b9-b1b9-1b447131e24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.338906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:05.340799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:05.353328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d8bea2ea-012e-40b9-b1b9-1b447131e24e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:05.359918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d8bea2ea-012e-40b9-b1b9-1b447131e24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:05.461034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3931/7890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:05.463228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976fd69-dd2c-4952-afe2-8d7cd6688763 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.463569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976fd69-dd2c-4952-afe2-8d7cd6688763 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.463880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976fd69-dd2c-4952-afe2-8d7cd6688763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.476898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66124a-e758-4e33-a80a-7acccd3f5012 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:05.490974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:05.491641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.491986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.493323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] PATH_INFO: `/v3/projects/4d2e5e0a89924e89ad187654825b3d1f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.494473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2e5e0a89924e89ad187654825b3d1f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:05.498668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:05.505209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2976fd69-dd2c-4952-afe2-8d7cd6688763 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:05.505209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2976fd69-dd2c-4952-afe2-8d7cd6688763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:05.538373 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3932/7891] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:05.541639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976fd69-dd2c-4952-afe2-8d7cd6688763 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:05.553039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976fd69-dd2c-4952-afe2-8d7cd6688763 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:05.559078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:05.560065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.560551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.561672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.562644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:05.563122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:05.565373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:05.590857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2e5e0a89924e89ad187654825b3d1f', 'request_id': 'req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.603161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2e5e0a89924e89ad187654825b3d1f', 'request_id': 'req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.622146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2e5e0a89924e89ad187654825b3d1f', 'request_id': 'req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.624154 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3933/7892] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:05.629803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:05.638020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] The token cache is being invalidate because project 4d2e5e0a89924e89ad187654825b3d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:05.638461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.639318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f0e2c38e-1f58-4bff-abbb-8dc956ca883b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:05.642261 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3960/7893] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:05 2022] DELETE /identity/v3/projects/4d2e5e0a89924e89ad187654825b3d1f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:05.648387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08e08-0f11-4b4d-bfbc-5d548809fe56 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.648571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:05.648678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08e08-0f11-4b4d-bfbc-5d548809fe56 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.648678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08e08-0f11-4b4d-bfbc-5d548809fe56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.683356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49f0e9-6452-45c5-ad92-f6ad54ebd5dc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:05.705574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-45a08e08-0f11-4b4d-bfbc-5d548809fe56 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:05.706812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-45a08e08-0f11-4b4d-bfbc-5d548809fe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:05.814570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:05.815676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.816284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.817185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.819576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:05.823124 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3961/7894] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:05.823388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:05.832306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962de87-9c36-4ff0-95a1-a15b53630879 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:05.832647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962de87-9c36-4ff0-95a1-a15b53630879 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:05.832862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962de87-9c36-4ff0-95a1-a15b53630879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:05.855196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a962de87-9c36-4ff0-95a1-a15b53630879 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:05.856253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a962de87-9c36-4ff0-95a1-a15b53630879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:05.878981 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3962/7895] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:05.879956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962de87-9c36-4ff0-95a1-a15b53630879 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:05.897477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962de87-9c36-4ff0-95a1-a15b53630879 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:05.898980 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3934/7896] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:05 2022] POST /identity/v3/projects => generated 371 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:05.899909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:05.916179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:05.922825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:06.017000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d0b18-58d9-4b63-9332-440d6d0cadd3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.020772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.021539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.021996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.022659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.023118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.023408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.024067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.025307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.025517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.026054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.026171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:06.026476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.030613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.064870 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3963/7897] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:06.066398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.073641 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3935/7898] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:05 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:06.079113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.080367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19dd1e-d2b4-4fdd-980f-d4744808aadb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.086535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.087471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.087847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.088213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.088980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.091864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.103821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43751b-7809-419f-9070-4f1b9606a344 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.122694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.123581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.124068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.124630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.125538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.129224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.138009 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3964/7899] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:06 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:06.139358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.151392 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3936/7900] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:06.152474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.163535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a96da-556f-4ab3-adfe-39426e2df1f7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.259021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda472-95a2-4059-96c6-9306cc944957 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.264538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.265179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.265576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.265860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.266517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.269132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.298508 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3965/7901] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:06 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:06.303732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.310881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.311536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.311781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.312059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.312718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.314552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.324279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4d724-b60e-4f7f-a1b4-6af28912eab9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.325617 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3937/7902] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:06.329748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.329890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.330699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.331225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.331504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.332165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.333854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.341237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf108-6fb2-46ca-a520-507049213c7f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.347302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73df3d5-c283-4c02-a187-e886092c2949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.348254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df3d5-c283-4c02-a187-e886092c2949 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.348699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df3d5-c283-4c02-a187-e886092c2949 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.348849 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3966/7903] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:06.349105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df3d5-c283-4c02-a187-e886092c2949 admin admin] PATH_INFO: `/v3/projects/903bafa67cf6463e8af7e7fce1c2a9fa/users/f9b8a414f27947d5816d82437007456d/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.352953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.374143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d51d4-c85c-42ba-ac8b-c91af64fc3ac admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.375525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73df3d5-c283-4c02-a187-e886092c2949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903bafa67cf6463e8af7e7fce1c2a9fa, user_id=f9b8a414f27947d5816d82437007456d, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.381669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73df3d5-c283-4c02-a187-e886092c2949 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.390604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.391426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.391722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.392066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] PATH_INFO: `/v3/projects/f66e298d1c544e5a840e160ae1a3e027/users/e8f5cd43545945c4a78eb5f1ce78a513/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.412915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66e298d1c544e5a840e160ae1a3e027, user_id=e8f5cd43545945c4a78eb5f1ce78a513, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.416528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.423330 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3938/7904] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:06 2022] PUT /identity/v3/projects/903bafa67cf6463e8af7e7fce1c2a9fa/users/f9b8a414f27947d5816d82437007456d/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:06.434305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd0c6-b794-496e-afc5-bfdc77f1a5d9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.434980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd0c6-b794-496e-afc5-bfdc77f1a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.435602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd0c6-b794-496e-afc5-bfdc77f1a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.455861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3967/7905] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:06 2022] PUT /identity/v3/projects/f66e298d1c544e5a840e160ae1a3e027/users/e8f5cd43545945c4a78eb5f1ce78a513/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:06.460287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.487144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2adbd0c6-b794-496e-afc5-bfdc77f1a5d9 None None] MFA Rules not processed for user `f9b8a414f27947d5816d82437007456d`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:06.488831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2adbd0c6-b794-496e-afc5-bfdc77f1a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:06.538997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:06.584301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec60865-693b-4cf4-a1bc-fc856e282a51 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.591139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.592264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.592849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.593449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.594547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.596423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.604059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:06.616561 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3968/7906] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:06.618337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.653872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a5ec-14ba-410c-8b58-1d8019dbe24d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.671495 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3939/7907] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:06.671742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.671742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.671742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.671742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] PATH_INFO: `/v3/projects/f66e298d1c544e5a840e160ae1a3e027/users/e8f5cd43545945c4a78eb5f1ce78a513/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.676448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66e298d1c544e5a840e160ae1a3e027, user_id=e8f5cd43545945c4a78eb5f1ce78a513, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.680149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.710240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd112-3c0e-4238-be03-76cc429f4703 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.710528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd112-3c0e-4238-be03-76cc429f4703 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.710805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd112-3c0e-4238-be03-76cc429f4703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.711105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3969/7908] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:06 2022] PUT /identity/v3/projects/f66e298d1c544e5a840e160ae1a3e027/users/e8f5cd43545945c4a78eb5f1ce78a513/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:06.725671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.727915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:06.751600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-726bd112-3c0e-4238-be03-76cc429f4703 None None] MFA Rules not processed for user `e8f5cd43545945c4a78eb5f1ce78a513`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:06.753326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-726bd112-3c0e-4238-be03-76cc429f4703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:06.818351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873203-4af5-4f16-86d9-98c9a8441a7c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.822845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.823695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.823989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.824257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.836679 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3940/7909] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:06.842127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd112-3c0e-4238-be03-76cc429f4703 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.900737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.903026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.915749 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3970/7910] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:06.920501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd112-3c0e-4238-be03-76cc429f4703 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:06.925216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:06.925880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:06.926161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:06.926847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:06.927079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:06.929084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:06.971302 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3941/7911] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:06 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:06.973019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:06.986955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e83475-698d-47be-8812-10acc834afc6 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:07.070593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:07.071571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.071571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.071763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.072412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:07.074602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:07.110831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:07.121155 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3971/7912] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:06 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:07.123452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccf9a1-9c12-4992-88e2-933933a9a4ce admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:07.132777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:07.133326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.133656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.133931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.134594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:07.136316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:07.154141 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3942/7913] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:07.157836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:07.170233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a91cc-dc76-45cf-8b45-885276ea24ac admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:07.178867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df795b-9b28-4c56-9c05-bfa71fa59dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:07.179396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df795b-9b28-4c56-9c05-bfa71fa59dca admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.179674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df795b-9b28-4c56-9c05-bfa71fa59dca admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.179941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df795b-9b28-4c56-9c05-bfa71fa59dca admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.181017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df795b-9b28-4c56-9c05-bfa71fa59dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:07.182565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df795b-9b28-4c56-9c05-bfa71fa59dca admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:07.185693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:07.197023 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3972/7914] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:07.207345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:07.227225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78de93-ed54-42e6-994b-739a6278aaca admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:07.245401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:07.246559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.247708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.248384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] PATH_INFO: `/v3/projects/5e1e49abfd664e938ce7c29c5ec2c066/users/b4f6ef27865a450287d060a339077421/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.278249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1e49abfd664e938ce7c29c5ec2c066, user_id=b4f6ef27865a450287d060a339077421, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:07.281736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:07.303235 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3943/7915] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:07 2022] PUT /identity/v3/projects/5e1e49abfd664e938ce7c29c5ec2c066/users/b4f6ef27865a450287d060a339077421/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:07.316539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194144a-2cf6-425f-936f-c5957a3403b3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.316539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194144a-2cf6-425f-936f-c5957a3403b3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.316539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194144a-2cf6-425f-936f-c5957a3403b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.352526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e194144a-2cf6-425f-936f-c5957a3403b3 None None] MFA Rules not processed for user `b4f6ef27865a450287d060a339077421`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:07.353858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e194144a-2cf6-425f-936f-c5957a3403b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:07.466938 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3973/7916] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:07.479802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:07.482661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:07.523250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:07.577498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0448cf-584d-45a9-87d7-f91470417d90 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:07.582339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94d291-036f-4e2e-a2f7-4dac6ad4b0ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:07.583318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d291-036f-4e2e-a2f7-4dac6ad4b0ac service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.585595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d291-036f-4e2e-a2f7-4dac6ad4b0ac service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.585595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d291-036f-4e2e-a2f7-4dac6ad4b0ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.675957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94d291-036f-4e2e-a2f7-4dac6ad4b0ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:07.678783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94d291-036f-4e2e-a2f7-4dac6ad4b0ac service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:07.700879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194144a-2cf6-425f-936f-c5957a3403b3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:07.707793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3944/7917] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:07.782990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194144a-2cf6-425f-936f-c5957a3403b3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:07.788172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:07.788957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.789429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.789739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] PATH_INFO: `/v3/users/f9b8a414f27947d5816d82437007456d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.790916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b8a414f27947d5816d82437007456d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:07.792998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:07.854092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b8a414f27947d5816d82437007456d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:07.854747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b8a414f27947d5816d82437007456d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:07.865446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b8a414f27947d5816d82437007456d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:07.866326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b8a414f27947d5816d82437007456d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:07.880700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b8a414f27947d5816d82437007456d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:07.900753 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3974/7918] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:07 2022] DELETE /identity/v3/users/f9b8a414f27947d5816d82437007456d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:07.905657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:07.906316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:07.906819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:07.940751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:07.942304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:08.003460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:08.004976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:08.015597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:08.022500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3945/7919] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:08.033686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:08.035782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:08.103449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf5578-d62c-45d9-a0fd-1ee517480470 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:08.110565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:08.111238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:08.111518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:08.112570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] PATH_INFO: `/v3/users/e8f5cd43545945c4a78eb5f1ce78a513` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:08.116462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f5cd43545945c4a78eb5f1ce78a513)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:08.117930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9d45-4e24-4119-bdf4-79bef1733060 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:08.118880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:08.122894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe789a-40fe-4a22-891a-38d2dfe1fbcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:08.123916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe789a-40fe-4a22-891a-38d2dfe1fbcd service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:08.124395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe789a-40fe-4a22-891a-38d2dfe1fbcd service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:08.124873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe789a-40fe-4a22-891a-38d2dfe1fbcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:08.135435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe789a-40fe-4a22-891a-38d2dfe1fbcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:08.137197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe789a-40fe-4a22-891a-38d2dfe1fbcd service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:08.148202 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3946/7920] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:21:08.188289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f5cd43545945c4a78eb5f1ce78a513'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:08.189124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f5cd43545945c4a78eb5f1ce78a513'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:08.199444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f5cd43545945c4a78eb5f1ce78a513'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:08.200321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f5cd43545945c4a78eb5f1ce78a513'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:08.219838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f5cd43545945c4a78eb5f1ce78a513'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:08.241279 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3975/7921] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:08 2022] DELETE /identity/v3/users/e8f5cd43545945c4a78eb5f1ce78a513 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:08.244449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:08.244731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:08.244996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:08.275328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:08.276561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:08.291194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:08.345540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3947/7922] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:08.364025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:08.365986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:08.454345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8295ef-95c7-4eb5-a891-da34391f6a3c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:08.459323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31aa10-575d-4266-8640-bf242ced1f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:08.460040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31aa10-575d-4266-8640-bf242ced1f26 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:08.460645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31aa10-575d-4266-8640-bf242ced1f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:08.460995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31aa10-575d-4266-8640-bf242ced1f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:08.472402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31aa10-575d-4266-8640-bf242ced1f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:08.474020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31aa10-575d-4266-8640-bf242ced1f26 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:08.488429 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3976/7923] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:09.222323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:09.225715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:09.239471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a00598-d16c-467a-88af-0b33f7601eeb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:09.246763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:09.246763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:09.247070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:09.247294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] PATH_INFO: `/v3/projects/903bafa67cf6463e8af7e7fce1c2a9fa` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:09.249727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=903bafa67cf6463e8af7e7fce1c2a9fa)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:09.252875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:09.342022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903bafa67cf6463e8af7e7fce1c2a9fa', 'request_id': 'req-731d7825-6bae-48d0-97b3-5cb7bb805a0c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.358611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903bafa67cf6463e8af7e7fce1c2a9fa', 'request_id': 'req-731d7825-6bae-48d0-97b3-5cb7bb805a0c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.376063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903bafa67cf6463e8af7e7fce1c2a9fa', 'request_id': 'req-731d7825-6bae-48d0-97b3-5cb7bb805a0c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.401248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] The token cache is being invalidate because project 903bafa67cf6463e8af7e7fce1c2a9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:09.402277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.405120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.413802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfbe2e-f179-4c08-8906-d043497ee775 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:09.414390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3948/7924] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:09 2022] DELETE /identity/v3/projects/903bafa67cf6463e8af7e7fce1c2a9fa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:09.414882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfbe2e-f179-4c08-8906-d043497ee775 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:09.415367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfbe2e-f179-4c08-8906-d043497ee775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:09.438277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-28dfbe2e-f179-4c08-8906-d043497ee775 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:09.440045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-28dfbe2e-f179-4c08-8906-d043497ee775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:09.491419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:09.534364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:09.536688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:09.576787 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3977/7925] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:09.578413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df34fd-cbdd-49f4-a477-c9cabe4ad0c3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:09.578703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df34fd-cbdd-49f4-a477-c9cabe4ad0c3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:09.578985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df34fd-cbdd-49f4-a477-c9cabe4ad0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:09.608228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-06df34fd-cbdd-49f4-a477-c9cabe4ad0c3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:09.610289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-06df34fd-cbdd-49f4-a477-c9cabe4ad0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:09.626279 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3978/7926] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:09.631525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df34fd-cbdd-49f4-a477-c9cabe4ad0c3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:09.644187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df34fd-cbdd-49f4-a477-c9cabe4ad0c3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:09.650425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:09.651096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:09.651476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:09.651655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:09.652284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:09.652596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:09.655802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:09.668942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d7825-6bae-48d0-97b3-5cb7bb805a0c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:09.673966 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3979/7927] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:09.674922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:09.691574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cba56-772f-4bed-9ed9-d30eb7840b01 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:09.696529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:09.698931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:09.699427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:09.700137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] PATH_INFO: `/v3/projects/f66e298d1c544e5a840e160ae1a3e027` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:09.700623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:09.701813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:09.702115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:09.702374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:09.703020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:09.703447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66e298d1c544e5a840e160ae1a3e027)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:09.704959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:09.706461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:09.750551 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3980/7928] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:09 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:09.754616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:09.801270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:09.805900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66e298d1c544e5a840e160ae1a3e027', 'request_id': 'req-5cd68651-d09e-49a2-a5df-9b9c63e84419'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.832311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66e298d1c544e5a840e160ae1a3e027', 'request_id': 'req-5cd68651-d09e-49a2-a5df-9b9c63e84419'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.845064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66e298d1c544e5a840e160ae1a3e027', 'request_id': 'req-5cd68651-d09e-49a2-a5df-9b9c63e84419'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.859449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bace5-48ff-4850-8c14-570f80d3cb79 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:09.861435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] The token cache is being invalidate because project f66e298d1c544e5a840e160ae1a3e027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:09.861824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.862588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:09.867955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3949/7929] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:09 2022] DELETE /identity/v3/projects/f66e298d1c544e5a840e160ae1a3e027 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:09.869170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:09.870939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:09.934820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:09.935370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:09.935939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:09.935939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:09.936547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:09.938837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:09.970781 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3981/7930] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:09 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:09.976710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:09.981447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:09.986137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68651-d09e-49a2-a5df-9b9c63e84419 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:10.005660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:10.006215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896584e3-d4fc-437e-a8d8-5f5f2be43b95 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:10.006400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:10.006630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:10.006890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] PATH_INFO: `/v3/users/b4f6ef27865a450287d060a339077421` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:10.008164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f6ef27865a450287d060a339077421)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:10.010235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:10.022342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:10.024533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:10.024812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:10.025083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:10.025780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:10.027398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:10.041219 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3982/7931] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:10.048161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:10.063440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f3342-bd50-49a7-ae04-6828294709da admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:10.070224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:10.070749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:10.071292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:10.071654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:10.073065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:10.078569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:10.103492 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3983/7932] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:10.107946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:10.116672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:10.124526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c69f4-06aa-44e8-856e-30fb71b3e5fd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:10.128201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f6ef27865a450287d060a339077421'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:10.133609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:10.134439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:10.134815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:10.135280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] PATH_INFO: `/v3/projects/6e2c4ca90cf748cd9f74234e6983a2bf/users/0a047d9921dc448586b7eeb8bdf2818b/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:10.144410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f6ef27865a450287d060a339077421'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:10.155716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f6ef27865a450287d060a339077421'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:10.156637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f6ef27865a450287d060a339077421'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:10.157535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-afd944eb-10c5-4e81-87ea-7093ec918a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f6ef27865a450287d060a339077421'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:10.160446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2c4ca90cf748cd9f74234e6983a2bf, user_id=0a047d9921dc448586b7eeb8bdf2818b, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:10.165845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:10.180588 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3950/7933] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:09 2022] DELETE /identity/v3/users/b4f6ef27865a450287d060a339077421 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:10.191504 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3984/7934] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:10 2022] PUT /identity/v3/projects/6e2c4ca90cf748cd9f74234e6983a2bf/users/0a047d9921dc448586b7eeb8bdf2818b/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:10.205016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:10.206086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:10.206546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:10.241097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] MFA Rules not processed for user `0a047d9921dc448586b7eeb8bdf2818b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:10.243487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:10.302463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:10.342060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3951/7935] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:10.359000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:10.361144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:10.438606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17cdde-5487-45b5-84f1-3149a28229e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:10.489343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:10.489985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:10.490358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:10.490661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:10.499973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:10.501401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:10.511134 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3985/7936] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:10.936819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:10.939859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:11.012513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1208a-8f9a-4e78-83a1-8b3501e8cb99 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.018123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.018780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.019442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.019716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.019984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] PATH_INFO: `/v3/users/0a047d9921dc448586b7eeb8bdf2818b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.020634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:11.021365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a047d9921dc448586b7eeb8bdf2818b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.024378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.036640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf9f8e-1e56-4e46-9503-fce574baf118 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.042810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.043965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.044232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.044495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] PATH_INFO: `/v3/projects/5e1e49abfd664e938ce7c29c5ec2c066` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.045708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1e49abfd664e938ce7c29c5ec2c066)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.048552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.100763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a047d9921dc448586b7eeb8bdf2818b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.112435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1e49abfd664e938ce7c29c5ec2c066', 'request_id': 'req-5143dd13-da36-43a2-8dbb-c08bce323038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.119667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1e49abfd664e938ce7c29c5ec2c066', 'request_id': 'req-5143dd13-da36-43a2-8dbb-c08bce323038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.123348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a047d9921dc448586b7eeb8bdf2818b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.130003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1e49abfd664e938ce7c29c5ec2c066', 'request_id': 'req-5143dd13-da36-43a2-8dbb-c08bce323038'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.133454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a047d9921dc448586b7eeb8bdf2818b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.134170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a047d9921dc448586b7eeb8bdf2818b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.134772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-6f0cbe83-c1e0-46ce-8de7-08b5c00761cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a047d9921dc448586b7eeb8bdf2818b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.142331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] The token cache is being invalidate because project 5e1e49abfd664e938ce7c29c5ec2c066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:11.142985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.143945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5143dd13-da36-43a2-8dbb-c08bce323038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:11.149247 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3986/7937] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:11 2022] DELETE /identity/v3/projects/5e1e49abfd664e938ce7c29c5ec2c066 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:11.149912 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3952/7938] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:10 2022] DELETE /identity/v3/users/0a047d9921dc448586b7eeb8bdf2818b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:11.153745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.154032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.154231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2772da-5759-497b-b887-b7e11c8dfd5d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.154339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.154750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2772da-5759-497b-b887-b7e11c8dfd5d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.155097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2772da-5759-497b-b887-b7e11c8dfd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.175625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:11.177269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:11.178540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0d2772da-5759-497b-b887-b7e11c8dfd5d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:11.179865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d2772da-5759-497b-b887-b7e11c8dfd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:11.195620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:11.284554 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3953/7939] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:11.286350 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3987/7940] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:11.296322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834115e7-06cc-44e3-b18b-0d0031f96b4c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.296789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834115e7-06cc-44e3-b18b-0d0031f96b4c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.297201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834115e7-06cc-44e3-b18b-0d0031f96b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.307543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:11.314962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.321235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:11.328334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-834115e7-06cc-44e3-b18b-0d0031f96b4c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:11.329921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-834115e7-06cc-44e3-b18b-0d0031f96b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:11.343390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3954/7941] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:11.344976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834115e7-06cc-44e3-b18b-0d0031f96b4c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.355394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834115e7-06cc-44e3-b18b-0d0031f96b4c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.362119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.363017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.363592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.364015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.364827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:11.365328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.367508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.386688 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3955/7942] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:11.388030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.409401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b78a1-c2dd-4f50-904a-82550e2574fa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.428106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.428743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.429024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.429296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.430032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.431926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.464557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122503e8-6e9e-4974-8c80-493cf5ce8854 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.469497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.470183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.470459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.470735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.485932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.488206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.494263 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3956/7943] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:11 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:11.499121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.508094 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3988/7944] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:11.592562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235aa46-0c04-4a6a-bb04-cb0b14427703 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.598775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.599612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.600593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.600941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.601704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.603959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.644749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3957/7945] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:11 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:11.648945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.663492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a3c28-a88a-4e10-9f82-00f4c744d1fe service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.684482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.685850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.687000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.687529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.688659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.691380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.708998 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3989/7946] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:11.713194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.728916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5df2b-dee8-42ff-8483-61cee1553c9d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.742186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbcaba-01d2-4e99-ad59-7a3b8e1fe5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.744542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcaba-01d2-4e99-ad59-7a3b8e1fe5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.746556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcaba-01d2-4e99-ad59-7a3b8e1fe5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.747328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcaba-01d2-4e99-ad59-7a3b8e1fe5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.750228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbcaba-01d2-4e99-ad59-7a3b8e1fe5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.758232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbcaba-01d2-4e99-ad59-7a3b8e1fe5f6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.798216 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3958/7947] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:11.802678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:11.822334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf9cb-e9a3-4e57-95e9-902a208e9d59 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:11.832027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:11.832751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.832987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.833257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] PATH_INFO: `/v3/projects/e19c5ea21b2042ef8daad540bbe2599f/users/debb310701914ef5a73647ad5abaceaf/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.859978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19c5ea21b2042ef8daad540bbe2599f, user_id=debb310701914ef5a73647ad5abaceaf, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:11.862849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:11.888304 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3990/7948] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:11 2022] PUT /identity/v3/projects/e19c5ea21b2042ef8daad540bbe2599f/users/debb310701914ef5a73647ad5abaceaf/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:11.899151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb3a02-5416-4528-a90f-14faf12d1976 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:11.899465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb3a02-5416-4528-a90f-14faf12d1976 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:11.899756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb3a02-5416-4528-a90f-14faf12d1976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:11.941684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-83fb3a02-5416-4528-a90f-14faf12d1976 None None] MFA Rules not processed for user `debb310701914ef5a73647ad5abaceaf`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:11.945625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-83fb3a02-5416-4528-a90f-14faf12d1976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.065493 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3959/7949] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:12.073941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:12.076939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.184147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a44e4-613e-4274-8854-04a152bfbcf1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:12.188419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:12.189652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:12.189937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:12.190204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:12.204965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:12.207166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:12.227018 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3991/7950] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:21:12.357597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb3a02-5416-4528-a90f-14faf12d1976 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:12.404316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:12.406166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.420396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:12.465798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:12.484326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb3a02-5416-4528-a90f-14faf12d1976 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:12.494765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:12.495700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:12.495904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:12.496806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] PATH_INFO: `/v3/users/debb310701914ef5a73647ad5abaceaf` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:12.498887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] RBAC: Authorizing `identity:delete_user(user_id=debb310701914ef5a73647ad5abaceaf)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:12.501227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:12.510903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005fa6f-83aa-4bd2-9a4b-a95596611fdb service glance] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:12.516672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:12.517539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:12.517734 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:12.518019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] PATH_INFO: `/v3/projects/6e2c4ca90cf748cd9f74234e6983a2bf` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:12.520589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2c4ca90cf748cd9f74234e6983a2bf)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:12.521402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:12.559179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debb310701914ef5a73647ad5abaceaf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.575195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debb310701914ef5a73647ad5abaceaf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.576990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2c4ca90cf748cd9f74234e6983a2bf', 'request_id': 'req-e7453882-f393-4a30-a372-3f9eb57b02a4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.586296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2c4ca90cf748cd9f74234e6983a2bf', 'request_id': 'req-e7453882-f393-4a30-a372-3f9eb57b02a4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.594593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debb310701914ef5a73647ad5abaceaf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.595329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2c4ca90cf748cd9f74234e6983a2bf', 'request_id': 'req-e7453882-f393-4a30-a372-3f9eb57b02a4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.596994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debb310701914ef5a73647ad5abaceaf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.598327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-936bd742-7fb4-45cc-820c-70dfb6ea495b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debb310701914ef5a73647ad5abaceaf'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.608871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] The token cache is being invalidate because project 6e2c4ca90cf748cd9f74234e6983a2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:12.609157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.610507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:12.618181 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3992/7951] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:12 2022] DELETE /identity/v3/projects/6e2c4ca90cf748cd9f74234e6983a2bf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:12.619141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:12.621371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.626107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3960/7952] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:12 2022] DELETE /identity/v3/users/debb310701914ef5a73647ad5abaceaf => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:12.627965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ee126-4758-48c7-b1a8-53619f887521 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:12.629563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ee126-4758-48c7-b1a8-53619f887521 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:12.630258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ee126-4758-48c7-b1a8-53619f887521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:12.697695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9c1ee126-4758-48c7-b1a8-53619f887521 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:12.710188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9c1ee126-4758-48c7-b1a8-53619f887521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.719698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453882-f393-4a30-a372-3f9eb57b02a4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:12.811760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309d39e-52e1-4f02-9b50-4e205f211aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:12.812599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309d39e-52e1-4f02-9b50-4e205f211aab service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:12.812720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309d39e-52e1-4f02-9b50-4e205f211aab service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:12.812915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309d39e-52e1-4f02-9b50-4e205f211aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:12.819482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3309d39e-52e1-4f02-9b50-4e205f211aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.848066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309d39e-52e1-4f02-9b50-4e205f211aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:12.851492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309d39e-52e1-4f02-9b50-4e205f211aab service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:12.871531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3993/7953] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:12.874607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:12.874944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:12.875189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:12.913144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:12.915279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.921536 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3961/7954] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:12.929838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8dea6-6405-44be-99c8-078c80102d76 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:12.930206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8dea6-6405-44be-99c8-078c80102d76 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:12.930543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8dea6-6405-44be-99c8-078c80102d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:12.943041 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3994/7955] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:12.963240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:12.974574 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:12.985556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e1b8dea6-6405-44be-99c8-078c80102d76 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:12.986781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e1b8dea6-6405-44be-99c8-078c80102d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:13.003792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e704646-978a-4a40-a5b1-881a25f31ecf None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.005010 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3962/7956] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:13.006638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8dea6-6405-44be-99c8-078c80102d76 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.011155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.011794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.012248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.012530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.033335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.034860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.038382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8dea6-6405-44be-99c8-078c80102d76 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.043530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:13.045170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.047268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.047734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.048103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.049656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:13.049656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.051927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.058117 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3995/7957] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:13.075814 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3963/7958] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:13.076098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.094121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd424d-4f30-4370-9ee5-e916c2c22217 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.110900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.111423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.111717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.111981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.112635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.115159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.189104 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3996/7959] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:13 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:13.190758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.214600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6fc45-d742-4882-8311-c904f5a639e4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.260514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:13.309017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.309964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.313720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.314460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.315508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.318325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.363665 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3964/7960] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:13 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:13.368830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.386029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b0bbf-b5a7-4e53-9b7d-98e966125d7e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.397744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.399332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.399784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.400065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.400727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.406392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.439959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.452022 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3997/7961] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:13.459747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4ae64-5fbc-45a0-b025-9ea4470fa430 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.467695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77597bf-6a32-4420-a117-36b5ccb00505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.470800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77597bf-6a32-4420-a117-36b5ccb00505 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.471696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77597bf-6a32-4420-a117-36b5ccb00505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.472377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77597bf-6a32-4420-a117-36b5ccb00505 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.474199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77597bf-6a32-4420-a117-36b5ccb00505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.477307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77597bf-6a32-4420-a117-36b5ccb00505 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.491781 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3965/7962] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:13.496558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.508326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b7c15-3c96-4979-abc5-7f6a32b9c70e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.513610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.514278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.514552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.514819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] PATH_INFO: `/v3/projects/89ac3ed817a84d97aee955fc79bdd75e/users/6f348e9d8be94f5badac4d1d8a58e2e3/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.546773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ac3ed817a84d97aee955fc79bdd75e, user_id=6f348e9d8be94f5badac4d1d8a58e2e3, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.546773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.581336 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3998/7963] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:13 2022] PUT /identity/v3/projects/89ac3ed817a84d97aee955fc79bdd75e/users/6f348e9d8be94f5badac4d1d8a58e2e3/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:13.582890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.583172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.583442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.630136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] MFA Rules not processed for user `6f348e9d8be94f5badac4d1d8a58e2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:13.631675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:13.745127 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3966/7964] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:13.757278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.777753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:13.779905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:13.870536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6b076-9e5f-4f7d-a55e-c3dd4cde2167 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.885572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.890576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.890576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.890576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] PATH_INFO: `/v3/projects/e19c5ea21b2042ef8daad540bbe2599f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.890906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6957f3d-6374-4166-8663-0a44bec817a3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:13.891035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19c5ea21b2042ef8daad540bbe2599f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.891035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.900810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbff07-0ed8-4898-beda-f524fa448a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:13.901607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbff07-0ed8-4898-beda-f524fa448a34 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:13.901837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbff07-0ed8-4898-beda-f524fa448a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:13.902060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbff07-0ed8-4898-beda-f524fa448a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:13.912093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbff07-0ed8-4898-beda-f524fa448a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:13.913940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbff07-0ed8-4898-beda-f524fa448a34 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:13.940134 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 3999/7965] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:13.960555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:13.973127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19c5ea21b2042ef8daad540bbe2599f', 'request_id': 'req-63098fc2-761b-466e-9852-9d4f566291c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:13.985406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19c5ea21b2042ef8daad540bbe2599f', 'request_id': 'req-63098fc2-761b-466e-9852-9d4f566291c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:13.996208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19c5ea21b2042ef8daad540bbe2599f', 'request_id': 'req-63098fc2-761b-466e-9852-9d4f566291c2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:14.014065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] The token cache is being invalidate because project e19c5ea21b2042ef8daad540bbe2599f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:14.014484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:14.015258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-63098fc2-761b-466e-9852-9d4f566291c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:14.017318 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3967/7966] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:13 2022] DELETE /identity/v3/projects/e19c5ea21b2042ef8daad540bbe2599f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:14.022948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734d989-7164-46ae-9c2f-6c57f120f0e7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.024167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734d989-7164-46ae-9c2f-6c57f120f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.024711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734d989-7164-46ae-9c2f-6c57f120f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.061606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6734d989-7164-46ae-9c2f-6c57f120f0e7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:14.063118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6734d989-7164-46ae-9c2f-6c57f120f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:14.221038 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4000/7967] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:14.224249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dae779-27e2-48aa-a028-01dd63441c8c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.225088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dae779-27e2-48aa-a028-01dd63441c8c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.225578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dae779-27e2-48aa-a028-01dd63441c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.273545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c3dae779-27e2-48aa-a028-01dd63441c8c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:14.275115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c3dae779-27e2-48aa-a028-01dd63441c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:14.296437 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3968/7968] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:14.297904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734d989-7164-46ae-9c2f-6c57f120f0e7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:14.310556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734d989-7164-46ae-9c2f-6c57f120f0e7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:14.320513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:14.321408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.321943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.322439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.324030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:14.324631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:14.326727 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:14.365916 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4001/7969] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:14.366584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dae779-27e2-48aa-a028-01dd63441c8c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:14.379598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dae779-27e2-48aa-a028-01dd63441c8c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:14.384739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:14.385380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.385695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.385984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.386817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:14.388901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:14.478323 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3969/7970] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:14 2022] POST /identity/v3/projects => generated 407 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:14.482833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:14.498417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd756a-1eea-4214-afd6-846b5d0445b5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:14.622480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:14.673712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:14.674503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.674875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.675240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.676948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:14.680439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:14.755287 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4002/7971] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:14 2022] POST /identity/v3/users => generated 304 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:14.757095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:14.773103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfb1dc-027b-487e-a45e-3d9bbc98696e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:14.782141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:14.783047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.785592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.785991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.787802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:14.792569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:14.820848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3970/7972] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:14.824285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:14.837480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6ca32-bb0f-48d0-8a2f-0dee2259d18c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:14.855874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc919f3-75a9-462b-9775-eef6955330d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:14.857016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc919f3-75a9-462b-9775-eef6955330d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.858335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc919f3-75a9-462b-9775-eef6955330d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.858876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc919f3-75a9-462b-9775-eef6955330d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.862205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc919f3-75a9-462b-9775-eef6955330d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:14.864265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc919f3-75a9-462b-9775-eef6955330d6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:14.900981 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4003/7973] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:14.902665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:14.919788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf7935-b88a-43ff-b359-277072fe6186 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:14.926760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:14.927781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.928447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.928914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] PATH_INFO: `/v3/projects/886e5284ea6841949c59477f969f2a90/users/cea4f128fceb411c81017f8abe1b62a0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:14.953655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=886e5284ea6841949c59477f969f2a90, user_id=cea4f128fceb411c81017f8abe1b62a0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:14.957940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:14.979316 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3971/7974] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:14 2022] PUT /identity/v3/projects/886e5284ea6841949c59477f969f2a90/users/cea4f128fceb411c81017f8abe1b62a0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:14.992045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224e537-a2f9-438d-9935-9ea8f53a4799 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:14.993115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224e537-a2f9-438d-9935-9ea8f53a4799 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:14.993450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224e537-a2f9-438d-9935-9ea8f53a4799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:15.074815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e224e537-a2f9-438d-9935-9ea8f53a4799 None None] MFA Rules not processed for user `cea4f128fceb411c81017f8abe1b62a0`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:15.076325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e224e537-a2f9-438d-9935-9ea8f53a4799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:15.185950 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4004/7975] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:14 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:15.186532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:15.300584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8a160-3d8b-494a-8786-8a6f71876877 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:15.306877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:15.307878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:15.308200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:15.308568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:15.309345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:15.312914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:15.358566 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3972/7976] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:15 2022] POST /identity/v3/projects => generated 403 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:15.360433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224e537-a2f9-438d-9935-9ea8f53a4799 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:15.375976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224e537-a2f9-438d-9935-9ea8f53a4799 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:15.510029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:15.512447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:15.513398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:15.513935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:15.514780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:15.519793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:15.565687 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4005/7977] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:15 2022] POST /identity/v3/users => generated 308 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:15.567355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:15.595125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a60be-0454-4a4e-aeeb-6cbfdbd87d3e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:15.607685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:15.608703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:15.609197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:15.609843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:15.610751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:15.612988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:15.641219 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3973/7978] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:15.641431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:15.670454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cf209-ed4a-4914-94b9-ffe47ac8d64c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:15.681551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:15.682924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:15.683725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:15.683725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] PATH_INFO: `/v3/projects/415b8169e8234ea7825ce73b200877fe/users/4124666613fd45b29b1143681eac94f0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:15.716492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415b8169e8234ea7825ce73b200877fe, user_id=4124666613fd45b29b1143681eac94f0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:15.720912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:15.745239 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4006/7979] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:15 2022] PUT /identity/v3/projects/415b8169e8234ea7825ce73b200877fe/users/4124666613fd45b29b1143681eac94f0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:15.750144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:15.775753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d441-5f17-41af-acd9-cde4bddaa73c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:15.854358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:15.918121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca7049-4862-4cef-8cc1-f380787ef281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:15.919217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca7049-4862-4cef-8cc1-f380787ef281 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:15.919743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca7049-4862-4cef-8cc1-f380787ef281 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:15.920249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca7049-4862-4cef-8cc1-f380787ef281 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:15.921510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca7049-4862-4cef-8cc1-f380787ef281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:15.924308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca7049-4862-4cef-8cc1-f380787ef281 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:15.958116 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3974/7980] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:15.960837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:15.976251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1432-a891-4caa-97bc-189027d62eb2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:15.982969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:15.984038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:15.984350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:15.984787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] PATH_INFO: `/v3/projects/415b8169e8234ea7825ce73b200877fe/users/4124666613fd45b29b1143681eac94f0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:15.987086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415b8169e8234ea7825ce73b200877fe, user_id=4124666613fd45b29b1143681eac94f0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:15.991187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:15.994771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:16.019588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:16.019827 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4007/7981] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:15 2022] PUT /identity/v3/projects/415b8169e8234ea7825ce73b200877fe/users/4124666613fd45b29b1143681eac94f0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:16.020105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:16.020385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:16.067680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] MFA Rules not processed for user `4124666613fd45b29b1143681eac94f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:16.070134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:16.232690 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3975/7982] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:16 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:16.263205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:16.266535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:16.361508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3586cd-e411-4885-af79-b2be40b99877 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:16.372539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:16.373549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:16.374018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:16.374512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:16.566031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:16.569303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:16.591627 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4008/7983] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:16 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:17.388904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:17.391681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:17.576062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b4130-f8f9-4fee-8441-ffc46d4ad981 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:17.597531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771485e-7315-4877-a608-90448f6dc448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:17.598340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771485e-7315-4877-a608-90448f6dc448 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:17.598688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771485e-7315-4877-a608-90448f6dc448 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:17.599391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771485e-7315-4877-a608-90448f6dc448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:17.615372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771485e-7315-4877-a608-90448f6dc448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:17.617054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771485e-7315-4877-a608-90448f6dc448 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:17.656688 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3976/7984] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:17 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:26.578154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=1,keystone:SELECT=50 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:27.635146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=80 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:31.244368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:31.336179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28987b12-802c-407b-9465-4546b9e64d66 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:31.342239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:31.342960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:31.343297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:31.344085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] PATH_INFO: `/v3/users/cea4f128fceb411c81017f8abe1b62a0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:31.344953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea4f128fceb411c81017f8abe1b62a0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:31.346921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:31.403964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea4f128fceb411c81017f8abe1b62a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:31.404678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea4f128fceb411c81017f8abe1b62a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:31.418988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea4f128fceb411c81017f8abe1b62a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:31.419651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea4f128fceb411c81017f8abe1b62a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:31.444192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea4f128fceb411c81017f8abe1b62a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:31.466773 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4009/7985] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:31 2022] DELETE /identity/v3/users/cea4f128fceb411c81017f8abe1b62a0 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:31.468624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:31.469480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:31.469952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:31.494715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:31.502225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:31.636718 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3977/7986] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:31.678913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:31.680502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:31.760891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617e0d-9e9f-4c57-a79f-aa15523a3799 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:31.765385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:31.766096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:31.766398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:31.766612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:31.778253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:31.779853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:31.791246 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4010/7987] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:32.364717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:32.366395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:32.379976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb6c4f-d15b-43fb-afaf-43730563959d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:32.384782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:32.385419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:32.385823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:32.386077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] PATH_INFO: `/v3/projects/886e5284ea6841949c59477f969f2a90` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:32.387183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] RBAC: Authorizing `identity:delete_project(project_id=886e5284ea6841949c59477f969f2a90)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:32.389112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:32.445952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886e5284ea6841949c59477f969f2a90', 'request_id': 'req-d2b80090-3ef3-4708-973c-86c08b89436d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.456067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886e5284ea6841949c59477f969f2a90', 'request_id': 'req-d2b80090-3ef3-4708-973c-86c08b89436d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.463448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886e5284ea6841949c59477f969f2a90', 'request_id': 'req-d2b80090-3ef3-4708-973c-86c08b89436d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.475979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] The token cache is being invalidate because project 886e5284ea6841949c59477f969f2a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:32.476600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.477089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.478954 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3978/7988] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:32 2022] DELETE /identity/v3/projects/886e5284ea6841949c59477f969f2a90 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:32.482807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:32.494355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b890-2915-4508-b0d8-6590e31fbb7d service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:32.521348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:32.567614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:32.568661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:32.568935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:32.569201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] PATH_INFO: `/v3/users/4124666613fd45b29b1143681eac94f0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:32.570889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4124666613fd45b29b1143681eac94f0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:32.572826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:32.630669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4124666613fd45b29b1143681eac94f0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.631425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4124666613fd45b29b1143681eac94f0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.641130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4124666613fd45b29b1143681eac94f0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.641876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4124666613fd45b29b1143681eac94f0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.655679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c12f2b7e-4ce1-40b1-b4c4-fa00b3018154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4124666613fd45b29b1143681eac94f0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:32.679340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4011/7989] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:32 2022] DELETE /identity/v3/users/4124666613fd45b29b1143681eac94f0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:32.876382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:32.878302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:32.893157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b80090-3ef3-4708-973c-86c08b89436d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:32.958165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:32.958881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:32.959160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:32.959432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] PATH_INFO: `/v3/projects/415b8169e8234ea7825ce73b200877fe` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:32.960563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=415b8169e8234ea7825ce73b200877fe)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:32.962613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.007657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:33.036367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415b8169e8234ea7825ce73b200877fe', 'request_id': 'req-d5f22ce7-f32c-4528-9e60-b3cfd1916286'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:33.044426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415b8169e8234ea7825ce73b200877fe', 'request_id': 'req-d5f22ce7-f32c-4528-9e60-b3cfd1916286'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:33.050608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415b8169e8234ea7825ce73b200877fe', 'request_id': 'req-d5f22ce7-f32c-4528-9e60-b3cfd1916286'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:33.062324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] The token cache is being invalidate because project 415b8169e8234ea7825ce73b200877fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:33.062681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:33.063419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d5f22ce7-f32c-4528-9e60-b3cfd1916286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:33.065378 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3979/7990] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:32 2022] DELETE /identity/v3/projects/415b8169e8234ea7825ce73b200877fe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:33.071129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930c8cb-9b26-42ec-91fb-cbf82e97caa8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.071415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930c8cb-9b26-42ec-91fb-cbf82e97caa8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.071682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930c8cb-9b26-42ec-91fb-cbf82e97caa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.093244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f930c8cb-9b26-42ec-91fb-cbf82e97caa8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:33.094264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f930c8cb-9b26-42ec-91fb-cbf82e97caa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:33.155632 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4012/7991] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:33.163962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112749-c73a-4121-9062-dcec197ebcbc None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.164246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112749-c73a-4121-9062-dcec197ebcbc None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.165015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112749-c73a-4121-9062-dcec197ebcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.185824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b9112749-c73a-4121-9062-dcec197ebcbc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:33.187039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b9112749-c73a-4121-9062-dcec197ebcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:33.195571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3980/7992] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:33.200304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930c8cb-9b26-42ec-91fb-cbf82e97caa8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.210278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930c8cb-9b26-42ec-91fb-cbf82e97caa8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.216085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.216685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.216991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.217238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.217944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:33.218242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.219833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.232314 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4013/7993] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:33.236486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112749-c73a-4121-9062-dcec197ebcbc None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.246443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112749-c73a-4121-9062-dcec197ebcbc None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.251204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.251808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.252093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.252379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.253035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.254984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.285599 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3981/7994] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:33 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:33.289945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.301169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d25b1d-7043-4a0a-ac16-a3348f30a34a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.329900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:33.375612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.376252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.376580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.376858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.377571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.379506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.403163 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4014/7995] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:33 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:33.407702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.418824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9e814-2c48-46ad-a2a3-dcb84a8929b2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.424581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.425490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.425932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.426354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.427216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.429073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.440582 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3982/7996] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:33.444481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.454845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14021f-0895-424e-a8b8-7f6cd39359b6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.459615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e7d92-8dea-48bd-9415-413da570c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.460222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e7d92-8dea-48bd-9415-413da570c313 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.460510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e7d92-8dea-48bd-9415-413da570c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.460775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e7d92-8dea-48bd-9415-413da570c313 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.461418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e7d92-8dea-48bd-9415-413da570c313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.463049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e7d92-8dea-48bd-9415-413da570c313 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.473617 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4015/7997] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:33.477905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.493753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576d0b1-968d-40ca-bb6a-5073e3115492 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.499097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.499764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.500040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.500305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] PATH_INFO: `/v3/projects/e568cca6a00e4767a6ac0e71ec772db3/users/92565c5d19d94cfc8f116d0167217d73/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.517700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e568cca6a00e4767a6ac0e71ec772db3, user_id=92565c5d19d94cfc8f116d0167217d73, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.520453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.539755 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3983/7998] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:33 2022] PUT /identity/v3/projects/e568cca6a00e4767a6ac0e71ec772db3/users/92565c5d19d94cfc8f116d0167217d73/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:33.551375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2afdbc-b2fe-4463-bf71-0dfbdca033f8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.551944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2afdbc-b2fe-4463-bf71-0dfbdca033f8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.552058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2afdbc-b2fe-4463-bf71-0dfbdca033f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.583453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bf2afdbc-b2fe-4463-bf71-0dfbdca033f8 None None] MFA Rules not processed for user `92565c5d19d94cfc8f116d0167217d73`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:33.585043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bf2afdbc-b2fe-4463-bf71-0dfbdca033f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:33.660792 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4016/7999] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:33.665342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.759597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296b02e-a0e9-4014-abf2-564c94b4d6d0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.776997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.777672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.777951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.778215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.778867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.780988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.814943 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3984/8000] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:33 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:33.819277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2afdbc-b2fe-4463-bf71-0dfbdca033f8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.830750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2afdbc-b2fe-4463-bf71-0dfbdca033f8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.917850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.919167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.920092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.920453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.921899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.926450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:33.959385 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4017/8001] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:33 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:33.961280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:33.979949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d4448-0d56-4159-bf8f-cfbf7f202942 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:33.993329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:33.993811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:33.994227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:33.994469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:33.995212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:33.997425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:34.014368 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3985/8002] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:34.018676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:34.033990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c84e4-5cf2-4194-b0af-4c1a26e5c6ea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:34.035504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:34.036186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:34.036701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:34.036987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] PATH_INFO: `/v3/projects/33052c1dff4e44fdae2e581534a4e70d/users/0fe14bff3c654ebd9d409892d6355316/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:34.054422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33052c1dff4e44fdae2e581534a4e70d, user_id=0fe14bff3c654ebd9d409892d6355316, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:34.057192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:34.077422 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4018/8003] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:34 2022] PUT /identity/v3/projects/33052c1dff4e44fdae2e581534a4e70d/users/0fe14bff3c654ebd9d409892d6355316/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:34.083319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:34.095833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb718253-fb07-4386-82d3-66e71a248174 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:34.120654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:34.163810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e762b-22ba-4bc9-868f-7795bdeb5a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:34.164423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e762b-22ba-4bc9-868f-7795bdeb5a8a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:34.164728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e762b-22ba-4bc9-868f-7795bdeb5a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:34.164995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e762b-22ba-4bc9-868f-7795bdeb5a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:34.165778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e762b-22ba-4bc9-868f-7795bdeb5a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:34.167643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e762b-22ba-4bc9-868f-7795bdeb5a8a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:34.179451 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3986/8004] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:34.184726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:34.193175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:34.202716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24812548-f5f6-409e-8733-145d5681551f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:34.208705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:34.209453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:34.209821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:34.210162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] PATH_INFO: `/v3/projects/33052c1dff4e44fdae2e581534a4e70d/users/0fe14bff3c654ebd9d409892d6355316/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:34.212152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33052c1dff4e44fdae2e581534a4e70d, user_id=0fe14bff3c654ebd9d409892d6355316, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:34.214923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:34.227779 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4019/8005] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:34 2022] PUT /identity/v3/projects/33052c1dff4e44fdae2e581534a4e70d/users/0fe14bff3c654ebd9d409892d6355316/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:34.239573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:34.240181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:34.240522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:34.337407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] MFA Rules not processed for user `0fe14bff3c654ebd9d409892d6355316`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:34.339831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:34.534720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3987/8006] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:34.549370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:34.551756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:34.748127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb62fb8-88e5-4880-8ee7-fcc10a8e96f9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:34.760116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:34.760861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:34.761206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:34.761568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:34.782613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:34.784264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:34.796166 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4020/8007] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:35.097799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:35.099268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:35.114787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf92230-25cf-460b-98bd-b7cbe511f61e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:35.119003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f2881-8aae-4967-aa2e-c5234fba2c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:35.119662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2881-8aae-4967-aa2e-c5234fba2c1c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:35.119943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2881-8aae-4967-aa2e-c5234fba2c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:35.121085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2881-8aae-4967-aa2e-c5234fba2c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:35.128068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-166f2881-8aae-4967-aa2e-c5234fba2c1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:35.152274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f2881-8aae-4967-aa2e-c5234fba2c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:35.154000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f2881-8aae-4967-aa2e-c5234fba2c1c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:35.166276 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3988/8008] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:37.748627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:37.750263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:37.821318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d3d5a-c3c5-40cd-97ef-3dd556da41fe service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:37.826577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:37.827219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:37.827493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:37.827862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] PATH_INFO: `/v3/users/6f348e9d8be94f5badac4d1d8a58e2e3` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:37.829239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f348e9d8be94f5badac4d1d8a58e2e3)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:37.831222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:37.882741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f348e9d8be94f5badac4d1d8a58e2e3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:37.883387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f348e9d8be94f5badac4d1d8a58e2e3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:37.892525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f348e9d8be94f5badac4d1d8a58e2e3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:37.893447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f348e9d8be94f5badac4d1d8a58e2e3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:37.906253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f348e9d8be94f5badac4d1d8a58e2e3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:37.920533 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4021/8009] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:37 2022] DELETE /identity/v3/users/6f348e9d8be94f5badac4d1d8a58e2e3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:37.925358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:37.925905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:37.925905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:37.945576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:37.946826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.021838 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3989/8010] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:38.030294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:38.031825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.038643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:38.040133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.046898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8565e-8da7-4ff0-917f-d16703ae3b3c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:38.053269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:38.053992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.054264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.054713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] PATH_INFO: `/v3/users/92565c5d19d94cfc8f116d0167217d73` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.056051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=92565c5d19d94cfc8f116d0167217d73)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:38.058556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:38.064464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:38.100682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:38.132558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e02e7b-4d67-4401-82d7-3cc845ef38e0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:38.136711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92565c5d19d94cfc8f116d0167217d73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.138269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92565c5d19d94cfc8f116d0167217d73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.138864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046ee1d-9eef-4b3a-a07d-22ade133e242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:38.139544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046ee1d-9eef-4b3a-a07d-22ade133e242 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.139821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046ee1d-9eef-4b3a-a07d-22ade133e242 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.140094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046ee1d-9eef-4b3a-a07d-22ade133e242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.150416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92565c5d19d94cfc8f116d0167217d73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.151630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92565c5d19d94cfc8f116d0167217d73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.151760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046ee1d-9eef-4b3a-a07d-22ade133e242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:38.153835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046ee1d-9eef-4b3a-a07d-22ade133e242 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:38.167979 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3990/8011] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:38.168866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92565c5d19d94cfc8f116d0167217d73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.183306 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4022/8012] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:38 2022] DELETE /identity/v3/users/92565c5d19d94cfc8f116d0167217d73 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:38.187925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.188381 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.188835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.209680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:38.210815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.288370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3991/8013] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:38.310298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:38.311926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.390466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903930b-bb30-4423-a77a-d8ec86b2f4ba admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:38.394750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ec287-8962-4a13-a6cf-96b2f4ff61f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:38.395382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec287-8962-4a13-a6cf-96b2f4ff61f9 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.395701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec287-8962-4a13-a6cf-96b2f4ff61f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.396137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec287-8962-4a13-a6cf-96b2f4ff61f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.406207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ec287-8962-4a13-a6cf-96b2f4ff61f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:38.407709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ec287-8962-4a13-a6cf-96b2f4ff61f9 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:38.418423 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4023/8014] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:21:38.635217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:38.638180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.651799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe14909-6263-43ff-b6fa-557215f250c0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:38.657729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:38.658626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.659006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.659373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] PATH_INFO: `/v3/projects/89ac3ed817a84d97aee955fc79bdd75e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.660718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ac3ed817a84d97aee955fc79bdd75e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:38.662823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:38.748001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ac3ed817a84d97aee955fc79bdd75e', 'request_id': 'req-4283da0c-08e0-44be-b16a-3c52cd3a23e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.761681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ac3ed817a84d97aee955fc79bdd75e', 'request_id': 'req-4283da0c-08e0-44be-b16a-3c52cd3a23e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.770291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ac3ed817a84d97aee955fc79bdd75e', 'request_id': 'req-4283da0c-08e0-44be-b16a-3c52cd3a23e6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.784676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] The token cache is being invalidate because project 89ac3ed817a84d97aee955fc79bdd75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:38.785131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.785912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4283da0c-08e0-44be-b16a-3c52cd3a23e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:38.787943 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3992/8015] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:38 2022] DELETE /identity/v3/projects/89ac3ed817a84d97aee955fc79bdd75e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:38.792829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea7700-54dc-452f-b29a-a560b6da5f3e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.793387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea7700-54dc-452f-b29a-a560b6da5f3e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.793587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea7700-54dc-452f-b29a-a560b6da5f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.814699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c7ea7700-54dc-452f-b29a-a560b6da5f3e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:38.816070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c7ea7700-54dc-452f-b29a-a560b6da5f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.891215 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4024/8016] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:38.900431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b383-4515-4e2a-9a90-20424788a2c8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.900714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b383-4515-4e2a-9a90-20424788a2c8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.900990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b383-4515-4e2a-9a90-20424788a2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.923208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9fb6b383-4515-4e2a-9a90-20424788a2c8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:38.924463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9fb6b383-4515-4e2a-9a90-20424788a2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:38.933834 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3993/8017] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:38.938252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea7700-54dc-452f-b29a-a560b6da5f3e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:38.949240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea7700-54dc-452f-b29a-a560b6da5f3e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:38.954213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:38.954841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.955118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.955387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.956046 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:38.956348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:38.958010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:38.972297 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4025/8018] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:38.976775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6b383-4515-4e2a-9a90-20424788a2c8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:38.989307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6b383-4515-4e2a-9a90-20424788a2c8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:38.996173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:38.997045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:38.997486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:38.997932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:38.998739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.001261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.049761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3994/8019] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:38 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:39.053851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.066958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2914c-031c-43b1-a207-169a41f49ecf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.082077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:39.141699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.142324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.142602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.142879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.143509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.145592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.170544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4026/8020] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:39 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:39.174479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.186103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60500ef-fadb-40f4-a08c-d9ba0658eb38 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.190833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.191465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.191742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.192024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.192726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.194464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.209818 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3995/8021] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:39.211235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.224644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119150e3-077b-43d7-a2be-9c6cc58313b7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.239092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1029ec3-a414-48a8-803a-c2ffd5a9a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.239869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1029ec3-a414-48a8-803a-c2ffd5a9a0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.240130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1029ec3-a414-48a8-803a-c2ffd5a9a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.240489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1029ec3-a414-48a8-803a-c2ffd5a9a0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.245429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1029ec3-a414-48a8-803a-c2ffd5a9a0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.261087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1029ec3-a414-48a8-803a-c2ffd5a9a0c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.279234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4027/8022] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:39.283525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.299169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b582d0-df19-4bdf-923c-85fe94284823 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.306668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.308033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.308430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.308957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] PATH_INFO: `/v3/projects/ca47da09910d414b8bb961b89e7faf91/users/db49a2f921b94f8b8122affaf5d384ec/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.359679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca47da09910d414b8bb961b89e7faf91, user_id=db49a2f921b94f8b8122affaf5d384ec, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.363899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.372912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:39.399648 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3996/8023] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:39 2022] PUT /identity/v3/projects/ca47da09910d414b8bb961b89e7faf91/users/db49a2f921b94f8b8122affaf5d384ec/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:39.401493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.401950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.402286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.446956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] MFA Rules not processed for user `db49a2f921b94f8b8122affaf5d384ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:39.448199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:39.490680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.494999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:39.552109 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4028/8024] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:39.580615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.582770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:39.624231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98a52a-90b7-4136-8cd1-c9bbee123de7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.629966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.630683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.633019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.633019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] PATH_INFO: `/v3/projects/e568cca6a00e4767a6ac0e71ec772db3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.633401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e568cca6a00e4767a6ac0e71ec772db3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.635568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.660489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cbc45-1922-4ab8-8813-4a3dbff48835 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.664277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.665188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.665408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.665712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.675804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.677234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.688258 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4029/8025] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:21:39.702432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.713974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4acf2-61a9-4286-b437-ddeb4747a258 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.718775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e568cca6a00e4767a6ac0e71ec772db3', 'request_id': 'req-79fc99ea-ca5c-4fb4-a566-c88d881dad28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.728526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e568cca6a00e4767a6ac0e71ec772db3', 'request_id': 'req-79fc99ea-ca5c-4fb4-a566-c88d881dad28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.736057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e568cca6a00e4767a6ac0e71ec772db3', 'request_id': 'req-79fc99ea-ca5c-4fb4-a566-c88d881dad28'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.740340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:39.749356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] The token cache is being invalidate because project e568cca6a00e4767a6ac0e71ec772db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:39.749725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.750474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.752494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3997/8026] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:39 2022] DELETE /identity/v3/projects/e568cca6a00e4767a6ac0e71ec772db3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:39.756598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:39.758378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:39.785663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.787062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.787517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.788188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] PATH_INFO: `/v3/users/db49a2f921b94f8b8122affaf5d384ec` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.789880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db49a2f921b94f8b8122affaf5d384ec)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.792170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.832235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc99ea-ca5c-4fb4-a566-c88d881dad28 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:39.837248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:39.838042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.838321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.838703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] PATH_INFO: `/v3/users/0fe14bff3c654ebd9d409892d6355316` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.839851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe14bff3c654ebd9d409892d6355316)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:39.842168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:39.850979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db49a2f921b94f8b8122affaf5d384ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.852345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db49a2f921b94f8b8122affaf5d384ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.862949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db49a2f921b94f8b8122affaf5d384ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.864024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db49a2f921b94f8b8122affaf5d384ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.876410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:39.879100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0a6b6a52-2ff0-4a8b-b413-125a28e8f738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db49a2f921b94f8b8122affaf5d384ec'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.896956 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4030/8027] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:39 2022] DELETE /identity/v3/users/db49a2f921b94f8b8122affaf5d384ec => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:39.901567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:39.901854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:39.902130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:39.904014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe14bff3c654ebd9d409892d6355316'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.921192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:39.922464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:39.923424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe14bff3c654ebd9d409892d6355316'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.933840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe14bff3c654ebd9d409892d6355316'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.935625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe14bff3c654ebd9d409892d6355316'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.936731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe14bff3c654ebd9d409892d6355316'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:39.952561 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3998/8028] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:39 2022] DELETE /identity/v3/users/0fe14bff3c654ebd9d409892d6355316 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:39.997678 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4031/8029] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:40.019660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.024844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:40.101326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1323ef-cee9-453b-ae71-69b1815e47a3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:40.105156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:40.105832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.106104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.106379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.177060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:40.179206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:40.192171 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 3999/8030] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:40.474514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.476466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:40.485778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.487286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:40.525423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f61cd-76ce-43e5-ab37-6e71aed84f70 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:40.526778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66d79c-ae25-4b9c-a3cb-414758a275cf None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:40.546868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:40.547605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.547883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.548191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] PATH_INFO: `/v3/projects/33052c1dff4e44fdae2e581534a4e70d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.549007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:40.549319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=33052c1dff4e44fdae2e581534a4e70d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:40.550077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.550396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.550672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] PATH_INFO: `/v3/projects/ca47da09910d414b8bb961b89e7faf91` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.551282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:40.552051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca47da09910d414b8bb961b89e7faf91)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:40.554390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:40.613541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:40.616725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca47da09910d414b8bb961b89e7faf91', 'request_id': 'req-f012e819-43e7-49f5-a331-372feab124e2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.620418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:40.623696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33052c1dff4e44fdae2e581534a4e70d', 'request_id': 'req-be51b8f9-871d-4c49-a650-9785aa9f7eaf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.632175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33052c1dff4e44fdae2e581534a4e70d', 'request_id': 'req-be51b8f9-871d-4c49-a650-9785aa9f7eaf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.632373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca47da09910d414b8bb961b89e7faf91', 'request_id': 'req-f012e819-43e7-49f5-a331-372feab124e2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.641439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33052c1dff4e44fdae2e581534a4e70d', 'request_id': 'req-be51b8f9-871d-4c49-a650-9785aa9f7eaf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.641699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca47da09910d414b8bb961b89e7faf91', 'request_id': 'req-f012e819-43e7-49f5-a331-372feab124e2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.660887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] The token cache is being invalidate because project 33052c1dff4e44fdae2e581534a4e70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:40.661191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.662249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] The token cache is being invalidate because project ca47da09910d414b8bb961b89e7faf91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:40.662669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-be51b8f9-871d-4c49-a650-9785aa9f7eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.662992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.663377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f012e819-43e7-49f5-a331-372feab124e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:40.665202 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4000/8032] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:40 2022] DELETE /identity/v3/projects/ca47da09910d414b8bb961b89e7faf91 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:40.665532 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4032/8032] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:40 2022] DELETE /identity/v3/projects/33052c1dff4e44fdae2e581534a4e70d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:40.675085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c15347-4050-48bc-ab36-ef61cbcd3746 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.675265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b845298-f0a8-4efe-b2c9-6225c6489849 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.675357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c15347-4050-48bc-ab36-ef61cbcd3746 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.675459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b845298-f0a8-4efe-b2c9-6225c6489849 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.675704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c15347-4050-48bc-ab36-ef61cbcd3746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.675811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b845298-f0a8-4efe-b2c9-6225c6489849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.699592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-65c15347-4050-48bc-ab36-ef61cbcd3746 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:40.699842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4b845298-f0a8-4efe-b2c9-6225c6489849 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:40.700984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-65c15347-4050-48bc-ab36-ef61cbcd3746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:40.701612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4b845298-f0a8-4efe-b2c9-6225c6489849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:40.776940 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4033/8033] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:40.778399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6ffd-4afa-4a03-ad67-6e8211bde44a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.778635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6ffd-4afa-4a03-ad67-6e8211bde44a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.778910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6ffd-4afa-4a03-ad67-6e8211bde44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.786899 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4001/8034] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:40.795524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4c4d9-5920-4228-b9d7-e73292ec0829 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.795833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4c4d9-5920-4228-b9d7-e73292ec0829 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.796337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4c4d9-5920-4228-b9d7-e73292ec0829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.807621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c84b6ffd-4afa-4a03-ad67-6e8211bde44a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:40.809290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c84b6ffd-4afa-4a03-ad67-6e8211bde44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:40.819033 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4034/8035] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:40.823543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b6ffd-4afa-4a03-ad67-6e8211bde44a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.823699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6ca4c4d9-5920-4228-b9d7-e73292ec0829 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:40.825741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6ca4c4d9-5920-4228-b9d7-e73292ec0829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:40.837716 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4002/8036] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:40.837936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b6ffd-4afa-4a03-ad67-6e8211bde44a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:40.839013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4c4d9-5920-4228-b9d7-e73292ec0829 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.842378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:40.843001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.843332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.843607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.844264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:40.844573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:40.846275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:40.850533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4c4d9-5920-4228-b9d7-e73292ec0829 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:40.860238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:40.860854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.861130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.861609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.865430 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4035/8037] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:21:40.865933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:40.865933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:40.865933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:40.866568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.881816 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4003/8038] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:21:40.882052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dbb76-3593-4b20-8b3b-38a176b064a7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:40.882801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.890497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:40.891222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.892398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.892398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.893491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:40.895406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:40.895560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24ed4c-d759-408a-b7c5-3f09daf73c58 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:40.901257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:40.902011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:40.902357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:40.902688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:40.903387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:40.905368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:40.931787 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4036/8039] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:40.935749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4004/8040] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:40.938023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:40.939432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.032290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099b2d5-bf32-410b-ac75-400bc3b869e7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.037038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf855-9735-40d5-82c4-215734f15c82 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.039341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.040191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.040637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.041051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.041877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.042627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.043350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.043713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.044050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.044757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.045061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.047076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.077063 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4005/8041] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:41.082098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.083324 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4037/8042] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:40 2022] POST /identity/v3/users => generated 283 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:41.087507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.099194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92d404-ac1e-46ec-b154-3465bbb39fa2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.103743 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835796-07d8-402e-98c6-892c081093c3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.111234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.111996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.112346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.112678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.113588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.113841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.114748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.115195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.115692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.116521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.119366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.119938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.135946 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4006/8043] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:41.140199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.145644 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4038/8044] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:41.150613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.159996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:41.167472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0b3cd-f14a-4f13-8b0f-4744e12c6e85 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.168461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a19a32-19c0-43d6-8cc7-30dd1f0f7c4b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.176809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.179185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.179916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.180399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.181022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.181295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.181666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.182201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.182321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.183175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.183885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.186500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.206918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4039/8045] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:41.213221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.226626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682c19a-4882-4802-a17c-da3d2924924d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.234814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cefab5-0582-410e-b0dc-8f8bcc851332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.235546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefab5-0582-410e-b0dc-8f8bcc851332 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.235881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefab5-0582-410e-b0dc-8f8bcc851332 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.236248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefab5-0582-410e-b0dc-8f8bcc851332 admin admin] PATH_INFO: `/v3/projects/f56985d1ef0d4933a198777d54291188/users/d1985084753149a8adff9093ae646e63/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.245508 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4007/8046] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:41.246964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.259951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:41.263914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cefab5-0582-410e-b0dc-8f8bcc851332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56985d1ef0d4933a198777d54291188, user_id=d1985084753149a8adff9093ae646e63, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.264409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8919d21-319f-4681-b02b-a77c46a3af3e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.272665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cefab5-0582-410e-b0dc-8f8bcc851332 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.280988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d7a84-9c1b-4cd5-837e-6539c2b8afb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.281771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d7a84-9c1b-4cd5-837e-6539c2b8afb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.282100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d7a84-9c1b-4cd5-837e-6539c2b8afb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.282396 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d7a84-9c1b-4cd5-837e-6539c2b8afb7 admin admin] PATH_INFO: `/v3/projects/7f5383a872224bc1baa4de128268596c/users/29f13ae6935146599f9ac42e2b752f43/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.315660 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4040/8047] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:41 2022] PUT /identity/v3/projects/f56985d1ef0d4933a198777d54291188/users/d1985084753149a8adff9093ae646e63/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:41.316919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d7a84-9c1b-4cd5-837e-6539c2b8afb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5383a872224bc1baa4de128268596c, user_id=29f13ae6935146599f9ac42e2b752f43, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.322319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d7a84-9c1b-4cd5-837e-6539c2b8afb7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.354365 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4008/8048] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:41 2022] PUT /identity/v3/projects/7f5383a872224bc1baa4de128268596c/users/29f13ae6935146599f9ac42e2b752f43/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:41.379339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.379614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.379887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.420143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a12c3-6e42-4ee0-ac9e-eb47627c57d5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.420425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a12c3-6e42-4ee0-ac9e-eb47627c57d5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.420831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a12c3-6e42-4ee0-ac9e-eb47627c57d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.433254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] MFA Rules not processed for user `29f13ae6935146599f9ac42e2b752f43`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:41.435023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:41.467688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-459a12c3-6e42-4ee0-ac9e-eb47627c57d5 None None] MFA Rules not processed for user `d1985084753149a8adff9093ae646e63`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:41.469009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-459a12c3-6e42-4ee0-ac9e-eb47627c57d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:41.529392 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4041/8049] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:41.540633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.546658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:41.548193 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4009/8050] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:41.552588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a12c3-6e42-4ee0-ac9e-eb47627c57d5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.629601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58aa5a-4dd1-4ccc-b180-6efe98c986f0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.633958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.634779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a12c3-6e42-4ee0-ac9e-eb47627c57d5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.635198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.635677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.636351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.640111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.640851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.641225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.642418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.643163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.645696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.650166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.651630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.662103 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4042/8051] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:41.697199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.707284 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4010/8052] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:41 2022] POST /identity/v3/projects => generated 365 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:41.726588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0a7b6-54ee-46e5-ad91-03acb84ebd3a service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.807937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.808588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.808869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.809142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.809839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.813328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.829505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:41.848599 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4043/8053] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:41 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:41.852755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.865635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456edfb-7f76-4a23-97ba-eb7a4880b76b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.870761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.871882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.872523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.872523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.873064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.875441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.890156 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4011/8054] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:41.893063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:41.910424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2ee4b-126f-4b11-8b74-26d2e687bcd3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:41.915471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:41.916140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:41.916414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:41.916682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] PATH_INFO: `/v3/projects/011a13fb43c744d3959edc787411bb4d/users/dd2ce8de644b4ba6a9418329de84ee0e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:41.940666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] RBAC: Authorizing `identity:create_grant(project_id=011a13fb43c744d3959edc787411bb4d, user_id=dd2ce8de644b4ba6a9418329de84ee0e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:41.944515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:41.988512 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4044/8055] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:41 2022] PUT /identity/v3/projects/011a13fb43c744d3959edc787411bb4d/users/dd2ce8de644b4ba6a9418329de84ee0e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:41.990128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:42.002321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b9434-79e7-4516-bb95-4afa0c4e3895 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:42.076023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:42.102695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b28032-9969-4c04-ae51-b2b0fbb80c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:42.103771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b28032-9969-4c04-ae51-b2b0fbb80c04 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:42.104239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b28032-9969-4c04-ae51-b2b0fbb80c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:42.105043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b28032-9969-4c04-ae51-b2b0fbb80c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:42.106293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b28032-9969-4c04-ae51-b2b0fbb80c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:42.109027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b28032-9969-4c04-ae51-b2b0fbb80c04 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:42.129074 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4012/8056] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:42.133535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:42.144645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e7e5-0dba-498b-aedc-6ce072c3bdde admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:42.149948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:42.150275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:42.150558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:42.150835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] PATH_INFO: `/v3/projects/011a13fb43c744d3959edc787411bb4d/users/dd2ce8de644b4ba6a9418329de84ee0e/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:42.152818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=011a13fb43c744d3959edc787411bb4d, user_id=dd2ce8de644b4ba6a9418329de84ee0e, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:42.155713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:42.176106 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4045/8057] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:42 2022] PUT /identity/v3/projects/011a13fb43c744d3959edc787411bb4d/users/dd2ce8de644b4ba6a9418329de84ee0e/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:42.183256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ba24b-e9db-4c29-8847-9ba26f80fc5f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:42.183595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ba24b-e9db-4c29-8847-9ba26f80fc5f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:42.183814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ba24b-e9db-4c29-8847-9ba26f80fc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:42.238659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c04ba24b-e9db-4c29-8847-9ba26f80fc5f None None] MFA Rules not processed for user `dd2ce8de644b4ba6a9418329de84ee0e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:42.239809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c04ba24b-e9db-4c29-8847-9ba26f80fc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:42.347253 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4013/8058] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:42.359638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:42.361394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:42.441834 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1bc18-c846-45b7-8da3-9dc5a5b802ba admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:42.445621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:42.446252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:42.446533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:42.446806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:42.456746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:42.458278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:42.469176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4046/8059] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:42.579880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ba24b-e9db-4c29-8847-9ba26f80fc5f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:42.610454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ba24b-e9db-4c29-8847-9ba26f80fc5f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:42.619637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:42.621155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:42.621578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:42.624529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:42.779956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:42.781560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:42.793018 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4014/8060] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:43.405523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:43.406664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:43.581930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217be0d-6f38-4e1c-911e-1068dcea536d service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:43.593252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea81bc-fe99-4485-9cc9-c742e257a4c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:43.595503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea81bc-fe99-4485-9cc9-c742e257a4c5 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:43.597059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea81bc-fe99-4485-9cc9-c742e257a4c5 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:43.597612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea81bc-fe99-4485-9cc9-c742e257a4c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:43.600865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e6ea81bc-fe99-4485-9cc9-c742e257a4c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:43.624810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea81bc-fe99-4485-9cc9-c742e257a4c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:43.626964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea81bc-fe99-4485-9cc9-c742e257a4c5 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:43.648053 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4047/8061] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:46.018096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:46.103273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378906a-2138-4005-bc78-1f633a6eb4ae service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:46.111727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:46.113600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:46.114015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:46.114298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] PATH_INFO: `/v3/users/d1985084753149a8adff9093ae646e63` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:46.115595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1985084753149a8adff9093ae646e63)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:46.118693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:46.126855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:46.215206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1985084753149a8adff9093ae646e63'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:46.236143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1985084753149a8adff9093ae646e63'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:46.251307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1985084753149a8adff9093ae646e63'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:46.253301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1985084753149a8adff9093ae646e63'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:46.266171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1985084753149a8adff9093ae646e63'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:46.305746 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4015/8062] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:46 2022] DELETE /identity/v3/users/d1985084753149a8adff9093ae646e63 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:46.310234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:46.310778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:46.310778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:46.349275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:46.350469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:46.406358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:46.451676 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4048/8063] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:46.470670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:46.472194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:46.660831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c5bd8-8b3b-4299-9dab-8ed3184ff8b2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:46.695098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:46.695760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:46.696040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:46.696313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:46.716151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:46.717989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:46.739161 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4016/8064] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:47.531358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:47.533124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:47.546062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f2b60-5922-4641-a1de-8f3c5793f568 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:47.551190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:47.552040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:47.559464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:47.560001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] PATH_INFO: `/v3/projects/f56985d1ef0d4933a198777d54291188` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:47.561656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f56985d1ef0d4933a198777d54291188)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:47.563920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:47.644202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f56985d1ef0d4933a198777d54291188', 'request_id': 'req-13719d3c-7d36-4066-bc03-8fd16410f828'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.650912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f56985d1ef0d4933a198777d54291188', 'request_id': 'req-13719d3c-7d36-4066-bc03-8fd16410f828'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.663539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f56985d1ef0d4933a198777d54291188', 'request_id': 'req-13719d3c-7d36-4066-bc03-8fd16410f828'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.683908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] The token cache is being invalidate because project f56985d1ef0d4933a198777d54291188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:47.684199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.685129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.691846 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4049/8065] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:47 2022] DELETE /identity/v3/projects/f56985d1ef0d4933a198777d54291188 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:47.697377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:47.730504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a06b2-9e8e-45a8-bcc9-7149c8740849 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:47.813337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:47.814315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:47.814801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:47.815261 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] PATH_INFO: `/v3/users/dd2ce8de644b4ba6a9418329de84ee0e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:47.816618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2ce8de644b4ba6a9418329de84ee0e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:47.818702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:47.853861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:47.897774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2ce8de644b4ba6a9418329de84ee0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.918053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2ce8de644b4ba6a9418329de84ee0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.929751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2ce8de644b4ba6a9418329de84ee0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.932678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2ce8de644b4ba6a9418329de84ee0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.933429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4c8604d1-5192-4de4-b264-4fe1ce889865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2ce8de644b4ba6a9418329de84ee0e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:47.992560 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4017/8066] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:47 2022] DELETE /identity/v3/users/dd2ce8de644b4ba6a9418329de84ee0e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:48.436558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:48.438199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:48.455621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13719d3c-7d36-4066-bc03-8fd16410f828 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:48.529354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:48.530282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:48.530282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:48.530474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] PATH_INFO: `/v3/users/b4d3effbde7243bf859a8bb58dc6924d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:48.531725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d3effbde7243bf859a8bb58dc6924d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:48.534004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:48.589188 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d3effbde7243bf859a8bb58dc6924d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:48.590244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d3effbde7243bf859a8bb58dc6924d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:48.602041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d3effbde7243bf859a8bb58dc6924d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:48.602877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d3effbde7243bf859a8bb58dc6924d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:48.617626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:48.622123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d3effbde7243bf859a8bb58dc6924d'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:48.639442 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4050/8067] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:48 2022] DELETE /identity/v3/users/b4d3effbde7243bf859a8bb58dc6924d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:48.649176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:48.649317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:48.649644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:48.672138 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:48.673392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:48.697228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:48.699453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:48.803815 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4018/8068] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:48.818398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:48.820580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:48.828866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b71210-1069-4a31-ae1f-d46d2d727e15 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:48.835633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:48.836524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:48.837006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:48.837440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] PATH_INFO: `/v3/projects/011a13fb43c744d3959edc787411bb4d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:48.838820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011a13fb43c744d3959edc787411bb4d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:48.841034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:48.898286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27d134-dd39-4c7f-8361-426861f5408c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:48.902752 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d3b6e-7be4-403b-a330-7f6ebecfc43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:48.903313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d3b6e-7be4-403b-a330-7f6ebecfc43b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:48.903598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d3b6e-7be4-403b-a330-7f6ebecfc43b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:48.903877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d3b6e-7be4-403b-a330-7f6ebecfc43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:48.915102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d3b6e-7be4-403b-a330-7f6ebecfc43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:48.916628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d3b6e-7be4-403b-a330-7f6ebecfc43b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:48.928888 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4019/8069] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:49.092504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011a13fb43c744d3959edc787411bb4d', 'request_id': 'req-97bb8815-65ae-420b-957f-477a60ad79a3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.101640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011a13fb43c744d3959edc787411bb4d', 'request_id': 'req-97bb8815-65ae-420b-957f-477a60ad79a3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.124299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011a13fb43c744d3959edc787411bb4d', 'request_id': 'req-97bb8815-65ae-420b-957f-477a60ad79a3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.143297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] The token cache is being invalidate because project 011a13fb43c744d3959edc787411bb4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:49.144179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.145258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-97bb8815-65ae-420b-957f-477a60ad79a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.153682 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4051/8070] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:48 2022] DELETE /identity/v3/projects/011a13fb43c744d3959edc787411bb4d => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:49.154372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cec566-9db9-48a3-b1c3-7b5db049973c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.154698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cec566-9db9-48a3-b1c3-7b5db049973c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.155160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cec566-9db9-48a3-b1c3-7b5db049973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.183301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-51cec566-9db9-48a3-b1c3-7b5db049973c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:49.187317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-51cec566-9db9-48a3-b1c3-7b5db049973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:49.375132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:49.416509 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4020/8071] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:21:49.425977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926dc38-76e1-41bf-a2e3-a939c653ed1f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.426285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926dc38-76e1-41bf-a2e3-a939c653ed1f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.426543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926dc38-76e1-41bf-a2e3-a939c653ed1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.468642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2926dc38-76e1-41bf-a2e3-a939c653ed1f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:49.469925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2926dc38-76e1-41bf-a2e3-a939c653ed1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:49.490853 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4052/8072] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:49.491516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cec566-9db9-48a3-b1c3-7b5db049973c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:49.506073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cec566-9db9-48a3-b1c3-7b5db049973c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:49.529786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:49.531473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.533063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.533643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.534547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:49.535143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:49.538233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:49.561102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926dc38-76e1-41bf-a2e3-a939c653ed1f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:49.592768 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4021/8073] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:49.600430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926dc38-76e1-41bf-a2e3-a939c653ed1f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:49.607894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:49.609040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.609622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.610104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.611009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:49.613209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:49.625047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:49.628360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:49.647145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055f9f-aa26-41d5-b722-b91bf7992343 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:49.654201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:49.655483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.656164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.656684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] PATH_INFO: `/v3/projects/2007b313194d4a2c8a266e8ca407bd34` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.659032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2007b313194d4a2c8a266e8ca407bd34)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:49.661715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:49.690081 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4053/8074] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:49 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:49.694349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:49.762484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2007b313194d4a2c8a266e8ca407bd34', 'request_id': 'req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.775248 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2007b313194d4a2c8a266e8ca407bd34', 'request_id': 'req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.781604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:49.788249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2007b313194d4a2c8a266e8ca407bd34', 'request_id': 'req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.810873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9c5dc-0714-46b0-a101-3896d4e7bc0d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:49.813001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] The token cache is being invalidate because project 2007b313194d4a2c8a266e8ca407bd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:49.814193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.815504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:49.822320 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4022/8075] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:49 2022] DELETE /identity/v3/projects/2007b313194d4a2c8a266e8ca407bd34 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:49.826608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:49.830225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:49.897825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:49.898769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.899211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.899650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.900621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:49.902952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:49.937974 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4054/8076] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:49 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:21:49.939899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:49.942601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:49.943746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32adb-43c6-4f7a-8402-1e0a4d6bc58e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:49.950121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:49.951175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.951660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.952087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] PATH_INFO: `/v3/users/a157ec614b95412a820ead30ee3d8691` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.953440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a157ec614b95412a820ead30ee3d8691)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:49.955613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:49.962147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a31b7e-9696-47f6-92db-ddeae2266419 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:49.969380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:49.970141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:49.971159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:49.971523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:49.972639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:49.976489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:49.996213 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4055/8077] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:50.003651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:50.019392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d494f-dcc9-4031-ad29-d35f39c0c42d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:50.028199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:50.029164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.029833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.030681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.032106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:50.034384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:50.043934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a157ec614b95412a820ead30ee3d8691'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.051488 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4056/8078] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:50.057487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:50.078048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:50.084629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a157ec614b95412a820ead30ee3d8691'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.087183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a058-0f88-4393-80e9-033570dcc8e5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:50.093854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:50.094662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.094923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.095194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] PATH_INFO: `/v3/projects/d37289e9d7394e5b86c0b7c10f97b365/users/d46bf9c0b90f4589989b468cb4feec28/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.106772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a157ec614b95412a820ead30ee3d8691'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.107739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a157ec614b95412a820ead30ee3d8691'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.109008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3ad6d362-e050-463e-b606-9c6c0f9b48a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a157ec614b95412a820ead30ee3d8691'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.117570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37289e9d7394e5b86c0b7c10f97b365, user_id=d46bf9c0b90f4589989b468cb4feec28, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:50.121030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:50.129239 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4023/8079] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:49 2022] DELETE /identity/v3/users/a157ec614b95412a820ead30ee3d8691 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:50.147101 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4057/8080] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:50 2022] PUT /identity/v3/projects/d37289e9d7394e5b86c0b7c10f97b365/users/d46bf9c0b90f4589989b468cb4feec28/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:50.159204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.159632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.160069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.209717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] MFA Rules not processed for user `d46bf9c0b90f4589989b468cb4feec28`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:50.210958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:50.302100 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4024/8081] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:50.315295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:50.317443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:50.356719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:50.358218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:50.442652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314298e5-351b-4957-ac70-7a1cde469f3c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:50.452256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc34cfc-310d-4597-9611-0e98ec8b1a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:50.453006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc34cfc-310d-4597-9611-0e98ec8b1a85 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.453356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc34cfc-310d-4597-9611-0e98ec8b1a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.454085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc34cfc-310d-4597-9611-0e98ec8b1a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.466542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc34cfc-310d-4597-9611-0e98ec8b1a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:50.468576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc34cfc-310d-4597-9611-0e98ec8b1a85 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:50.474763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd18b42-ec23-4126-b308-e59f4eea4008 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:50.484374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:50.485097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.485227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.485535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] PATH_INFO: `/v3/projects/152496f26d16469a9df94d5abe65ef6b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.488443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=152496f26d16469a9df94d5abe65ef6b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:50.488734 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4058/8082] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:50.490447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:50.560245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152496f26d16469a9df94d5abe65ef6b', 'request_id': 'req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.572536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152496f26d16469a9df94d5abe65ef6b', 'request_id': 'req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.583051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152496f26d16469a9df94d5abe65ef6b', 'request_id': 'req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.593536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:50.600029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] The token cache is being invalidate because project 152496f26d16469a9df94d5abe65ef6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:50.600456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.601236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-92e103e2-fd2c-45c5-8954-dd4065a6bd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:50.606394 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4025/8083] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:50 2022] DELETE /identity/v3/projects/152496f26d16469a9df94d5abe65ef6b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:21:50.608472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6f0c5-1324-4d17-ad80-9d2fa110b404 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.608936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6f0c5-1324-4d17-ad80-9d2fa110b404 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.609154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6f0c5-1324-4d17-ad80-9d2fa110b404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.643880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6db6f0c5-1324-4d17-ad80-9d2fa110b404 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:50.646858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6db6f0c5-1324-4d17-ad80-9d2fa110b404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:50.706778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:50.773787 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4059/8084] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:50.780504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea30d3-10b6-4006-9b14-caf7ffd4e48c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.780853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea30d3-10b6-4006-9b14-caf7ffd4e48c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.781073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea30d3-10b6-4006-9b14-caf7ffd4e48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.815007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8cea30d3-10b6-4006-9b14-caf7ffd4e48c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:50.816220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8cea30d3-10b6-4006-9b14-caf7ffd4e48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:50.833792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6f0c5-1324-4d17-ad80-9d2fa110b404 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:50.842026 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4026/8085] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:50.850206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6f0c5-1324-4d17-ad80-9d2fa110b404 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:50.856542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:50.857221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.857558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.857829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.859145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:21:50.859453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:50.861406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:50.902098 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4060/8086] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:21:50.907006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea30d3-10b6-4006-9b14-caf7ffd4e48c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:50.919027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea30d3-10b6-4006-9b14-caf7ffd4e48c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:50.952942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:50.954129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:50.954394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:50.954669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:50.955331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:50.957291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:51.012023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4027/8087] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:21:50 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:51.016659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:51.032242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70642433-3aeb-4375-8205-99f6768636ac admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:51.146464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:51.148593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:51.148900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:51.149171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:51.151141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:51.157964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:51.218987 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4061/8088] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:21:51 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:21:51.219928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:51.233227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534ae23-a6a1-46bd-ace6-72b2b3b57b28 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:51.238435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:51.239238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:51.239800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:51.239907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:51.240845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:51.243264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:51.259840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:51.267791 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4028/8089] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:21:51.271937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bd6d1-43a5-4c86-ae02-58391ce8b0cd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:51.277603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da5393-9c30-44f8-b0a0-c4f7523eb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:51.278380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da5393-9c30-44f8-b0a0-c4f7523eb16e admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:51.278655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da5393-9c30-44f8-b0a0-c4f7523eb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:51.278971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da5393-9c30-44f8-b0a0-c4f7523eb16e admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:51.279794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da5393-9c30-44f8-b0a0-c4f7523eb16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:51.281777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da5393-9c30-44f8-b0a0-c4f7523eb16e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:51.297687 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4062/8090] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:21:51.303231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:51.317556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe4a25-bc54-48ff-819e-62854cedbb4c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:51.322605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:51.323410 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:51.323919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:51.324191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] PATH_INFO: `/v3/projects/0a6ae6b02e0a44148cfa744c94b9953b/users/14e08de43e3f4b5a83b00674f426e438/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:51.343128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6ae6b02e0a44148cfa744c94b9953b, user_id=14e08de43e3f4b5a83b00674f426e438, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:51.353841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:51.376185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4029/8091] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:21:51 2022] PUT /identity/v3/projects/0a6ae6b02e0a44148cfa744c94b9953b/users/14e08de43e3f4b5a83b00674f426e438/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:51.389682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:51.389982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:51.391192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:51.443548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] MFA Rules not processed for user `14e08de43e3f4b5a83b00674f426e438`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:51.446033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:51.548608 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4063/8092] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:51.561890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:51.563629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:51.634868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c4c01-52aa-4a0d-88b4-007c3f3b5a4f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:51.638714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4223d21-5c0e-4605-ba62-0a6016146dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:51.639454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223d21-5c0e-4605-ba62-0a6016146dac service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:51.639808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223d21-5c0e-4605-ba62-0a6016146dac service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:51.640123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223d21-5c0e-4605-ba62-0a6016146dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:51.649790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4223d21-5c0e-4605-ba62-0a6016146dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:51.651325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4223d21-5c0e-4605-ba62-0a6016146dac service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:51.668511 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4030/8093] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:58.609055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:58.611191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:58.683124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:58.728570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24291f9d-8c63-4898-9822-86c899b394c5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:58.735041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:58.736352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:58.736352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:58.736609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] PATH_INFO: `/v3/users/d46bf9c0b90f4589989b468cb4feec28` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:58.738265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46bf9c0b90f4589989b468cb4feec28)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:58.740794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:58.826825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46bf9c0b90f4589989b468cb4feec28'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:58.828055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46bf9c0b90f4589989b468cb4feec28'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:58.842064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46bf9c0b90f4589989b468cb4feec28'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:58.846565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46bf9c0b90f4589989b468cb4feec28'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:58.890201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46bf9c0b90f4589989b468cb4feec28'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:58.925836 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4064/8094] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:21:58 2022] DELETE /identity/v3/users/d46bf9c0b90f4589989b468cb4feec28 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:58.929190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:58.929792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:58.929923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:58.955575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:21:58.958493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:59.051368 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4031/8095] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:21:59.066146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:59.070094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:59.154278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f295f-3dd2-4162-a276-75d8b2a60979 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:59.160616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37d60a-0c14-497f-b8fb-8c494a19411b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:59.161527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37d60a-0c14-497f-b8fb-8c494a19411b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:59.161924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37d60a-0c14-497f-b8fb-8c494a19411b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:59.162264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37d60a-0c14-497f-b8fb-8c494a19411b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:59.174517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37d60a-0c14-497f-b8fb-8c494a19411b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:59.178353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37d60a-0c14-497f-b8fb-8c494a19411b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:59.190309 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4065/8096] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:21:59.813452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:59.815996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:59.831700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe214d-b996-4432-a810-33a991ef5ef8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:59.839941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:59.840899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:59.841274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:59.842484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] PATH_INFO: `/v3/projects/d37289e9d7394e5b86c0b7c10f97b365` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:59.845121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37289e9d7394e5b86c0b7c10f97b365)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:59.848317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:59.907674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:59.932189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37289e9d7394e5b86c0b7c10f97b365', 'request_id': 'req-c58adab9-c88f-44cc-beb1-66e9859da86a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:59.946115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37289e9d7394e5b86c0b7c10f97b365', 'request_id': 'req-c58adab9-c88f-44cc-beb1-66e9859da86a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:59.954253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37289e9d7394e5b86c0b7c10f97b365', 'request_id': 'req-c58adab9-c88f-44cc-beb1-66e9859da86a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:59.968894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] The token cache is being invalidate because project d37289e9d7394e5b86c0b7c10f97b365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:21:59.969558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:59.970449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c58adab9-c88f-44cc-beb1-66e9859da86a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:21:59.974225 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4032/8097] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:21:59 2022] DELETE /identity/v3/projects/d37289e9d7394e5b86c0b7c10f97b365 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:21:59.980624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3866c2-bb52-41a8-a484-84c10632e59d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:59.981227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3866c2-bb52-41a8-a484-84c10632e59d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:59.981556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3866c2-bb52-41a8-a484-84c10632e59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.005890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3b3866c2-bb52-41a8-a484-84c10632e59d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:00.007130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3b3866c2-bb52-41a8-a484-84c10632e59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:00.070533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:00.096953 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4066/8098] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:00.099083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e789d5-7b94-4f8d-b2f8-1e2d8157ba5a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.099607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e789d5-7b94-4f8d-b2f8-1e2d8157ba5a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.100100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e789d5-7b94-4f8d-b2f8-1e2d8157ba5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.132893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-39e789d5-7b94-4f8d-b2f8-1e2d8157ba5a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:00.134609 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-39e789d5-7b94-4f8d-b2f8-1e2d8157ba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:00.144642 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4033/8099] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:00.149969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3866c2-bb52-41a8-a484-84c10632e59d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:00.160788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3866c2-bb52-41a8-a484-84c10632e59d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:00.165929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:00.166833 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.167208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.167567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.168320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:22:00.168714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:00.170441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:00.185380 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4067/8100] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:22:00.190741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e789d5-7b94-4f8d-b2f8-1e2d8157ba5a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:00.202805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e789d5-7b94-4f8d-b2f8-1e2d8157ba5a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:00.210822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:00.212238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.212699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.213187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.214055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:00.216742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:00.291345 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4034/8101] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:22:00 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:00.295246 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:00.306088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5ec3c-0c9b-48ec-8807-3d787922ce21 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:00.409010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:00.409619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.409916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.410194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.410839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:00.413030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:00.441300 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4068/8102] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:00 2022] POST /identity/v3/users => generated 285 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:22:00.445993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:00.457797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad768c-1311-424c-b625-58358ea09095 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:00.465147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:00.466013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.466357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.466724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.467627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:00.469453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:00.485955 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4035/8103] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:22:00.486400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:00.503526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62d4fc-4848-4132-be32-10662e453bbb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:00.509111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbca75-24b2-498d-89f1-c5d01f1d7723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:00.509952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbca75-24b2-498d-89f1-c5d01f1d7723 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.510322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbca75-24b2-498d-89f1-c5d01f1d7723 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.510755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbca75-24b2-498d-89f1-c5d01f1d7723 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.511534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbca75-24b2-498d-89f1-c5d01f1d7723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:00.513262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbca75-24b2-498d-89f1-c5d01f1d7723 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:00.528533 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4069/8104] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:00.529858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:00.541577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff9ec9-f926-4520-ad08-4f56e4951080 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:00.548025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:00.549206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.549809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.550343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] PATH_INFO: `/v3/projects/0106885959194ffe9fdb6d6ced1c6e21/users/2bf729e890a94334901003c39d13850e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.568290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0106885959194ffe9fdb6d6ced1c6e21, user_id=2bf729e890a94334901003c39d13850e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:00.570779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:00.588754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4036/8105] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:00 2022] PUT /identity/v3/projects/0106885959194ffe9fdb6d6ced1c6e21/users/2bf729e890a94334901003c39d13850e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:00.609686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.609970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.610310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.644070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] MFA Rules not processed for user `2bf729e890a94334901003c39d13850e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:00.645328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:00.757590 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4070/8106] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:00.774274 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:00.776689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:00.869156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98f727-441e-4662-ad23-4c858ef021b5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:00.874527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9b757-36b2-4d4f-9dc5-eddbecb58608 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:00.875245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9b757-36b2-4d4f-9dc5-eddbecb58608 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:00.875727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9b757-36b2-4d4f-9dc5-eddbecb58608 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:00.876169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9b757-36b2-4d4f-9dc5-eddbecb58608 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:00.888619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9b757-36b2-4d4f-9dc5-eddbecb58608 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:00.890394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9b757-36b2-4d4f-9dc5-eddbecb58608 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:00.900844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4037/8107] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:02.125206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:02.126715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:02.249341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:02.266985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eca3b-49cc-4c7f-9438-b03f7d174ab3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:02.279025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:02.280070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:02.280581 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:02.281065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] PATH_INFO: `/v3/users/29f13ae6935146599f9ac42e2b752f43` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:02.282620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f13ae6935146599f9ac42e2b752f43)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:02.285422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:02.356430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f13ae6935146599f9ac42e2b752f43'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:02.361914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f13ae6935146599f9ac42e2b752f43'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:02.372708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f13ae6935146599f9ac42e2b752f43'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:02.373500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f13ae6935146599f9ac42e2b752f43'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:02.390300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f13ae6935146599f9ac42e2b752f43'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:02.408789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4071/8108] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:02 2022] DELETE /identity/v3/users/29f13ae6935146599f9ac42e2b752f43 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:02.414796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:02.416031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:02.416525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:02.445992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:02.448129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:02.588481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:02.614188 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4038/8109] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:02.621383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:02.623650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:02.815083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f2cc4-09a1-43f4-9c1c-110f6b767f2e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:02.820909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4d19f-5733-47b1-bee2-15ba8930c3ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:02.821942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4d19f-5733-47b1-bee2-15ba8930c3ee service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:02.822406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4d19f-5733-47b1-bee2-15ba8930c3ee service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:02.823028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4d19f-5733-47b1-bee2-15ba8930c3ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:02.851392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4d19f-5733-47b1-bee2-15ba8930c3ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:02.853058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4d19f-5733-47b1-bee2-15ba8930c3ee service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:02.877420 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4072/8110] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:03.331448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:03.334266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:03.352885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1712b-aac0-4b07-aa3f-df6b5620d0ad None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:03.359967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:03.360823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:03.361221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:03.361663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] PATH_INFO: `/v3/projects/7f5383a872224bc1baa4de128268596c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:03.363688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5383a872224bc1baa4de128268596c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:03.367402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:03.456613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5383a872224bc1baa4de128268596c', 'request_id': 'req-9c047a8c-12a8-42d5-b034-f72765e2f252'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:03.477755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5383a872224bc1baa4de128268596c', 'request_id': 'req-9c047a8c-12a8-42d5-b034-f72765e2f252'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:03.495858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5383a872224bc1baa4de128268596c', 'request_id': 'req-9c047a8c-12a8-42d5-b034-f72765e2f252'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:03.523275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] The token cache is being invalidate because project 7f5383a872224bc1baa4de128268596c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:03.524426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:03.525797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-9c047a8c-12a8-42d5-b034-f72765e2f252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:03.530079 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4039/8111] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:03 2022] DELETE /identity/v3/projects/7f5383a872224bc1baa4de128268596c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:03.535572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc84b73-1431-4b7f-a171-51cec63ba721 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:03.536919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc84b73-1431-4b7f-a171-51cec63ba721 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:03.537537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc84b73-1431-4b7f-a171-51cec63ba721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:03.573433 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-adc84b73-1431-4b7f-a171-51cec63ba721 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:03.575410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-adc84b73-1431-4b7f-a171-51cec63ba721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:03.681011 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4073/8112] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:03.695871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942c7c-c79d-4b7c-986c-e81d7c3aa55b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:03.695871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942c7c-c79d-4b7c-986c-e81d7c3aa55b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:03.695871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942c7c-c79d-4b7c-986c-e81d7c3aa55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:03.731182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-07942c7c-c79d-4b7c-986c-e81d7c3aa55b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:03.732525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-07942c7c-c79d-4b7c-986c-e81d7c3aa55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:03.746106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4040/8113] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:03.750189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc84b73-1431-4b7f-a171-51cec63ba721 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:03.761738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc84b73-1431-4b7f-a171-51cec63ba721 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:03.770034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:03.771343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:03.771932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:03.772079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:03.772856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:22:03.773323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:03.774851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:03.791483 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4074/8114] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:22:03.792540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07942c7c-c79d-4b7c-986c-e81d7c3aa55b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:03.803645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07942c7c-c79d-4b7c-986c-e81d7c3aa55b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:03.823813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:03.824512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:03.824790 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:03.825075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:03.825730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:03.827689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:03.880362 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4041/8115] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:22:03 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:03.884567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:03.891489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:03.903281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ce9b5-b517-4b41-bf35-5fc46d3d56be admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:03.990908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:03.992721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:03.993829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:03.994272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:03.995528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:04.001072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:04.045480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4075/8116] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:03 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:04.046898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:04.063967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d72d6-b336-402c-9d2b-ad01bf31665b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:04.077490 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:04.078305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:04.078652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:04.079016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:04.079794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:04.081687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:04.104749 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4042/8117] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:04.111015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:04.131383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f7202-4305-4736-854e-bdadcb496036 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:04.143980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0dfa6-fa1d-4c11-80fb-c59556ab7a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:04.144764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0dfa6-fa1d-4c11-80fb-c59556ab7a58 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:04.146051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0dfa6-fa1d-4c11-80fb-c59556ab7a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:04.146377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0dfa6-fa1d-4c11-80fb-c59556ab7a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:04.147552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0dfa6-fa1d-4c11-80fb-c59556ab7a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:04.151897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0dfa6-fa1d-4c11-80fb-c59556ab7a58 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:04.180735 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4076/8118] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:04.182780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:04.197339 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb2154-4ece-40e2-8a9e-3d359e0912c6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:04.210271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:04.210598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:04.210947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:04.211286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] PATH_INFO: `/v3/projects/dcf9f765ede5418794aab46838d0d65f/users/d2030ac4ab3547a3a6a550300573c89c/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:04.236542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf9f765ede5418794aab46838d0d65f, user_id=d2030ac4ab3547a3a6a550300573c89c, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:04.239437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:04.275926 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4043/8119] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:04 2022] PUT /identity/v3/projects/dcf9f765ede5418794aab46838d0d65f/users/d2030ac4ab3547a3a6a550300573c89c/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:22:04.287738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:04.288517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:04.288859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:04.389073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] MFA Rules not processed for user `d2030ac4ab3547a3a6a550300573c89c`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:04.390875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:04.631973 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4077/8120] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:04.661217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:04.663053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:04.898428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586a24d-0401-4d1d-8997-bd27c248e6a4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:04.905278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7919c0-ece7-4d62-93e8-5fd61425b8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:04.906912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7919c0-ece7-4d62-93e8-5fd61425b8fe service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:04.907208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7919c0-ece7-4d62-93e8-5fd61425b8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:04.907487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7919c0-ece7-4d62-93e8-5fd61425b8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:04.923191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7919c0-ece7-4d62-93e8-5fd61425b8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:04.924875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7919c0-ece7-4d62-93e8-5fd61425b8fe service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:04.942730 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4044/8121] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:22:06.786984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:06.790510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:06.969356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773d886-52b7-45fd-ba48-fd4b757b32e8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:06.980531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:06.981702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:06.981987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:06.982257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:06.984435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:06.996407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:07.086565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:07.088186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:07.141443 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4078/8122] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:13.458821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc12e5-3f71-4798-a59e-e1eb69b00920 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:13.460220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc12e5-3f71-4798-a59e-e1eb69b00920 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:13.461428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc12e5-3f71-4798-a59e-e1eb69b00920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:13.504243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-73dc12e5-3f71-4798-a59e-e1eb69b00920 None None] MFA Rules not processed for user `2fc6b11f473e4231adf4278688619f24`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:13.506101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-73dc12e5-3f71-4798-a59e-e1eb69b00920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:13.545943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:13.647945 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4045/8123] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:14.995891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:15.000885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:15.105451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d6db-32f7-4baf-8728-b2d64c5c718f service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:15.111681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbc124-9d8e-41d8-bd03-82a3e1efc622 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:15.112390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc124-9d8e-41d8-bd03-82a3e1efc622 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:15.112667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc124-9d8e-41d8-bd03-82a3e1efc622 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:15.112968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbc124-9d8e-41d8-bd03-82a3e1efc622 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:15.115028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-45dbc124-9d8e-41d8-bd03-82a3e1efc622 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:15.133424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbc124-9d8e-41d8-bd03-82a3e1efc622 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:15.135691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbc124-9d8e-41d8-bd03-82a3e1efc622 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:15.156589 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4079/8124] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:15.459880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc12e5-3f71-4798-a59e-e1eb69b00920 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:15.585931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc12e5-3f71-4798-a59e-e1eb69b00920 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:15.597368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:15.598240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:15.598775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:15.599156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] PATH_INFO: `/v3/users/d2030ac4ab3547a3a6a550300573c89c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:15.600724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2030ac4ab3547a3a6a550300573c89c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:15.602858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:15.750658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2030ac4ab3547a3a6a550300573c89c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:15.792717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2030ac4ab3547a3a6a550300573c89c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:15.806639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2030ac4ab3547a3a6a550300573c89c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:15.807617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2030ac4ab3547a3a6a550300573c89c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:15.808850 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2030ac4ab3547a3a6a550300573c89c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:15.838367 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4046/8125] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:15 2022] DELETE /identity/v3/users/d2030ac4ab3547a3a6a550300573c89c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:15.843992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:15.844539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:15.845121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:15.871696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:15.873257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:15.964785 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4080/8126] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:15.972990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:15.974543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:16.083839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:16.097312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e0db1-cc76-4540-93df-16e746a5a3f2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:16.102175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:16.102931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:16.103773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:16.104269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:16.115513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:16.117839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:16.133516 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4047/8127] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:16.419405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:16.422002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:16.450503 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda21f9-7e99-4a0a-95ab-c3a472c388b8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:16.464203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c23bd-69da-4cd1-87ae-aad9b4856744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:16.464837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c23bd-69da-4cd1-87ae-aad9b4856744 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:16.468917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c23bd-69da-4cd1-87ae-aad9b4856744 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:16.469292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c23bd-69da-4cd1-87ae-aad9b4856744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:16.471122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4b8c23bd-69da-4cd1-87ae-aad9b4856744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:16.502792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c23bd-69da-4cd1-87ae-aad9b4856744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:16.504904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c23bd-69da-4cd1-87ae-aad9b4856744 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:16.522477 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4081/8128] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:16.548822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:16.550828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:16.568499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440bd22-0e1c-4fdc-9b30-d5d1121a014e service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:16.574040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:16.575316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:16.575818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:16.577296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] PATH_INFO: `/v3/projects/dcf9f765ede5418794aab46838d0d65f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:16.578923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf9f765ede5418794aab46838d0d65f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:16.581754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:16.665607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf9f765ede5418794aab46838d0d65f', 'request_id': 'req-a4ca1654-2f64-442d-a9bd-54fb8884fa15'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:16.685333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf9f765ede5418794aab46838d0d65f', 'request_id': 'req-a4ca1654-2f64-442d-a9bd-54fb8884fa15'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:16.696206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf9f765ede5418794aab46838d0d65f', 'request_id': 'req-a4ca1654-2f64-442d-a9bd-54fb8884fa15'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:16.712170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] The token cache is being invalidate because project dcf9f765ede5418794aab46838d0d65f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:16.713878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:16.715912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a4ca1654-2f64-442d-a9bd-54fb8884fa15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:16.723817 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4048/8129] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:16 2022] DELETE /identity/v3/projects/dcf9f765ede5418794aab46838d0d65f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:16.728696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d25bf0-85e4-4ee3-9eb1-9077ec367120 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:16.729055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d25bf0-85e4-4ee3-9eb1-9077ec367120 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:16.729397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d25bf0-85e4-4ee3-9eb1-9077ec367120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:16.761661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-17d25bf0-85e4-4ee3-9eb1-9077ec367120 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:16.766199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-17d25bf0-85e4-4ee3-9eb1-9077ec367120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:16.793378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:16.876510 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4082/8130] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:16.884956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539aa5f2-f4ad-47fb-ac1b-a97254da4d13 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:16.885222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539aa5f2-f4ad-47fb-ac1b-a97254da4d13 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:16.886401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539aa5f2-f4ad-47fb-ac1b-a97254da4d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:16.912132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-539aa5f2-f4ad-47fb-ac1b-a97254da4d13 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:16.913374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-539aa5f2-f4ad-47fb-ac1b-a97254da4d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:16.928554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d25bf0-85e4-4ee3-9eb1-9077ec367120 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:16.935498 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4049/8131] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:16.941765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d25bf0-85e4-4ee3-9eb1-9077ec367120 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:16.947187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:16.948148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:16.948426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:16.948693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:16.949519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:22:16.949836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:16.953312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:16.981190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539aa5f2-f4ad-47fb-ac1b-a97254da4d13 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:16.985052 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4083/8132] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:22:16.993678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539aa5f2-f4ad-47fb-ac1b-a97254da4d13 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.004178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.005116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.005669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.005781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.006792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.008507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.056159 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4050/8133] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:22:16 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:17.056410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.071093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde455f-a9b3-4ccb-9bef-e4d7aa272852 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.170069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.170721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.171000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.171333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.171998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.174104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.210674 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4084/8134] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:17 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:17.215152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.227238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01a313-1981-4d0e-a410-6b3cec7f3e14 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.235861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.237202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.237874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.238747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.240222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.245375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.259101 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4051/8135] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:17.264030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.280231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee06ebd-f4c2-4ad0-bcb5-77337fbd8eef admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.289416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e4b4b-c3cd-49a2-b7ac-d04df1e53bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.290538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e4b4b-c3cd-49a2-b7ac-d04df1e53bbd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.290850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e4b4b-c3cd-49a2-b7ac-d04df1e53bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.291196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e4b4b-c3cd-49a2-b7ac-d04df1e53bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.291828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e4b4b-c3cd-49a2-b7ac-d04df1e53bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.293799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e4b4b-c3cd-49a2-b7ac-d04df1e53bbd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.309526 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4085/8136] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:17.313591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.326547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaace8e-f6eb-4aa8-851e-75abcec44379 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.331898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.333011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.333350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.333747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] PATH_INFO: `/v3/projects/6998087fa7414f11b57328f5b5d532bd/users/fe56b01feed140889a0f1a991b4c4666/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.350455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6998087fa7414f11b57328f5b5d532bd, user_id=fe56b01feed140889a0f1a991b4c4666, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.353705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.373782 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4052/8137] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:17 2022] PUT /identity/v3/projects/6998087fa7414f11b57328f5b5d532bd/users/fe56b01feed140889a0f1a991b4c4666/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:22:17.389121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.389835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.390351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.428348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] MFA Rules not processed for user `fe56b01feed140889a0f1a991b4c4666`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:17.430601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:17.516480 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4086/8138] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:17.553400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.555672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:17.663399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858da8a-fb33-44f9-9edb-0c90d1d9beb0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.672162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6bb19a-cf19-4983-b495-6f08f116b438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.672872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bb19a-cf19-4983-b495-6f08f116b438 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.673226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bb19a-cf19-4983-b495-6f08f116b438 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.673636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bb19a-cf19-4983-b495-6f08f116b438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.702089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6bb19a-cf19-4983-b495-6f08f116b438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.703823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6bb19a-cf19-4983-b495-6f08f116b438 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.717123 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4053/8139] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:17.900052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.901791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:17.930647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:18.017331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7987fd8-dee0-4af0-898f-a865b6517318 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:18.023738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:18.024653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:18.025224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:18.025514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] PATH_INFO: `/v3/users/2bf729e890a94334901003c39d13850e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:18.027118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf729e890a94334901003c39d13850e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:18.029108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:18.096418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf729e890a94334901003c39d13850e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:18.097454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf729e890a94334901003c39d13850e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:18.108902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf729e890a94334901003c39d13850e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:18.109780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf729e890a94334901003c39d13850e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:18.127017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf729e890a94334901003c39d13850e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:18.145511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4087/8140] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:17 2022] DELETE /identity/v3/users/2bf729e890a94334901003c39d13850e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:18.148684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:18.148997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:18.149305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:18.178867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:18.180284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:18.188273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:18.268110 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4054/8141] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:22:18.325116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:18.327323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:18.455288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecace5-2932-4b11-b4aa-5b7b69e0108c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:18.468510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ef15b-4a2f-4ca2-827b-9d25121deb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:18.470751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ef15b-4a2f-4ca2-827b-9d25121deb3c service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:18.471048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ef15b-4a2f-4ca2-827b-9d25121deb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:18.471321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ef15b-4a2f-4ca2-827b-9d25121deb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:18.489029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ef15b-4a2f-4ca2-827b-9d25121deb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:18.494219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ef15b-4a2f-4ca2-827b-9d25121deb3c service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:18.533632 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4088/8142] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:19.673945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:19.686386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:19.719660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863319c-026b-4afc-8ac1-b1b81590d41c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:19.728698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:19.729474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:19.729829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:19.730193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] PATH_INFO: `/v3/projects/0106885959194ffe9fdb6d6ced1c6e21` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:19.731936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0106885959194ffe9fdb6d6ced1c6e21)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:19.736378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:19.825389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0106885959194ffe9fdb6d6ced1c6e21', 'request_id': 'req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:19.840448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0106885959194ffe9fdb6d6ced1c6e21', 'request_id': 'req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:19.859299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0106885959194ffe9fdb6d6ced1c6e21', 'request_id': 'req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:19.890597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] The token cache is being invalidate because project 0106885959194ffe9fdb6d6ced1c6e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:19.891061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:19.892244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4a8398c8-249a-4ffb-98cc-b0a3ddeda843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:19.897596 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4055/8143] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:19 2022] DELETE /identity/v3/projects/0106885959194ffe9fdb6d6ced1c6e21 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:19.902976 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc266bb-a14d-4e1a-8393-2bec8a73a84d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:19.904125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc266bb-a14d-4e1a-8393-2bec8a73a84d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:19.904580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc266bb-a14d-4e1a-8393-2bec8a73a84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:19.935060 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dbc266bb-a14d-4e1a-8393-2bec8a73a84d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:19.936657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dbc266bb-a14d-4e1a-8393-2bec8a73a84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:19.977079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:20.041944 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4089/8144] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:20.049987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf0b6c-7e2b-4f7c-9519-51117ccc3347 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.050555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf0b6c-7e2b-4f7c-9519-51117ccc3347 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.052165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf0b6c-7e2b-4f7c-9519-51117ccc3347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.077737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0daf0b6c-7e2b-4f7c-9519-51117ccc3347 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:20.079161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0daf0b6c-7e2b-4f7c-9519-51117ccc3347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:20.090308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4056/8145] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:20.094470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc266bb-a14d-4e1a-8393-2bec8a73a84d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:20.107601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc266bb-a14d-4e1a-8393-2bec8a73a84d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:20.122229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:20.122229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.122229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.122229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.122636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:22:20.122636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:20.122636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:20.137499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4090/8146] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:22:20.139487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf0b6c-7e2b-4f7c-9519-51117ccc3347 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:20.149304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf0b6c-7e2b-4f7c-9519-51117ccc3347 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:20.155049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:20.155898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.156246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.156552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.157204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:20.159267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:20.213220 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4057/8147] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:22:20 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:20.217748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:20.233853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd5e32-c00c-40c8-8026-7a6d164c68b0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:20.325551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:20.326769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.327131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.328693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.328693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:20.332021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:20.368054 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4091/8148] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:20 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:20.372555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:20.397978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e5127-22ba-4e5c-8244-2a4bd4aa44fc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:20.404927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:20.405502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.405636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.405972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.406618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:20.408611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:20.420698 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4058/8149] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:20.425174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:20.443579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab4681-1969-4ac8-aac6-3587b83da216 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:20.473231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3864d-ff02-4c32-ab8e-47b4699b226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:20.474190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3864d-ff02-4c32-ab8e-47b4699b226d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.474475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3864d-ff02-4c32-ab8e-47b4699b226d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.474860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3864d-ff02-4c32-ab8e-47b4699b226d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.475514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3864d-ff02-4c32-ab8e-47b4699b226d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:20.477126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3864d-ff02-4c32-ab8e-47b4699b226d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:20.511426 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4092/8150] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:20.525152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:20.561402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276c83c-e520-445d-b2f0-003bbcaf4658 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:20.572920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:20.573650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.573934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.574210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] PATH_INFO: `/v3/projects/364569c3608b459ebc6ca73cfed86303/users/0302d735014e41e9aa98697c3de5f850/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.626689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] RBAC: Authorizing `identity:create_grant(project_id=364569c3608b459ebc6ca73cfed86303, user_id=0302d735014e41e9aa98697c3de5f850, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:20.633076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:20.670751 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4059/8151] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:20 2022] PUT /identity/v3/projects/364569c3608b459ebc6ca73cfed86303/users/0302d735014e41e9aa98697c3de5f850/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:20.675718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.675999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.676270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.710045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:20.710045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:20.796834 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4093/8152] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:20.879988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:20.881928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:20.929129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:20.979355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b9c18-9ba8-4048-8d09-6fd03986558c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:20.984201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa12ae9-4869-4db7-b7eb-be656beb117d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:20.985298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa12ae9-4869-4db7-b7eb-be656beb117d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:20.985298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa12ae9-4869-4db7-b7eb-be656beb117d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:20.985622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa12ae9-4869-4db7-b7eb-be656beb117d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:20.997510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa12ae9-4869-4db7-b7eb-be656beb117d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:20.999578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa12ae9-4869-4db7-b7eb-be656beb117d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:21.025909 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4060/8153] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:27.547922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:27.550034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:27.656886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:27.755809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f34f4-97ee-4eed-a327-69806e23ffbc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:27.761661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:27.762356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:27.762703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:27.763445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:27.767445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:27.788691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:27.791560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:27.809216 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4094/8154] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:22:30.709110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3e2d6-9f91-46fd-940a-1a8911c8d1da None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:30.710172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3e2d6-9f91-46fd-940a-1a8911c8d1da None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:30.710673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3e2d6-9f91-46fd-940a-1a8911c8d1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:30.774549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bbb3e2d6-9f91-46fd-940a-1a8911c8d1da None None] MFA Rules not processed for user `0302d735014e41e9aa98697c3de5f850`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:30.776011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bbb3e2d6-9f91-46fd-940a-1a8911c8d1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:30.916031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:30.921317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4061/8155] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:30.933161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beef11-4112-4b4b-af58-b1a86cdbe0a4 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:30.946078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:30.946714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:30.946993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:30.947266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:30.948315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:30.950364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:31.027414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3e2d6-9f91-46fd-940a-1a8911c8d1da None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:31.036953 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4095/8156] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:22:30 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:31.045103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3e2d6-9f91-46fd-940a-1a8911c8d1da None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:31.127232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:31.128029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:31.128359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:31.128682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:31.129433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:31.132222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:31.177355 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4062/8157] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:31 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:31.177718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:31.193932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33162cc-0a2a-4a07-8e97-07c6349117bd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:31.201292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:31.202611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:31.203913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:31.204407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:31.205297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:31.207186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:31.225878 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4096/8158] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:31.231646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:31.247109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aefbb-7ec9-479d-97ba-c0dceca2c432 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:31.259431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:31.260228 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:31.260620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:31.260990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] PATH_INFO: `/v3/projects/718efb2481024b278d50730fec729eeb/users/70b0d8337e20432584f80476b46453f0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:31.285347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718efb2481024b278d50730fec729eeb, user_id=70b0d8337e20432584f80476b46453f0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:31.291273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:31.315957 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4063/8159] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:31 2022] PUT /identity/v3/projects/718efb2481024b278d50730fec729eeb/users/70b0d8337e20432584f80476b46453f0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:22:31.320321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:31.336670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3cd3d-c508-4f33-8709-84e4fe3164f2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:31.474935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:31.475679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:31.476201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:31.476591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:31.477438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:31.483336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:31.497490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4097/8160] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:31.501923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:31.514719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a1483-3be8-4661-8fd5-9be4e668f074 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:31.529109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:31.530092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:31.530462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:31.530863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] PATH_INFO: `/v3/projects/718efb2481024b278d50730fec729eeb/users/70b0d8337e20432584f80476b46453f0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:31.533529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718efb2481024b278d50730fec729eeb, user_id=70b0d8337e20432584f80476b46453f0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:31.536422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:31.552198 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4064/8161] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:31 2022] PUT /identity/v3/projects/718efb2481024b278d50730fec729eeb/users/70b0d8337e20432584f80476b46453f0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:35.874079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:35.877148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:35.895739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85f56b-823a-4c10-8933-6b7301dfa1c6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:36.009316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:36.010370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:36.010507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:36.010884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] PATH_INFO: `/v3/users/df6cec0e04954a3eba096b48a7cd2189` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:36.012248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df6cec0e04954a3eba096b48a7cd2189)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:36.014347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:36.094749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:36.117127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df6cec0e04954a3eba096b48a7cd2189'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:36.117762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df6cec0e04954a3eba096b48a7cd2189'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:36.129777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df6cec0e04954a3eba096b48a7cd2189'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:36.130463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df6cec0e04954a3eba096b48a7cd2189'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:36.151313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df6cec0e04954a3eba096b48a7cd2189'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:36.168340 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4098/8162] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:35 2022] DELETE /identity/v3/users/df6cec0e04954a3eba096b48a7cd2189 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:37.647402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:37.656618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:37.708709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492128c-720a-43b6-b053-4d22717eea94 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:37.747532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:37.861358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:37.862120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:37.862407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:37.862685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] PATH_INFO: `/v3/projects/09f16a815901466791a114b9a19cc763` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:37.864518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f16a815901466791a114b9a19cc763)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:37.867079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:37.965077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f16a815901466791a114b9a19cc763', 'request_id': 'req-c1df0bb3-2d51-40c7-8c12-2327011091e4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:37.979240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f16a815901466791a114b9a19cc763', 'request_id': 'req-c1df0bb3-2d51-40c7-8c12-2327011091e4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.028149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f16a815901466791a114b9a19cc763', 'request_id': 'req-c1df0bb3-2d51-40c7-8c12-2327011091e4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.074883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] The token cache is being invalidate because project 09f16a815901466791a114b9a19cc763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:38.075774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.077594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.080586 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4065/8163] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:37 2022] DELETE /identity/v3/projects/09f16a815901466791a114b9a19cc763 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:38.084883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:38.108966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02be02-7bd8-4231-8c0e-4b7953d88225 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:38.224312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:38.226377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:38.226949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:38.227409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] PATH_INFO: `/v3/users/e5cff400a3ce426abe10ebc16d8c1bca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:38.231563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5cff400a3ce426abe10ebc16d8c1bca)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:38.249753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:38.369606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5cff400a3ce426abe10ebc16d8c1bca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.370413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5cff400a3ce426abe10ebc16d8c1bca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.384557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5cff400a3ce426abe10ebc16d8c1bca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.385560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5cff400a3ce426abe10ebc16d8c1bca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.429625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-87da8d7e-e3a2-42d6-8cce-a5820ca04aab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5cff400a3ce426abe10ebc16d8c1bca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:38.450887 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4099/8164] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:38 2022] DELETE /identity/v3/users/e5cff400a3ce426abe10ebc16d8c1bca => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:39.198764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:39.201189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:39.222290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df0bb3-2d51-40c7-8c12-2327011091e4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:39.362795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:39.364184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:39.364471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:39.364751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] PATH_INFO: `/v3/projects/350d95806ad747439a4745ee873235d8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:39.372341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=350d95806ad747439a4745ee873235d8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:39.375202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:39.485661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:39.488268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350d95806ad747439a4745ee873235d8', 'request_id': 'req-e612d533-f5de-4ec5-94d0-667878c09659'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:39.509970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350d95806ad747439a4745ee873235d8', 'request_id': 'req-e612d533-f5de-4ec5-94d0-667878c09659'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:39.516777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350d95806ad747439a4745ee873235d8', 'request_id': 'req-e612d533-f5de-4ec5-94d0-667878c09659'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:39.540561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] The token cache is being invalidate because project 350d95806ad747439a4745ee873235d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:39.540858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:39.541749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e612d533-f5de-4ec5-94d0-667878c09659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:39.550812 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4066/8165] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:39 2022] DELETE /identity/v3/projects/350d95806ad747439a4745ee873235d8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:39.552460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00234084-f945-4bef-96e9-38739744f6bb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:39.553336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00234084-f945-4bef-96e9-38739744f6bb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:39.554002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00234084-f945-4bef-96e9-38739744f6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:39.608291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-00234084-f945-4bef-96e9-38739744f6bb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:39.610075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-00234084-f945-4bef-96e9-38739744f6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:39.662420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:39.756531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4100/8166] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:22:39.765733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872dbd-4039-49a1-92bc-cca6750e54b2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:39.766883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872dbd-4039-49a1-92bc-cca6750e54b2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:39.767448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872dbd-4039-49a1-92bc-cca6750e54b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:39.825668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-30872dbd-4039-49a1-92bc-cca6750e54b2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:39.827015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-30872dbd-4039-49a1-92bc-cca6750e54b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:39.840628 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4067/8167] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:39.845533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00234084-f945-4bef-96e9-38739744f6bb None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:39.863313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00234084-f945-4bef-96e9-38739744f6bb None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:39.872332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:39.874211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:39.874494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:39.874770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:39.877023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:22:39.877351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:39.879036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:39.894233 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4101/8168] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:22:39.899399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30872dbd-4039-49a1-92bc-cca6750e54b2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:39.912164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30872dbd-4039-49a1-92bc-cca6750e54b2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:39.922862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:39.923774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:39.924220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:39.924684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:39.925546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:39.927862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:39.983743 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4068/8169] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:22:39 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:39.988263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:40.042125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b70d3d-566e-4f41-b459-f9fc33433329 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:40.165847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:40.166481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.166759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.167032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:40.167706 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:40.169870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:40.209524 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4102/8170] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:39 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:40.210130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:40.234662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10171819-3b10-4f0d-8b6e-b2471771471b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:40.243358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:40.244938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.245494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.246009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:40.246971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:40.249079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:40.289823 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4069/8171] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:40.295048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:40.309219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30194a-a441-4c1f-93d0-2dc453638649 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:40.314872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cf621-289d-4d03-ada5-85d16564b0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:40.316185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cf621-289d-4d03-ada5-85d16564b0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.316185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cf621-289d-4d03-ada5-85d16564b0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.316185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cf621-289d-4d03-ada5-85d16564b0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:40.316505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cf621-289d-4d03-ada5-85d16564b0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:40.320273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cf621-289d-4d03-ada5-85d16564b0c3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:40.343668 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4103/8172] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:40.348515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:40.362680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3ad9a-9573-4513-8b88-c28f10300d3b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:40.369891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:40.370916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.371402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.371902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] PATH_INFO: `/v3/projects/86f949258d744bb18c4b140621d617f0/users/3f976f55624549d0890c884bc47f5848/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:40.405524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f949258d744bb18c4b140621d617f0, user_id=3f976f55624549d0890c884bc47f5848, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:40.408593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:40.439447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4070/8173] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:40 2022] PUT /identity/v3/projects/86f949258d744bb18c4b140621d617f0/users/3f976f55624549d0890c884bc47f5848/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:40.444652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.445053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.445284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:40.474509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:40.475781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:40.585631 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4104/8174] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:40.601726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:40.603739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:40.724099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0e137-5b35-4c3a-ad3c-3245856d4917 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:40.731202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724880f6-67ed-4d09-b949-74d1c1343716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:40.734375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724880f6-67ed-4d09-b949-74d1c1343716 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.734732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724880f6-67ed-4d09-b949-74d1c1343716 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.735711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724880f6-67ed-4d09-b949-74d1c1343716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:40.749081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724880f6-67ed-4d09-b949-74d1c1343716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:40.750639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724880f6-67ed-4d09-b949-74d1c1343716 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:40.771083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:40.773087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:40.774202 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4071/8175] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:40.787527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d6e6f-2558-48ca-a051-b5082176c636 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:40.800350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:40.802651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.802934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.803205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] PATH_INFO: `/v3/users/fe56b01feed140889a0f1a991b4c4666` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:40.810285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe56b01feed140889a0f1a991b4c4666)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:40.813220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:40.870033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:40.902965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe56b01feed140889a0f1a991b4c4666'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:40.903851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe56b01feed140889a0f1a991b4c4666'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:40.918453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe56b01feed140889a0f1a991b4c4666'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:40.919364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe56b01feed140889a0f1a991b4c4666'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:40.941367 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe56b01feed140889a0f1a991b4c4666'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:40.983146 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4105/8176] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:40 2022] DELETE /identity/v3/users/fe56b01feed140889a0f1a991b4c4666 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:40.985403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc8813-262e-4307-9710-421f58298ad4 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:40.985815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc8813-262e-4307-9710-421f58298ad4 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:40.986205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc8813-262e-4307-9710-421f58298ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:41.033432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c7dc8813-262e-4307-9710-421f58298ad4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:41.035387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c7dc8813-262e-4307-9710-421f58298ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:41.156276 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4072/8177] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:41.167253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:41.168677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:41.262448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c341c5-312b-4f69-bf5a-4cea51735ecc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:41.266179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:41.266824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:41.267082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:41.267348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:41.287437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:41.289545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:41.303621 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4106/8178] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:41.763898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315039-658e-475c-b3d8-b5ff03e04e17 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:41.764496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315039-658e-475c-b3d8-b5ff03e04e17 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:41.764496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315039-658e-475c-b3d8-b5ff03e04e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:41.815438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f3315039-658e-475c-b3d8-b5ff03e04e17 None None] MFA Rules not processed for user `70b0d8337e20432584f80476b46453f0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:41.819933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f3315039-658e-475c-b3d8-b5ff03e04e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:41.831388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:41.990145 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4073/8179] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:41.993629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:41.996199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:42.103570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a5771-e554-4fb7-990c-9553a99ebae6 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:42.112943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:42.113634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:42.114001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:42.114279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:42.126174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:42.128401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:42.142476 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4107/8180] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:42.226449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af031c-3dc3-4e25-a710-ea082702da69 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:42.227018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af031c-3dc3-4e25-a710-ea082702da69 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:42.227462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af031c-3dc3-4e25-a710-ea082702da69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:42.231174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:42.232633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:42.263375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b4af031c-3dc3-4e25-a710-ea082702da69 None None] MFA Rules not processed for user `0302d735014e41e9aa98697c3de5f850`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:42.264888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd6801-b3ee-424d-9d14-68cf42f4cf72 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:42.265944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b4af031c-3dc3-4e25-a710-ea082702da69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:42.276450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:42.277111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:42.277415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:42.277695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] PATH_INFO: `/v3/projects/6998087fa7414f11b57328f5b5d532bd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:42.279872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6998087fa7414f11b57328f5b5d532bd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:42.284786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:42.356391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:42.370524 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4074/8181] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:42.413238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6998087fa7414f11b57328f5b5d532bd', 'request_id': 'req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:42.425792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6998087fa7414f11b57328f5b5d532bd', 'request_id': 'req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:42.440366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6998087fa7414f11b57328f5b5d532bd', 'request_id': 'req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:42.465756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] The token cache is being invalidate because project 6998087fa7414f11b57328f5b5d532bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:42.466211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:42.467816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a411dfc8-02c0-43d1-b6a4-3e963856b6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:42.470353 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4108/8182] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:42 2022] DELETE /identity/v3/projects/6998087fa7414f11b57328f5b5d532bd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:42.475226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba99fa-e33a-476b-87f7-3ba50291ba01 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:42.475988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba99fa-e33a-476b-87f7-3ba50291ba01 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:42.476427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba99fa-e33a-476b-87f7-3ba50291ba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:42.526555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-74ba99fa-e33a-476b-87f7-3ba50291ba01 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:42.528781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-74ba99fa-e33a-476b-87f7-3ba50291ba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:42.703794 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4075/8183] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:42.704788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9506c68-3a2d-498c-bf81-364f02df7d95 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:42.705236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9506c68-3a2d-498c-bf81-364f02df7d95 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:42.705729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9506c68-3a2d-498c-bf81-364f02df7d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:42.739953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c9506c68-3a2d-498c-bf81-364f02df7d95 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:42.741542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c9506c68-3a2d-498c-bf81-364f02df7d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:42.752641 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4109/8184] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:42.757608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba99fa-e33a-476b-87f7-3ba50291ba01 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:42.775697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba99fa-e33a-476b-87f7-3ba50291ba01 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:42.786335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:42.786981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:42.787259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:42.787536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:42.788989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:22:42.789484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:42.791173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:42.815789 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4076/8185] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:22:42.816361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9506c68-3a2d-498c-bf81-364f02df7d95 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:42.829842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9506c68-3a2d-498c-bf81-364f02df7d95 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:42.835445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:42.836062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:42.836340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:42.836601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:42.837235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:42.839316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:42.893614 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4110/8186] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:22:42 2022] POST /identity/v3/projects => generated 369 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:22:42.897837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:42.912479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cf09e-15b6-4790-8890-6c96d39a10c7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:42.954818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:43.039017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:43.040024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:43.042093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:43.042590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:43.043460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:43.049925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:43.103484 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4077/8187] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:42 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:43.106368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:43.121711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0671ab-eb99-4235-94b7-06d2c7da2fd9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:43.129694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de135923-e864-4945-aceb-139252fce4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:43.131070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135923-e864-4945-aceb-139252fce4be admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:43.131550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135923-e864-4945-aceb-139252fce4be admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:43.132170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de135923-e864-4945-aceb-139252fce4be admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:43.140911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de135923-e864-4945-aceb-139252fce4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:43.143552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de135923-e864-4945-aceb-139252fce4be admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:43.167128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:43.187983 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4111/8188] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:43.188594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cf3bf-b663-4b5a-817e-ea9ba2057877 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:43.195725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86571c-3798-4240-8593-28b5ec86cd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:43.196363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86571c-3798-4240-8593-28b5ec86cd27 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:43.196641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86571c-3798-4240-8593-28b5ec86cd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:43.196913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86571c-3798-4240-8593-28b5ec86cd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:43.197826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86571c-3798-4240-8593-28b5ec86cd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:43.199791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86571c-3798-4240-8593-28b5ec86cd27 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:43.218659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de135923-e864-4945-aceb-139252fce4be admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:43.225947 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4078/8189] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:43.230731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de135923-e864-4945-aceb-139252fce4be admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:43.235681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:43.236352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:43.236679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:43.237248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] PATH_INFO: `/v3/projects/2d7b965a444e4b079611adba36c73066/users/ce3d2418f5d24b0dbf1142079eade5dc/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:43.258346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d7b965a444e4b079611adba36c73066, user_id=ce3d2418f5d24b0dbf1142079eade5dc, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:43.261183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:43.286413 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4112/8190] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:22:43 2022] PUT /identity/v3/projects/2d7b965a444e4b079611adba36c73066/users/ce3d2418f5d24b0dbf1142079eade5dc/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:43.292586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22019894-6b25-49f8-a673-9fab39c1d94a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:43.292884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22019894-6b25-49f8-a673-9fab39c1d94a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:43.293777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22019894-6b25-49f8-a673-9fab39c1d94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:43.328826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-22019894-6b25-49f8-a673-9fab39c1d94a None None] MFA Rules not processed for user `ce3d2418f5d24b0dbf1142079eade5dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:43.331486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-22019894-6b25-49f8-a673-9fab39c1d94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:43.425340 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4079/8191] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:43.443301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:43.446862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:43.555861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21585b6a-2537-41da-a0f8-ad138f70b2d3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:43.561940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:43.562936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:43.564058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:43.564685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:43.575959 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:43.577888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:43.589965 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4113/8192] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:22:49.134378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7bc7a-f6c1-41de-a193-3fe530687372 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:49.135178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7bc7a-f6c1-41de-a193-3fe530687372 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:49.135595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7bc7a-f6c1-41de-a193-3fe530687372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:49.239329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c6b7bc7a-f6c1-41de-a193-3fe530687372 None None] MFA Rules not processed for user `3f976f55624549d0890c884bc47f5848`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:49.240716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c6b7bc7a-f6c1-41de-a193-3fe530687372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:49.410976 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4080/8193] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:22:49.416024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:49.417590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:49.508101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:49.528990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34599d2c-71ab-4a9b-9b9a-57d404b62511 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:49.543632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:49.544364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:49.544740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:49.545112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:49.545935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:49.556929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:49.591543 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4114/8194] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:22:49 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:22:49.596247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7bc7a-f6c1-41de-a193-3fe530687372 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:49.614362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7bc7a-f6c1-41de-a193-3fe530687372 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:49.636239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45960e5-96ab-4664-8683-c1a7099a84ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:49.636970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45960e5-96ab-4664-8683-c1a7099a84ae admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:49.637311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45960e5-96ab-4664-8683-c1a7099a84ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:49.638210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45960e5-96ab-4664-8683-c1a7099a84ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:49.639018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45960e5-96ab-4664-8683-c1a7099a84ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:49.640788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45960e5-96ab-4664-8683-c1a7099a84ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:49.661053 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4081/8195] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:49.664966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:49.682455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc8be9-4ead-42ca-a933-c32e34b99e25 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:49.692490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:49.693539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:49.693933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:49.694305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] PATH_INFO: `/v3/system/users/2902738a70834db7a7bbc5b372113281/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:49.722306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2902738a70834db7a7bbc5b372113281, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:49.727957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:49.746125 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4115/8196] 10.222.0.26 () {60 vars in 1378 bytes} [Sat Sep 17 21:22:49 2022] PUT /identity/v3/system/users/2902738a70834db7a7bbc5b372113281/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:22:49.755104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06fcc-c89d-4d8c-aaad-b48fcfe5fd74 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:49.756084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06fcc-c89d-4d8c-aaad-b48fcfe5fd74 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:49.756620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e06fcc-c89d-4d8c-aaad-b48fcfe5fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:49.818121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b8e06fcc-c89d-4d8c-aaad-b48fcfe5fd74 None None] MFA Rules not processed for user `2902738a70834db7a7bbc5b372113281`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:22:49.819579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b8e06fcc-c89d-4d8c-aaad-b48fcfe5fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:49.830461 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:49.936371 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4082/8197] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:22:49 2022] POST /identity/v3/auth/tokens => generated 2415 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 21:22:49.940429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:50.199698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a39bc8-2813-4c3a-a596-de5e7cff02a8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:50.247803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2902738a70834db7a7bbc5b372113281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:50.248611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:50.248980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:50.249361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] PATH_INFO: `/v3/services` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:50.250396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:50.252109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:50.263330 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4116/8198] 10.222.0.26 () {58 vars in 1110 bytes} [Sat Sep 17 21:22:49 2022] GET /identity/v3/services => generated 2191 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:22:50.267814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e06fcc-c89d-4d8c-aaad-b48fcfe5fd74 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:50.418481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e06fcc-c89d-4d8c-aaad-b48fcfe5fd74 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:50.481004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2902738a70834db7a7bbc5b372113281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:50.484234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:50.484441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:50.484863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] PATH_INFO: `/v3/limits` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:50.486513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:50.488579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:50.500143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Sep 17 21:22:50.538003 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application raise exception.NoLimitReference Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 17 21:22:50.539233 ncdv-master-1 devstack@keystone.service[99166]: ERROR keystone.server.flask.application Sep 17 21:22:50.540288 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4083/8199] 10.222.0.26 () {60 vars in 1126 bytes} [Sat Sep 17 21:22:50 2022] POST /identity/v3/limits => generated 124 bytes in 272 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 17 21:22:50.546064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:50.588211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:50.706585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d98b1c-9e72-494f-a510-44d91ddea898 None tempest-ImageQuotaTest-2020477875-system-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:50.789069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2902738a70834db7a7bbc5b372113281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:50.789850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:50.790829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:50.791205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:50.792092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:50.794462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:50.817823 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4117/8200] 10.222.0.26 () {58 vars in 1137 bytes} [Sat Sep 17 21:22:50 2022] GET /identity/v3/registered_limits => generated 129 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:22:50.825382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:51.105226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7704242-540e-466e-b75e-082112172443 None tempest-ImageQuotaTest-2020477875-system-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:51.194920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2902738a70834db7a7bbc5b372113281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:51.195571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:51.195854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:51.196767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:51.197406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:51.198559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:51.206722 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4084/8201] 10.222.0.26 () {58 vars in 1137 bytes} [Sat Sep 17 21:22:50 2022] GET /identity/v3/registered_limits => generated 129 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:23:00.807087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=52 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:00.881627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:00.897331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6747d72-5a87-4178-bca5-507882b6b842 None tempest-ImageQuotaTest-2020477875-system-admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:00.903878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:00.904531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:00.905001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:00.905450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] PATH_INFO: `/v3/users/3f976f55624549d0890c884bc47f5848` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:00.907862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f976f55624549d0890c884bc47f5848)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:00.912474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:00.987627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f976f55624549d0890c884bc47f5848'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:00.988341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f976f55624549d0890c884bc47f5848'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:01.001002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f976f55624549d0890c884bc47f5848'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:01.001757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f976f55624549d0890c884bc47f5848'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:01.044018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f976f55624549d0890c884bc47f5848'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:01.068701 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4118/8202] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:00 2022] DELETE /identity/v3/users/3f976f55624549d0890c884bc47f5848 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:23:01.203600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=71 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:01.802562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:01.804664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:01.937233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7c103-e3f5-4c3b-b9b8-eaf2bb111f62 None tempest-ImageQuotaTest-2020477875-system-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:01.943664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:01.944386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:01.944670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:01.944953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] PATH_INFO: `/v3/projects/86f949258d744bb18c4b140621d617f0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:01.946204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f949258d744bb18c4b140621d617f0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:01.948164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:02.052511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f949258d744bb18c4b140621d617f0', 'request_id': 'req-13893153-77b3-4046-aed8-c0cea1bb1eb6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.067227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f949258d744bb18c4b140621d617f0', 'request_id': 'req-13893153-77b3-4046-aed8-c0cea1bb1eb6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.078325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f949258d744bb18c4b140621d617f0', 'request_id': 'req-13893153-77b3-4046-aed8-c0cea1bb1eb6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.107569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] The token cache is being invalidate because project 86f949258d744bb18c4b140621d617f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:02.108088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.115746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-13893153-77b3-4046-aed8-c0cea1bb1eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.120873 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4085/8203] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:01 2022] DELETE /identity/v3/projects/86f949258d744bb18c4b140621d617f0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:02.122172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:02.137787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7789c1b-6f4a-49ab-bee5-acfd679bc39d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:02.306010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:02.307236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:02.308027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:02.308805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] PATH_INFO: `/v3/users/2902738a70834db7a7bbc5b372113281` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:02.310432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2902738a70834db7a7bbc5b372113281)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:02.318846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:02.465390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2902738a70834db7a7bbc5b372113281'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.471421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2902738a70834db7a7bbc5b372113281'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.494936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2902738a70834db7a7bbc5b372113281'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.496029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2902738a70834db7a7bbc5b372113281'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.511993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:02.528662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2902738a70834db7a7bbc5b372113281'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:02.572047 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4119/8204] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:02 2022] DELETE /identity/v3/users/2902738a70834db7a7bbc5b372113281 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:04.519491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:04.520107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:04.520380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:04.543426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] MFA Rules not processed for user `0302d735014e41e9aa98697c3de5f850`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:04.544964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:04.670595 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4086/8205] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:05.271830 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:05.273631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:05.420473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340d692-c11c-4045-939e-c56f0180b75e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:05.427350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4441d-2922-4e50-9d1d-3edf577cad12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:05.428649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4441d-2922-4e50-9d1d-3edf577cad12 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:05.428757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4441d-2922-4e50-9d1d-3edf577cad12 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:05.429146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4441d-2922-4e50-9d1d-3edf577cad12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:05.431377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0fa4441d-2922-4e50-9d1d-3edf577cad12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:05.447784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4441d-2922-4e50-9d1d-3edf577cad12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:05.449492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4441d-2922-4e50-9d1d-3edf577cad12 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:05.463869 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4120/8206] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:14.653846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:15.454913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=34,keystone:DELETE=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:17.385710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:17.387746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:17.541558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042f06a-9d36-476a-9b79-eea54edb7921 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:17.546709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:17.547771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:17.548255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:17.549173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] PATH_INFO: `/v3/users/2fc6b11f473e4231adf4278688619f24` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:17.551202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc6b11f473e4231adf4278688619f24)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:17.553788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:17.704808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc6b11f473e4231adf4278688619f24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:17.725890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc6b11f473e4231adf4278688619f24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:17.742817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc6b11f473e4231adf4278688619f24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:17.748462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc6b11f473e4231adf4278688619f24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:17.749344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc6b11f473e4231adf4278688619f24'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:17.791024 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4087/8207] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:17 2022] DELETE /identity/v3/users/2fc6b11f473e4231adf4278688619f24 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:17.793166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:17.794165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:17.794643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:17.828739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:17.835143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:17.949096 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4121/8208] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:17.967977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:17.971401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:18.060449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686ed42-e2fb-4300-ad56-11e7505bfa3d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:18.064944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:18.065832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:18.066323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:18.066907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:18.078622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:18.081218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:18.094751 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4088/8209] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:18.439649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:18.441524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:18.484717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b1aef-6a3b-4305-94f5-1c445afe2973 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:18.491650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:18.492608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:18.493113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:18.493843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] PATH_INFO: `/v3/projects/0608df66a18a4fb3855b6cd557ab2998` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:18.499372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0608df66a18a4fb3855b6cd557ab2998)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:18.509693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:18.624042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0608df66a18a4fb3855b6cd557ab2998', 'request_id': 'req-786ece8e-319f-4718-bba0-98158c87dc09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.630960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0608df66a18a4fb3855b6cd557ab2998', 'request_id': 'req-786ece8e-319f-4718-bba0-98158c87dc09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.649398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0608df66a18a4fb3855b6cd557ab2998', 'request_id': 'req-786ece8e-319f-4718-bba0-98158c87dc09'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.664942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] The token cache is being invalidate because project 0608df66a18a4fb3855b6cd557ab2998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:18.665318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.666298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.673575 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4122/8210] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:18 2022] DELETE /identity/v3/projects/0608df66a18a4fb3855b6cd557ab2998 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:18.673795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:18.691057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800c511-5abf-4cf4-8655-553d46067a66 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:18.719548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:18.781418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:18.782391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:18.782721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:18.783047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] PATH_INFO: `/v3/users/187f106f895d4af390c9cc8d7353f7af` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:18.784346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=187f106f895d4af390c9cc8d7353f7af)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:18.786758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:18.860439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187f106f895d4af390c9cc8d7353f7af'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.892412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187f106f895d4af390c9cc8d7353f7af'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.911567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187f106f895d4af390c9cc8d7353f7af'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.912658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187f106f895d4af390c9cc8d7353f7af'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.915680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187f106f895d4af390c9cc8d7353f7af'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:18.943490 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4089/8211] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:18 2022] DELETE /identity/v3/users/187f106f895d4af390c9cc8d7353f7af => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:20.391394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:20.393115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:20.440871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ece8e-319f-4718-bba0-98158c87dc09 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:20.527134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:20.527411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:20.527523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:20.528452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] PATH_INFO: `/v3/projects/76c02d00830244688d278acc8fd2cecc` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:20.529530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c02d00830244688d278acc8fd2cecc)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:20.531638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:20.576995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:20.602658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c02d00830244688d278acc8fd2cecc', 'request_id': 'req-549897e8-ef4d-472d-acfa-9a7d29659b8b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:20.616038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c02d00830244688d278acc8fd2cecc', 'request_id': 'req-549897e8-ef4d-472d-acfa-9a7d29659b8b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:20.626680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c02d00830244688d278acc8fd2cecc', 'request_id': 'req-549897e8-ef4d-472d-acfa-9a7d29659b8b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:20.638858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] The token cache is being invalidate because project 76c02d00830244688d278acc8fd2cecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:20.639276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:20.639940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-549897e8-ef4d-472d-acfa-9a7d29659b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:20.657484 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4123/8212] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:20 2022] DELETE /identity/v3/projects/76c02d00830244688d278acc8fd2cecc => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:21.953082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:21.954676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:21.970857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95132e-7d0c-44e5-a136-db945f8ca4bf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:22.076655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:22.077860 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:22.078478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:22.078920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] PATH_INFO: `/v3/users/14e08de43e3f4b5a83b00674f426e438` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:22.081101 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e08de43e3f4b5a83b00674f426e438)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:22.084106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:22.149646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:22.153112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e08de43e3f4b5a83b00674f426e438'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:22.176996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e08de43e3f4b5a83b00674f426e438'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:22.203920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e08de43e3f4b5a83b00674f426e438'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:22.204662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e08de43e3f4b5a83b00674f426e438'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:22.206019 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e08de43e3f4b5a83b00674f426e438'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:22.230102 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4090/8213] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:21 2022] DELETE /identity/v3/users/14e08de43e3f4b5a83b00674f426e438 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:22.230576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:22.231044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:22.231334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:22.255435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:22.256900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:22.364643 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4124/8214] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:23:22.372140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:22.374216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:22.502899 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc403b88-64b9-4dec-ab51-d8fe6558c60c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:22.508557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc24a37-af33-4a15-9e50-b8fd10dd264d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:22.509189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc24a37-af33-4a15-9e50-b8fd10dd264d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:22.509503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc24a37-af33-4a15-9e50-b8fd10dd264d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:22.509754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc24a37-af33-4a15-9e50-b8fd10dd264d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:22.529701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc24a37-af33-4a15-9e50-b8fd10dd264d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:22.532158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc24a37-af33-4a15-9e50-b8fd10dd264d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:22.546099 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4091/8215] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:22.903511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:22.905961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:22.927789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fc3d0-d9f1-4223-a94a-7a909b329649 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:22.936125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:22.938297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:22.938667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:22.939304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] PATH_INFO: `/v3/projects/0a6ae6b02e0a44148cfa744c94b9953b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:22.940769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6ae6b02e0a44148cfa744c94b9953b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:22.944288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.057368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6ae6b02e0a44148cfa744c94b9953b', 'request_id': 'req-6b315f42-4421-449d-9055-fecebc3fc5b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:23.071696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6ae6b02e0a44148cfa744c94b9953b', 'request_id': 'req-6b315f42-4421-449d-9055-fecebc3fc5b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:23.079993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6ae6b02e0a44148cfa744c94b9953b', 'request_id': 'req-6b315f42-4421-449d-9055-fecebc3fc5b6'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:23.099019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] The token cache is being invalidate because project 0a6ae6b02e0a44148cfa744c94b9953b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:23.099772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:23.100629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6b315f42-4421-449d-9055-fecebc3fc5b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:23.103219 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4125/8216] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:22 2022] DELETE /identity/v3/projects/0a6ae6b02e0a44148cfa744c94b9953b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:23.108342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f957c-a509-4fcb-aaf2-18419422a426 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.108915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f957c-a509-4fcb-aaf2-18419422a426 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.108915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f957c-a509-4fcb-aaf2-18419422a426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.136991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-b78f957c-a509-4fcb-aaf2-18419422a426 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:23.139847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b78f957c-a509-4fcb-aaf2-18419422a426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:23.224471 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4092/8217] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:23.232519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bef8d-d060-41f0-96c2-73e031a2ed5a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.233064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bef8d-d060-41f0-96c2-73e031a2ed5a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.233175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bef8d-d060-41f0-96c2-73e031a2ed5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.257492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-475bef8d-d060-41f0-96c2-73e031a2ed5a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:23.259128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-475bef8d-d060-41f0-96c2-73e031a2ed5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:23.268971 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4126/8218] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:23.273344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f957c-a509-4fcb-aaf2-18419422a426 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:23.283209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f957c-a509-4fcb-aaf2-18419422a426 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:23.289822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:23.290496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.290846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.291202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.291912 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:23:23.292277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:23.294008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.309281 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4093/8219] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:23:23.312953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bef8d-d060-41f0-96c2-73e031a2ed5a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:23.320040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bef8d-d060-41f0-96c2-73e031a2ed5a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:23.324783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:23.325393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.325702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.325971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.326608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:23.328625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.358780 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4127/8220] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:23 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:23.363292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:23.374819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ecd6-8bd8-4a52-a109-f0436398a69d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:23.424060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:23.443632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:23.444263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.444581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.444786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.445418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:23.448268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.474271 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4094/8221] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:23 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:23.478584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:23.490241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76e2fe-c16d-4616-a8ce-0e448f64eae3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:23.496047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:23.496572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.497308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.497795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.498482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:23.500109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.502476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:23.512638 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4128/8222] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:23.516541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:23.530174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972763d-5c75-4885-ba82-de0f19d3c1bf admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:23.535010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c3cca-efe6-4c15-9e09-af38fbd8dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:23.535686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c3cca-efe6-4c15-9e09-af38fbd8dd00 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.535971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c3cca-efe6-4c15-9e09-af38fbd8dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.536245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c3cca-efe6-4c15-9e09-af38fbd8dd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.536897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c3cca-efe6-4c15-9e09-af38fbd8dd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:23.538600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c3cca-efe6-4c15-9e09-af38fbd8dd00 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.552153 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4095/8223] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:23.556478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:23.566909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178ec0b-642c-4eab-9548-ba88079916b2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:23.571610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:23.572265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.572543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.572820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] PATH_INFO: `/v3/projects/daddbec9eb1b48328b14eb22ddfe139f/users/2a1210fc507a4c9bab87f09487dec9cb/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.589749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daddbec9eb1b48328b14eb22ddfe139f, user_id=2a1210fc507a4c9bab87f09487dec9cb, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:23.592710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.617901 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4129/8224] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:23 2022] PUT /identity/v3/projects/daddbec9eb1b48328b14eb22ddfe139f/users/2a1210fc507a4c9bab87f09487dec9cb/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:23.629448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.630180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.630619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.662611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] MFA Rules not processed for user `2a1210fc507a4c9bab87f09487dec9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:23.663989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:23.766612 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4096/8225] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:23.819262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:23.821929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:23.909995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd056e56-c6b5-4721-a9a3-e327e8db3853 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:23.918459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:23.919580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:23.920025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:23.920472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:23.931269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:23.944705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:23.971691 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4130/8226] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:27.156822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:27.159039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:27.275717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f45aa-ae82-4129-be7b-b99e0a5a2f4e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:27.281325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:27.282003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:27.282287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:27.282559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] PATH_INFO: `/v3/users/fd15b9d354eb44b0bfa83bcdcf5c1384` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:27.283729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd15b9d354eb44b0bfa83bcdcf5c1384)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:27.285831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:27.346786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd15b9d354eb44b0bfa83bcdcf5c1384'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.359332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:27.362412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd15b9d354eb44b0bfa83bcdcf5c1384'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.372044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd15b9d354eb44b0bfa83bcdcf5c1384'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.374141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd15b9d354eb44b0bfa83bcdcf5c1384'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.374773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1974f9bb-c872-4c40-a6df-a6e630847575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd15b9d354eb44b0bfa83bcdcf5c1384'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.391423 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4097/8227] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:27 2022] DELETE /identity/v3/users/fd15b9d354eb44b0bfa83bcdcf5c1384 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:27.605552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:27.629281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680906ff-5fc4-4aad-9770-d159a2426e79 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:27.712337 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:27.713297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:27.714299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:27.714583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] PATH_INFO: `/v3/projects/335955c0be3743eda2541b7d04ed2e06` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:27.715892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=335955c0be3743eda2541b7d04ed2e06)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:27.719730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:27.773385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335955c0be3743eda2541b7d04ed2e06', 'request_id': 'req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.780803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335955c0be3743eda2541b7d04ed2e06', 'request_id': 'req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.790926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335955c0be3743eda2541b7d04ed2e06', 'request_id': 'req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.803221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] The token cache is being invalidate because project 335955c0be3743eda2541b7d04ed2e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:27.803507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.804125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-8aca7643-dbc3-444f-98b6-0aa5f1a807ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:27.805945 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4131/8228] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:27 2022] DELETE /identity/v3/projects/335955c0be3743eda2541b7d04ed2e06 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:27.812554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572379b-3115-42dc-81ac-61ddf8d52c86 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:27.812842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572379b-3115-42dc-81ac-61ddf8d52c86 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:27.813113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572379b-3115-42dc-81ac-61ddf8d52c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:27.835805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e572379b-3115-42dc-81ac-61ddf8d52c86 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:27.836934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e572379b-3115-42dc-81ac-61ddf8d52c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:27.909624 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4098/8229] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:27.918312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ac0b-ba84-4dcc-948f-0718e213c747 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:27.918859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ac0b-ba84-4dcc-948f-0718e213c747 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:27.919195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ac0b-ba84-4dcc-948f-0718e213c747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:27.936144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:27.944812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e135ac0b-ba84-4dcc-948f-0718e213c747 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:27.946619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e135ac0b-ba84-4dcc-948f-0718e213c747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:27.965073 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4132/8230] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:27.965960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572379b-3115-42dc-81ac-61ddf8d52c86 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:27.978146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572379b-3115-42dc-81ac-61ddf8d52c86 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:27.983818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:27.984444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:27.984721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:27.984993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:27.985690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:23:27.986046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:27.987830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:28.004897 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4099/8231] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:23:28.006283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135ac0b-ba84-4dcc-948f-0718e213c747 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:28.018501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135ac0b-ba84-4dcc-948f-0718e213c747 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:28.023493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:28.024184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:28.024462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:28.024719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:28.025352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:28.040697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:28.077065 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4133/8232] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:28 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:28.081521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:28.091898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c212ba3-2efc-4c42-abbe-fcc8985c158f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:28.164819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:28.166346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:28.166746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:28.167124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:28.167858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:28.170097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:28.198956 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4100/8233] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:28 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:28.203585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:28.214919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff72c8-9b7a-45d1-8d77-5af165dbd5f1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:28.222183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:28.222917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:28.223186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:28.223460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:28.224142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:28.226232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:28.241568 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4134/8234] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:28.242754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:28.254362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812c69-332e-4a5b-9a29-d1cf6d3a4734 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:28.261097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2f377-c471-4ca3-93fa-60ef6184f808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:28.262141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2f377-c471-4ca3-93fa-60ef6184f808 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:28.262584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2f377-c471-4ca3-93fa-60ef6184f808 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:28.266365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2f377-c471-4ca3-93fa-60ef6184f808 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:28.266365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2f377-c471-4ca3-93fa-60ef6184f808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:28.266365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2f377-c471-4ca3-93fa-60ef6184f808 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:28.296170 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4101/8235] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:28.301592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:28.313223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634037ad-c10c-4f94-842b-c07717c33295 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:28.320617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:28.321619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:28.321998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:28.322370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] PATH_INFO: `/v3/projects/11f38ea8f2314c2c80bfcd3c793b2134/users/ee79165f64f84a0e9f685b99ac8a9e50/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:28.341842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f38ea8f2314c2c80bfcd3c793b2134, user_id=ee79165f64f84a0e9f685b99ac8a9e50, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:28.344981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:28.363509 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4135/8236] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:28 2022] PUT /identity/v3/projects/11f38ea8f2314c2c80bfcd3c793b2134/users/ee79165f64f84a0e9f685b99ac8a9e50/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:28.487207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f9042-ae44-4220-a68d-002e91f4d6d1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:28.487502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f9042-ae44-4220-a68d-002e91f4d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:28.487785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f9042-ae44-4220-a68d-002e91f4d6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:28.537402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-304f9042-ae44-4220-a68d-002e91f4d6d1 None None] MFA Rules not processed for user `ee79165f64f84a0e9f685b99ac8a9e50`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:28.539505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-304f9042-ae44-4220-a68d-002e91f4d6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:28.586006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:28.649977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:28.652622 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4102/8237] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:28.837187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348d30-a3be-46dc-959f-cffc0e52cdd2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:28.843601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:28.844944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:28.845412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:28.846048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:28.847025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:28.855589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:28.907115 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4136/8238] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:28 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:28.909772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f9042-ae44-4220-a68d-002e91f4d6d1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:28.924079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f9042-ae44-4220-a68d-002e91f4d6d1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:29.032372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:29.033280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:29.033690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:29.034104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:29.035127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:29.037363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:29.090235 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4103/8239] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:28 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:29.094178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:29.107042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3e9b1-af6f-4b04-a3b4-1c38cd5ca34e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:29.113077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe600d-9170-406d-897c-21ce6802ac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:29.113924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe600d-9170-406d-897c-21ce6802ac55 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:29.114211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe600d-9170-406d-897c-21ce6802ac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:29.114865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe600d-9170-406d-897c-21ce6802ac55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:29.115662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe600d-9170-406d-897c-21ce6802ac55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:29.118048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe600d-9170-406d-897c-21ce6802ac55 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:29.140951 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4137/8240] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:23:29.146472 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:29.185045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c044097-3d69-45fb-8ae8-426217bb8349 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:29.203662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:29.204700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:29.207185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:29.207185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] PATH_INFO: `/v3/projects/f7def931fe374db599afcd25b891688c/users/348e6263e76e47218a36a50c0e5c6ac6/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:29.250482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7def931fe374db599afcd25b891688c, user_id=348e6263e76e47218a36a50c0e5c6ac6, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:29.253448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:29.281676 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4104/8241] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:29 2022] PUT /identity/v3/projects/f7def931fe374db599afcd25b891688c/users/348e6263e76e47218a36a50c0e5c6ac6/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:29.292244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0b38e-8221-48ce-91ee-f670a11d6699 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:29.292806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0b38e-8221-48ce-91ee-f670a11d6699 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:29.293177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0b38e-8221-48ce-91ee-f670a11d6699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:29.373998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bae0b38e-8221-48ce-91ee-f670a11d6699 None None] MFA Rules not processed for user `348e6263e76e47218a36a50c0e5c6ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:29.375398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bae0b38e-8221-48ce-91ee-f670a11d6699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:29.511473 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4138/8242] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:29 2022] POST /identity/v3/auth/tokens => generated 3269 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:29.610053 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:29.612139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:29.757233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cc4c8-b4ed-4006-bcfa-30b14a8aa700 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:29.786955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:29.789375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:29.793584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:29.793893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:29.812051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:29.813799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:29.823841 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4105/8243] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:23:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 678 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:23:30.907438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0b38e-8221-48ce-91ee-f670a11d6699 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:30.921750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:31.010661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:31.012128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:31.043012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:31.060206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0b38e-8221-48ce-91ee-f670a11d6699 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:31.065582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:31.066311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:31.066622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:31.067113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] PATH_INFO: `/v3/users/ee79165f64f84a0e9f685b99ac8a9e50` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:31.068405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee79165f64f84a0e9f685b99ac8a9e50)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:31.070580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:31.134650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f12db-34e2-4b95-9b9a-986ab84115f3 service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:31.146690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:31.147412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:31.147695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:31.147975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] PATH_INFO: `/v3/users/ce3d2418f5d24b0dbf1142079eade5dc` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:31.149947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce3d2418f5d24b0dbf1142079eade5dc)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:31.153798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:31.155108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee79165f64f84a0e9f685b99ac8a9e50'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.156118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee79165f64f84a0e9f685b99ac8a9e50'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.168765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee79165f64f84a0e9f685b99ac8a9e50'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.171604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee79165f64f84a0e9f685b99ac8a9e50'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.224972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dbafbfb7-f460-4ea4-ba07-a0f428ac554b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee79165f64f84a0e9f685b99ac8a9e50'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.265054 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4139/8244] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:30 2022] DELETE /identity/v3/users/ee79165f64f84a0e9f685b99ac8a9e50 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:31.267784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:31.271192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:31.272673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:31.278412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce3d2418f5d24b0dbf1142079eade5dc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.310565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce3d2418f5d24b0dbf1142079eade5dc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.310814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:31.312150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:31.322639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce3d2418f5d24b0dbf1142079eade5dc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.325007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce3d2418f5d24b0dbf1142079eade5dc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.325997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e78f05a7-58ff-4894-ac1b-d75ea909eecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce3d2418f5d24b0dbf1142079eade5dc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:31.347719 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4106/8245] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:31 2022] DELETE /identity/v3/users/ce3d2418f5d24b0dbf1142079eade5dc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:23:31.353857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:31.354433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:31.355681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:31.387082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:31.388193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:31.410277 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4140/8246] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:31.428836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:31.436388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:31.492919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4107/8247] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:23:31.505683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:31.508241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:31.533865 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:31.594659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:31.604377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b08b9-715b-45ee-b37f-ba59103cbf4f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:31.631796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc9207-d49b-4011-8be4-25ce47cff470 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:31.636662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:31.637682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:31.637839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:31.638198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:31.651470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:31.653715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:31.691527 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4108/8248] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:31.701629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de53bfa-9744-4e48-b852-8705c809afff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:31.702656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de53bfa-9744-4e48-b852-8705c809afff service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:31.702656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de53bfa-9744-4e48-b852-8705c809afff service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:31.702879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de53bfa-9744-4e48-b852-8705c809afff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:31.716576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de53bfa-9744-4e48-b852-8705c809afff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:31.718326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de53bfa-9744-4e48-b852-8705c809afff service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:31.743704 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4141/8249] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:32.410431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:32.412415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:32.436897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4101a7-de2c-4caf-ba2b-1f6fba3016d7 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:32.443279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:32.445359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:32.445359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:32.445650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] PATH_INFO: `/v3/projects/2d7b965a444e4b079611adba36c73066` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:32.446679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d7b965a444e4b079611adba36c73066)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:32.459967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:32.585845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d7b965a444e4b079611adba36c73066', 'request_id': 'req-fcbe33fa-a1a2-477c-a5c5-a2faec728020'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.607452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d7b965a444e4b079611adba36c73066', 'request_id': 'req-fcbe33fa-a1a2-477c-a5c5-a2faec728020'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.620837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d7b965a444e4b079611adba36c73066', 'request_id': 'req-fcbe33fa-a1a2-477c-a5c5-a2faec728020'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.658499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] The token cache is being invalidate because project 2d7b965a444e4b079611adba36c73066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:32.658778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.661143 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.670952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd186bd-1ac8-4b2d-84bc-f38e8a19b2cc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:32.671544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd186bd-1ac8-4b2d-84bc-f38e8a19b2cc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:32.671544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd186bd-1ac8-4b2d-84bc-f38e8a19b2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:32.674611 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4109/8250] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:32 2022] DELETE /identity/v3/projects/2d7b965a444e4b079611adba36c73066 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:32.708816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ddd186bd-1ac8-4b2d-84bc-f38e8a19b2cc None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:32.710462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ddd186bd-1ac8-4b2d-84bc-f38e8a19b2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:32.732158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:32.734158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:32.869573 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4142/8251] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:32.872555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe33fa-a1a2-477c-a5c5-a2faec728020 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:32.877895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:32.878260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6eb2d-bd0f-49a7-884c-5a2061ef5698 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:32.878582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:32.878713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6eb2d-bd0f-49a7-884c-5a2061ef5698 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:32.879185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6eb2d-bd0f-49a7-884c-5a2061ef5698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:32.879541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:32.879926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] PATH_INFO: `/v3/projects/11f38ea8f2314c2c80bfcd3c793b2134` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:32.881922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f38ea8f2314c2c80bfcd3c793b2134)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:32.884100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:32.919465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-79a6eb2d-bd0f-49a7-884c-5a2061ef5698 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:32.920937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-79a6eb2d-bd0f-49a7-884c-5a2061ef5698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:32.934517 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4143/8252] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:32.936462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6eb2d-bd0f-49a7-884c-5a2061ef5698 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:32.949142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f38ea8f2314c2c80bfcd3c793b2134', 'request_id': 'req-89d6f6f0-d0fd-469e-828e-86094ee346cc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.949558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6eb2d-bd0f-49a7-884c-5a2061ef5698 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:32.963680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f38ea8f2314c2c80bfcd3c793b2134', 'request_id': 'req-89d6f6f0-d0fd-469e-828e-86094ee346cc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.977201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f38ea8f2314c2c80bfcd3c793b2134', 'request_id': 'req-89d6f6f0-d0fd-469e-828e-86094ee346cc'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:32.982214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:33.003320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] The token cache is being invalidate because project 11f38ea8f2314c2c80bfcd3c793b2134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:33.004204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:33.005185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:33.008657 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4110/8253] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:32 2022] DELETE /identity/v3/projects/11f38ea8f2314c2c80bfcd3c793b2134 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:23:33.013803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:33.016506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:33.128252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:33.129195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.129895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.130336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.131345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:23:33.131868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:33.141660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:33.162578 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4144/8254] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:23:33.164314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:33.166403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:33.211467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6f6f0-d0fd-469e-828e-86094ee346cc admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:33.211767 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9d9aa-500c-4c07-9b42-cb2c6ebd60c6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:33.218969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:33.219768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.220152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.220813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.221604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:33.223760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:33.224722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:33.225383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.225710 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.225979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] PATH_INFO: `/v3/users/348e6263e76e47218a36a50c0e5c6ac6` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.228062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348e6263e76e47218a36a50c0e5c6ac6)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:33.230096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:33.257333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:33.303431 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4145/8255] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:33 2022] POST /identity/v3/projects => generated 357 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:33.312644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:33.351526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348e6263e76e47218a36a50c0e5c6ac6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:33.380758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348e6263e76e47218a36a50c0e5c6ac6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:33.399526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348e6263e76e47218a36a50c0e5c6ac6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:33.400454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348e6263e76e47218a36a50c0e5c6ac6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:33.401707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348e6263e76e47218a36a50c0e5c6ac6'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:33.418098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06bd32-2685-41d8-99bf-c0d7b4932c60 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:33.423934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:33.424878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.425220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.425593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.426385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:33.429424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:33.429792 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4111/8256] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:33 2022] DELETE /identity/v3/users/348e6263e76e47218a36a50c0e5c6ac6 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:33.461630 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4146/8257] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:33 2022] POST /identity/v3/users => generated 279 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:33.462139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:33.463698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:33.552038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bacf6-86d7-400f-8367-1cca77403e11 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:33.560834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:33.561935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.562408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.562956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.565841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:33.568488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:33.589210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:33.614729 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4112/8258] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:33.619972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db84ece-2fc6-459a-918b-76eb0dba01fd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:33.630963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85206bbb-6d8d-4e9e-b830-3ff9a1b82d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:33.632132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85206bbb-6d8d-4e9e-b830-3ff9a1b82d18 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.633206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85206bbb-6d8d-4e9e-b830-3ff9a1b82d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.633556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85206bbb-6d8d-4e9e-b830-3ff9a1b82d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.634283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85206bbb-6d8d-4e9e-b830-3ff9a1b82d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:33.636125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85206bbb-6d8d-4e9e-b830-3ff9a1b82d18 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:33.655494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4147/8259] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:33.656147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:33.670135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70fe8c-a6e2-4da0-9e0f-f5a724512eb7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:33.675797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:33.676481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.676766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.677046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] PATH_INFO: `/v3/projects/c1e7eb5d14114976a662200156c13bc9/users/17e4d830f68543a6a5563ac020f20bd0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.681319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:33.695225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e7eb5d14114976a662200156c13bc9, user_id=17e4d830f68543a6a5563ac020f20bd0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:33.699804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:33.731094 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4113/8260] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:33 2022] PUT /identity/v3/projects/c1e7eb5d14114976a662200156c13bc9/users/17e4d830f68543a6a5563ac020f20bd0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:33.731341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:33.731891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:33.731891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:33.773352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] MFA Rules not processed for user `17e4d830f68543a6a5563ac020f20bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:33.775476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:33.895871 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4148/8261] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:23:33.928442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:33.930303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:34.018320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd18da9-b0f5-4875-accb-0afe16000550 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:34.023568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b5082-98cc-47cc-b2cf-0c79a22cb8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:34.024336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5082-98cc-47cc-b2cf-0c79a22cb8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:34.024618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5082-98cc-47cc-b2cf-0c79a22cb8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:34.024889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b5082-98cc-47cc-b2cf-0c79a22cb8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:34.035435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b5082-98cc-47cc-b2cf-0c79a22cb8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:34.036946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b5082-98cc-47cc-b2cf-0c79a22cb8b4 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:34.048576 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4114/8262] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:34.306885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:34.308943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:34.466017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5778d4-924d-4a25-8f30-42f3c5a070a9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:34.483455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:34.484133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:34.484410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:34.485133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] PATH_INFO: `/v3/projects/f7def931fe374db599afcd25b891688c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:34.487431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7def931fe374db599afcd25b891688c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:34.489381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:34.491700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:34.603802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7def931fe374db599afcd25b891688c', 'request_id': 'req-fc59baf5-87f5-4f5c-9bbf-47617dac9765'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:34.615925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7def931fe374db599afcd25b891688c', 'request_id': 'req-fc59baf5-87f5-4f5c-9bbf-47617dac9765'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:34.638866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7def931fe374db599afcd25b891688c', 'request_id': 'req-fc59baf5-87f5-4f5c-9bbf-47617dac9765'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:34.663451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] The token cache is being invalidate because project f7def931fe374db599afcd25b891688c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:34.663793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:34.665270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fc59baf5-87f5-4f5c-9bbf-47617dac9765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:34.669497 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4149/8263] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:34 2022] DELETE /identity/v3/projects/f7def931fe374db599afcd25b891688c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:34.672478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080bb1-ed43-4ade-8217-6de4f7f2e032 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:34.673289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080bb1-ed43-4ade-8217-6de4f7f2e032 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:34.674168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080bb1-ed43-4ade-8217-6de4f7f2e032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:34.734759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e5080bb1-ed43-4ade-8217-6de4f7f2e032 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:34.736346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e5080bb1-ed43-4ade-8217-6de4f7f2e032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:34.887415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4115/8264] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:34.898711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c878b-a11e-403a-8ec0-83b5320185aa None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:34.899281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c878b-a11e-403a-8ec0-83b5320185aa None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:34.899471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c878b-a11e-403a-8ec0-83b5320185aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:34.941916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5e5c878b-a11e-403a-8ec0-83b5320185aa None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:34.944930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5e5c878b-a11e-403a-8ec0-83b5320185aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:34.972231 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4150/8265] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:34.978390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080bb1-ed43-4ade-8217-6de4f7f2e032 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:34.992146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080bb1-ed43-4ade-8217-6de4f7f2e032 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:35.017419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:35.018540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.018995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.019479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:35.020417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:23:35.020760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:35.023557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:35.043741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c878b-a11e-403a-8ec0-83b5320185aa None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:35.049117 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4116/8266] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:23:35.054381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c878b-a11e-403a-8ec0-83b5320185aa None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:35.061632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:35.062315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.062594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.062860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:35.063505 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:35.065610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:35.120490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4151/8267] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:35 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:35.124885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:35.136535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0299b94-5586-4b5f-a1ea-e5594c400ed6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:35.236151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:35.236940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.237315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.237769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:35.239961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:35.245212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:35.279598 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4117/8268] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:35 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:35.283799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:35.295914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e9863-65d7-4d56-b0d8-6168bc7ca643 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:35.302031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:35.302697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.303039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.303944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:35.304093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:35.305669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:35.319289 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4152/8269] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:35.323480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:35.331168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:35.337800 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458fd54-8a53-4c3f-b23a-3d6fc929245e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:35.349368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed99dc4-af0b-4c4f-b0f0-b250878f94f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:35.350030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed99dc4-af0b-4c4f-b0f0-b250878f94f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.350326 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed99dc4-af0b-4c4f-b0f0-b250878f94f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.350597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed99dc4-af0b-4c4f-b0f0-b250878f94f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:35.351235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed99dc4-af0b-4c4f-b0f0-b250878f94f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:35.352876 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed99dc4-af0b-4c4f-b0f0-b250878f94f2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:35.398308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4118/8270] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:35.402418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:35.424989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2abd9a-7ec1-40ce-80c7-96ed29146469 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:35.444547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:35.445245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.445541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.445823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] PATH_INFO: `/v3/projects/81468f9a5a5d4b63bf62e9fe4bd19b56/users/2c778820854d4636ac34bd86ab7fbaff/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:35.503560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81468f9a5a5d4b63bf62e9fe4bd19b56, user_id=2c778820854d4636ac34bd86ab7fbaff, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:35.510760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:35.564421 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4153/8271] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:35 2022] PUT /identity/v3/projects/81468f9a5a5d4b63bf62e9fe4bd19b56/users/2c778820854d4636ac34bd86ab7fbaff/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:35.569379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383163-2bee-4500-a0c5-afd96a2386cb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.571774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383163-2bee-4500-a0c5-afd96a2386cb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.572178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383163-2bee-4500-a0c5-afd96a2386cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:35.615317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-08383163-2bee-4500-a0c5-afd96a2386cb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:35.619018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-08383163-2bee-4500-a0c5-afd96a2386cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:35.785745 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4119/8272] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:35.831589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:35.833676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:35.984776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df5016-bd91-445b-ba74-7e66dc14cc8e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:35.995479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:35.996605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:35.997067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:35.997548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:36.022823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:36.025506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:36.046230 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4154/8273] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:23:37.474284 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240042d4-2c18-448a-a7ee-acf66ecbb9a8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:37.474917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240042d4-2c18-448a-a7ee-acf66ecbb9a8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:37.476156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240042d4-2c18-448a-a7ee-acf66ecbb9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:37.537349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-240042d4-2c18-448a-a7ee-acf66ecbb9a8 None None] MFA Rules not processed for user `2c778820854d4636ac34bd86ab7fbaff`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:37.539006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-240042d4-2c18-448a-a7ee-acf66ecbb9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:37.680964 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4120/8274] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:37.685216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:37.698311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19c66-a837-467e-b87d-72e10375eb9a service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:37.709685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:37.711223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:37.711950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:37.712388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:37.713279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:37.715957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:37.764416 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4155/8275] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:37 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:37.767337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240042d4-2c18-448a-a7ee-acf66ecbb9a8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:37.786922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240042d4-2c18-448a-a7ee-acf66ecbb9a8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:37.915599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:37.916488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:37.916878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:37.917161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:37.918009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:37.921497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:37.960035 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4121/8276] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:37 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:37.960459 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:37.978689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c901da8-fad5-412e-97f6-ccb427548a6b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:37.987771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:37.988851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:37.990109 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:37.990949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:37.992758 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:37.997247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:37.999684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:38.020252 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4156/8277] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:38.020713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:38.043799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae919d0-5048-43d2-9840-8100299fb3d5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:38.057634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:38.058561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:38.058699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:38.058934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] PATH_INFO: `/v3/projects/c409519bdbbf4783b5184cd196e79626/users/2d96106fefe84b9ca41126ec77a25ef2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:38.075166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:38.090295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c409519bdbbf4783b5184cd196e79626, user_id=2d96106fefe84b9ca41126ec77a25ef2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:38.098296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:38.127132 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4122/8278] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:38 2022] PUT /identity/v3/projects/c409519bdbbf4783b5184cd196e79626/users/2d96106fefe84b9ca41126ec77a25ef2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:38.131849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:38.145356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e619239-cebc-4f66-bacd-bf72d56a92cf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:38.236461 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84282f74-c739-4cd5-a2bc-c60836317d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:38.237106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84282f74-c739-4cd5-a2bc-c60836317d89 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:38.237382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84282f74-c739-4cd5-a2bc-c60836317d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:38.237705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84282f74-c739-4cd5-a2bc-c60836317d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:38.238448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84282f74-c739-4cd5-a2bc-c60836317d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:38.240280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84282f74-c739-4cd5-a2bc-c60836317d89 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:38.252354 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4157/8279] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:38.257146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:38.273922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb0f9-ce3e-40ec-a389-0e0a72fda4e2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:38.288429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b405d6a-417f-4ef0-bc91-59a6dc9251a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:38.289277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b405d6a-417f-4ef0-bc91-59a6dc9251a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:38.289601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b405d6a-417f-4ef0-bc91-59a6dc9251a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:38.289878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b405d6a-417f-4ef0-bc91-59a6dc9251a6 admin admin] PATH_INFO: `/v3/projects/c409519bdbbf4783b5184cd196e79626/users/2d96106fefe84b9ca41126ec77a25ef2/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:38.291950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b405d6a-417f-4ef0-bc91-59a6dc9251a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c409519bdbbf4783b5184cd196e79626, user_id=2d96106fefe84b9ca41126ec77a25ef2, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:38.294707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b405d6a-417f-4ef0-bc91-59a6dc9251a6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:38.308489 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4123/8280] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:38 2022] PUT /identity/v3/projects/c409519bdbbf4783b5184cd196e79626/users/2d96106fefe84b9ca41126ec77a25ef2/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:39.387989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:39.388478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:39.388603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:39.450999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] MFA Rules not processed for user `2d96106fefe84b9ca41126ec77a25ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:39.454394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:39.591046 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4158/8281] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:23:39.596637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:39.597182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:39.597534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:39.644697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] MFA Rules not processed for user `2c778820854d4636ac34bd86ab7fbaff`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:39.646405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:39.812546 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4124/8282] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:39.827225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:39.830578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:40.014489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b77f4-f1ce-4605-93ab-76bd55e2ca3f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:40.021479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:40.022252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:40.022547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:40.022823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:40.032857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:40.034529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:40.048450 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4159/8283] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:47.252102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:47.253536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:47.386421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42bcae-abef-4f2d-9377-fe1875f5f6c1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:47.390225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:47.390926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:47.391266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:47.391593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:47.393537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:47.412078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:47.413835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:47.432878 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4125/8284] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:47.497276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:47.500592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:47.520410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:47.533288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09473d3a-a528-4de4-8c90-e799eddf1faa service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:47.539112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:47.539960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:47.540318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:47.541164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:47.543946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:47.562852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:47.565241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:47.600457 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4160/8285] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:47.661150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:47.664133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:47.778525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8a77-7192-4a4f-91af-bcf682590045 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:47.788725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:47.789403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:47.790314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:47.790604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] PATH_INFO: `/v3/users/0302d735014e41e9aa98697c3de5f850` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:47.796141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0302d735014e41e9aa98697c3de5f850)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:47.800348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:47.819198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:47.903801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0302d735014e41e9aa98697c3de5f850'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:47.924565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0302d735014e41e9aa98697c3de5f850'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:47.936928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0302d735014e41e9aa98697c3de5f850'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:47.937874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0302d735014e41e9aa98697c3de5f850'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:47.938673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0302d735014e41e9aa98697c3de5f850'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:47.955272 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4126/8286] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:47 2022] DELETE /identity/v3/users/0302d735014e41e9aa98697c3de5f850 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:48.149016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:48.167214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8501803-3b73-46e4-b0d1-8ac7bf1f4755 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:48.262193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:48.263427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:48.264032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:48.264531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] PATH_INFO: `/v3/projects/364569c3608b459ebc6ca73cfed86303` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:48.266247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=364569c3608b459ebc6ca73cfed86303)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:48.269175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:48.364124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364569c3608b459ebc6ca73cfed86303', 'request_id': 'req-a1e3d884-3f39-45e9-b2d2-11075ce77acf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.377840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364569c3608b459ebc6ca73cfed86303', 'request_id': 'req-a1e3d884-3f39-45e9-b2d2-11075ce77acf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.414966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364569c3608b459ebc6ca73cfed86303', 'request_id': 'req-a1e3d884-3f39-45e9-b2d2-11075ce77acf'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.437869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] The token cache is being invalidate because project 364569c3608b459ebc6ca73cfed86303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:48.438664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.439055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.445835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4161/8287] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:48 2022] DELETE /identity/v3/projects/364569c3608b459ebc6ca73cfed86303 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:48.446538 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:48.449214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:48.466045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79a764-3c74-4651-b9a5-dc1fbd960606 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:48.553554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:48.554318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:48.554670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:48.555015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] PATH_INFO: `/v3/users/70b0d8337e20432584f80476b46453f0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:48.556327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b0d8337e20432584f80476b46453f0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:48.558643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:48.638222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b0d8337e20432584f80476b46453f0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.654549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b0d8337e20432584f80476b46453f0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.665995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b0d8337e20432584f80476b46453f0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.667276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b0d8337e20432584f80476b46453f0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.668694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b0d8337e20432584f80476b46453f0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:48.692526 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4127/8288] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:48 2022] DELETE /identity/v3/users/70b0d8337e20432584f80476b46453f0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:49.154852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.156564 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:49.178916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3d884-3f39-45e9-b2d2-11075ce77acf admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.260905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.261826 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.262105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.262481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] PATH_INFO: `/v3/projects/718efb2481024b278d50730fec729eeb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.263557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] RBAC: Authorizing `identity:delete_project(project_id=718efb2481024b278d50730fec729eeb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.265988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.267370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.270758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:49.290961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d818c44-8da2-497f-b221-04add5ff5c8f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.296866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:49.297674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.299397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.299699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.300150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] PATH_INFO: `/v3/users/2a1210fc507a4c9bab87f09487dec9cb` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.301756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1210fc507a4c9bab87f09487dec9cb)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.303890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.308266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:49.328432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718efb2481024b278d50730fec729eeb', 'request_id': 'req-57f56528-c091-4406-937b-88fc7760a13a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.335383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718efb2481024b278d50730fec729eeb', 'request_id': 'req-57f56528-c091-4406-937b-88fc7760a13a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.344920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718efb2481024b278d50730fec729eeb', 'request_id': 'req-57f56528-c091-4406-937b-88fc7760a13a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.357770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] The token cache is being invalidate because project 718efb2481024b278d50730fec729eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:49.358116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.359097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-57f56528-c091-4406-937b-88fc7760a13a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.361203 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4162/8289] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:49 2022] DELETE /identity/v3/projects/718efb2481024b278d50730fec729eeb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:49.366805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaa4b2-d369-447d-a701-8e80aec8e06b None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.367174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaa4b2-d369-447d-a701-8e80aec8e06b None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.367696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaa4b2-d369-447d-a701-8e80aec8e06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.372842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1210fc507a4c9bab87f09487dec9cb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.389080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1210fc507a4c9bab87f09487dec9cb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.393469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3edaa4b2-d369-447d-a701-8e80aec8e06b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:49.395012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3edaa4b2-d369-447d-a701-8e80aec8e06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:49.399847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1210fc507a4c9bab87f09487dec9cb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.400772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1210fc507a4c9bab87f09487dec9cb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.403047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8fb6b00f-5203-421b-a446-3aacdc5090da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1210fc507a4c9bab87f09487dec9cb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:49.417266 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4128/8290] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:23:49 2022] DELETE /identity/v3/users/2a1210fc507a4c9bab87f09487dec9cb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:23:49.421791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.422082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.422355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.442885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:49.444382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:49.470469 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4163/8291] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:49.481227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9018e2c-0409-48c4-b7f5-ae1b49c7723f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.481617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9018e2c-0409-48c4-b7f5-ae1b49c7723f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.481918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9018e2c-0409-48c4-b7f5-ae1b49c7723f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.508792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d9018e2c-0409-48c4-b7f5-ae1b49c7723f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:49.510128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d9018e2c-0409-48c4-b7f5-ae1b49c7723f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:49.556413 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4129/8292] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:49.567523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.569211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:49.596882 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4164/8293] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:49.598615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9018e2c-0409-48c4-b7f5-ae1b49c7723f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.609430 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9018e2c-0409-48c4-b7f5-ae1b49c7723f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.614482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.615098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.615377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.615646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.616996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:23:49.617312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.618991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.631982 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4165/8294] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:23:49.637965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.646902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29af298-cc9a-4337-a7c2-49985cec06d5 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.650626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.651214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf836e2d-998a-4466-bb38-d5aa7e4d690f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.651347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.651528 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.651774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.656056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.656663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.657067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.657350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.658243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.660815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.662994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.675624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.688181 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4130/8295] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:23:49.701516 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4166/8296] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:49 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:49.704582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.714999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32e97c-8827-43fe-93af-afafa4818eda service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.736831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:49.781099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.781677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.781990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.782303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.783023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.785272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.811568 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4131/8297] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:49 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:49.813213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.824335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7938d-21fc-49de-97bf-66eec79f3470 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.829485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.830114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.830384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.830655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.831303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.832988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.846554 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4167/8298] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:49.848449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.863244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1b97-fc63-40e1-8cd2-e5278601cc49 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.869646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d7738-dad1-4296-8af4-f6dad56db42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.870265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7738-dad1-4296-8af4-f6dad56db42c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.870546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7738-dad1-4296-8af4-f6dad56db42c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.870817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d7738-dad1-4296-8af4-f6dad56db42c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.871464 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d7738-dad1-4296-8af4-f6dad56db42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.873086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d7738-dad1-4296-8af4-f6dad56db42c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.890001 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4132/8299] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:49.891532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:49.899536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:49.904092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e535e-38c5-47f2-973f-6181ad7de7ce admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:49.911565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:49.913343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.914329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.914754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] PATH_INFO: `/v3/projects/b99d2a0b4d7e40a38e682fc147e968af/users/342e60103c9b45bcb348b6f8bcd686ca/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:49.937079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99d2a0b4d7e40a38e682fc147e968af, user_id=342e60103c9b45bcb348b6f8bcd686ca, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:49.940851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:49.963801 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4168/8300] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:49 2022] PUT /identity/v3/projects/b99d2a0b4d7e40a38e682fc147e968af/users/342e60103c9b45bcb348b6f8bcd686ca/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:49.968982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097472-6c17-45cd-95a5-3e371135bc3e None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:49.969273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097472-6c17-45cd-95a5-3e371135bc3e None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:49.969624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097472-6c17-45cd-95a5-3e371135bc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.005226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c1097472-6c17-45cd-95a5-3e371135bc3e None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:50.007482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c1097472-6c17-45cd-95a5-3e371135bc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:50.079587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.081381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:50.085893 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4133/8301] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:50.095467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6774793-6bfd-4660-8459-13ada13c8df6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.099116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097472-6c17-45cd-95a5-3e371135bc3e None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.106507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.107193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.107476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.107748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] PATH_INFO: `/v3/projects/daddbec9eb1b48328b14eb22ddfe139f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.110705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daddbec9eb1b48328b14eb22ddfe139f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.113199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.176898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097472-6c17-45cd-95a5-3e371135bc3e None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.180542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daddbec9eb1b48328b14eb22ddfe139f', 'request_id': 'req-67540e72-abff-42c8-baf4-42a82693b7c7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:50.187867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daddbec9eb1b48328b14eb22ddfe139f', 'request_id': 'req-67540e72-abff-42c8-baf4-42a82693b7c7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:50.198970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daddbec9eb1b48328b14eb22ddfe139f', 'request_id': 'req-67540e72-abff-42c8-baf4-42a82693b7c7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:50.212220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] The token cache is being invalidate because project daddbec9eb1b48328b14eb22ddfe139f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:23:50.212651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:50.213383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-67540e72-abff-42c8-baf4-42a82693b7c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:23:50.214925 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4169/8302] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:23:50 2022] DELETE /identity/v3/projects/daddbec9eb1b48328b14eb22ddfe139f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:23:50.219961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784efc2-c68d-4287-a2d6-f9572c849bfb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.221192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784efc2-c68d-4287-a2d6-f9572c849bfb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.221555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784efc2-c68d-4287-a2d6-f9572c849bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.239523 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.240642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.241526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.241805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.243674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:50.246284 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1784efc2-c68d-4287-a2d6-f9572c849bfb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:50.247449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1784efc2-c68d-4287-a2d6-f9572c849bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:50.258137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:50.316706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.318212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.331020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4170/8303] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:50.333503 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4134/8304] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:50.338634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5c540-c863-4d7c-b17d-b06d91c36d99 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.338972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5c540-c863-4d7c-b17d-b06d91c36d99 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.339190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5c540-c863-4d7c-b17d-b06d91c36d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.359827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c5c5c540-c863-4d7c-b17d-b06d91c36d99 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:50.361082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c5c5c540-c863-4d7c-b17d-b06d91c36d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:50.374742 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4171/8305] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:50.381498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.396657 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239219ff-19c2-4d4b-ac68-cbcee7b2a30d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.406458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.406917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.407196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.407467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.408133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:23:50.408831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.410145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.436278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5c540-c863-4d7c-b17d-b06d91c36d99 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.447064 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4135/8306] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:23:50.453493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5c540-c863-4d7c-b17d-b06d91c36d99 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.460905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.462125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.462376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.462607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.463294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.465709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.531593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.543372 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4172/8307] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:50 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:50.550117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31354df8-62f1-4cc2-b45c-50210a6b2113 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.658154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.658827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.658827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.659006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.659585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.661617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.706023 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4136/8308] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:50 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:50.710619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.724122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf8c0d-5fb2-47d6-affe-3eb25b3f8e91 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.729259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.730751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.731033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.731308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.731966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.734806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.749371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4173/8309] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:50.750940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.764205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712d74-821f-4a82-9a73-646e070f5ca9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.770596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1b32a-4c3a-4e50-a482-30b6baea6e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.771409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1b32a-4c3a-4e50-a482-30b6baea6e4b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.772551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1b32a-4c3a-4e50-a482-30b6baea6e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.773081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1b32a-4c3a-4e50-a482-30b6baea6e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.773634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1b32a-4c3a-4e50-a482-30b6baea6e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.775841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1b32a-4c3a-4e50-a482-30b6baea6e4b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.793573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4137/8310] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:50.797860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:50.810601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af47415-469b-4d85-8bad-475f2610d842 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:50.817947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:50.818765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.819051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.819321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] PATH_INFO: `/v3/projects/2fa3b3167cc04ddc86173149ad7065cd/users/fbe252bb410a4f6e81b95c5778bef1c4/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.837591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa3b3167cc04ddc86173149ad7065cd, user_id=fbe252bb410a4f6e81b95c5778bef1c4, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:50.840432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:50.851649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:50.859844 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4174/8311] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:50 2022] PUT /identity/v3/projects/2fa3b3167cc04ddc86173149ad7065cd/users/fbe252bb410a4f6e81b95c5778bef1c4/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:50.868868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0c448-d5d6-4f7c-bf06-4c3e9f210d29 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:50.869229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0c448-d5d6-4f7c-bf06-4c3e9f210d29 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:50.869563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0c448-d5d6-4f7c-bf06-4c3e9f210d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:50.905015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a0c0c448-d5d6-4f7c-bf06-4c3e9f210d29 None None] MFA Rules not processed for user `fbe252bb410a4f6e81b95c5778bef1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:50.907007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a0c0c448-d5d6-4f7c-bf06-4c3e9f210d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:51.012721 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4138/8312] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:51.025819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:51.027625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:51.154368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb66bdc-7e86-4ce7-b781-55b60e86d156 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:51.158234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:51.159070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:51.159317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:51.159417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:51.185032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:51.187364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:51.202127 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4175/8313] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:58.987641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce139392-2432-42c7-bc9b-be03c57db88a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:58.988186 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce139392-2432-42c7-bc9b-be03c57db88a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:58.988289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce139392-2432-42c7-bc9b-be03c57db88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:59.001418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:59.064772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ce139392-2432-42c7-bc9b-be03c57db88a None None] MFA Rules not processed for user `342e60103c9b45bcb348b6f8bcd686ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:23:59.066221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ce139392-2432-42c7-bc9b-be03c57db88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:59.170667 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4139/8314] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:23:59.175089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:59.177196 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:59.259985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3ecf-8bf6-4b7d-b6e6-982d77feecb4 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:59.265335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:59.266076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:59.267033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:59.267291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:59.268054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:59.270209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:59.315092 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4176/8315] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:23:59 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:59.315581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce139392-2432-42c7-bc9b-be03c57db88a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:59.333413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce139392-2432-42c7-bc9b-be03c57db88a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:59.408349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:59.409017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:59.409294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:59.409744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:59.410530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:59.412512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:59.440283 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4140/8316] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:23:59 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:23:59.444540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:59.457872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b22c0-f1f7-4c99-8d39-de46da14044c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:59.467189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:59.468171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:59.469084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:59.469368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:59.470061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:59.471687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:59.483645 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4177/8317] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:59.490136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:59.511619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f5e1b-79af-4d58-a1cc-d875eb206f35 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:59.520617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:59.521580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:59.522206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:59.522648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] PATH_INFO: `/v3/projects/b67f323d906a456a9497b7cbe701dfd0/users/98278fb103484afdbbe3a8fab1ce6519/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:59.542662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67f323d906a456a9497b7cbe701dfd0, user_id=98278fb103484afdbbe3a8fab1ce6519, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:59.545823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:59.569143 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4141/8318] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:59 2022] PUT /identity/v3/projects/b67f323d906a456a9497b7cbe701dfd0/users/98278fb103484afdbbe3a8fab1ce6519/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:23:59.573629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:59.588288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0c97-ae5e-4564-a4fe-5b1d3ab7b3f8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:59.680753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:59.683158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:59.683614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:59.684018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:59.685380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:59.687591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:59.716318 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4178/8319] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:23:59.717008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:59.728239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c74e5-e29f-49a4-b6ab-145e01cc0db9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:59.733677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86900cd3-7f51-492d-88e9-7d3da4732deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:59.734546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900cd3-7f51-492d-88e9-7d3da4732deb admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:59.734997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900cd3-7f51-492d-88e9-7d3da4732deb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:59.735697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900cd3-7f51-492d-88e9-7d3da4732deb admin admin] PATH_INFO: `/v3/projects/b67f323d906a456a9497b7cbe701dfd0/users/98278fb103484afdbbe3a8fab1ce6519/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:59.737923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86900cd3-7f51-492d-88e9-7d3da4732deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67f323d906a456a9497b7cbe701dfd0, user_id=98278fb103484afdbbe3a8fab1ce6519, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:59.741574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86900cd3-7f51-492d-88e9-7d3da4732deb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:59.756152 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4142/8320] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:23:59 2022] PUT /identity/v3/projects/b67f323d906a456a9497b7cbe701dfd0/users/98278fb103484afdbbe3a8fab1ce6519/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:24:06.101919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:06.119771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae9153-3a56-48c6-a0a3-1dd5b386b72d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:06.159363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:06.261895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:06.263729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:06.264055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:06.264950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] PATH_INFO: `/v3/users/fbe252bb410a4f6e81b95c5778bef1c4` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:06.266765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbe252bb410a4f6e81b95c5778bef1c4)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:06.268912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:06.338280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbe252bb410a4f6e81b95c5778bef1c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:06.339346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbe252bb410a4f6e81b95c5778bef1c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:06.351076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbe252bb410a4f6e81b95c5778bef1c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:06.352986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbe252bb410a4f6e81b95c5778bef1c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:06.384171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbe252bb410a4f6e81b95c5778bef1c4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:06.417276 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4179/8321] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:06 2022] DELETE /identity/v3/users/fbe252bb410a4f6e81b95c5778bef1c4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:06.421638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:06.422123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:06.422904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:06.455839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:06.457246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:06.516685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:06.553545 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4143/8322] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:06.578971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:06.581193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:06.694790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd2b1f-4876-4e34-ab8a-6a4fab32a382 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:06.700541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e0ba0-8a96-4058-86d0-0cb8c141ecff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:06.709993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e0ba0-8a96-4058-86d0-0cb8c141ecff service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:06.711437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e0ba0-8a96-4058-86d0-0cb8c141ecff service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:06.712002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e0ba0-8a96-4058-86d0-0cb8c141ecff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:06.738097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e0ba0-8a96-4058-86d0-0cb8c141ecff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:06.743050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e0ba0-8a96-4058-86d0-0cb8c141ecff service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:06.772870 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4180/8323] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:24:07.239895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:07.242367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:07.264779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78118182-614a-4631-8e24-fc493eaf60d7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:07.293926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:07.296141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:07.296651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:07.297312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] PATH_INFO: `/v3/projects/2fa3b3167cc04ddc86173149ad7065cd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:07.299146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa3b3167cc04ddc86173149ad7065cd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:07.301773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:07.430208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa3b3167cc04ddc86173149ad7065cd', 'request_id': 'req-14cabc46-b653-4c5d-b72e-f48312671d52'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:07.445336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa3b3167cc04ddc86173149ad7065cd', 'request_id': 'req-14cabc46-b653-4c5d-b72e-f48312671d52'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:07.455506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa3b3167cc04ddc86173149ad7065cd', 'request_id': 'req-14cabc46-b653-4c5d-b72e-f48312671d52'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:07.486936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] The token cache is being invalidate because project 2fa3b3167cc04ddc86173149ad7065cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:07.487431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:07.488331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-14cabc46-b653-4c5d-b72e-f48312671d52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:07.490625 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4144/8324] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:07 2022] DELETE /identity/v3/projects/2fa3b3167cc04ddc86173149ad7065cd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:07.495943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6db9-d4f5-4c1f-84a6-99ff795c05f9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:07.496411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6db9-d4f5-4c1f-84a6-99ff795c05f9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:07.496482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6db9-d4f5-4c1f-84a6-99ff795c05f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:07.539333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7e0a6db9-d4f5-4c1f-84a6-99ff795c05f9 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:07.540586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e0a6db9-d4f5-4c1f-84a6-99ff795c05f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:07.649451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:07.752140 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4181/8325] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:07.761058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955edd-1643-4cfd-8d5f-ad2311ff9f8c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:07.761786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955edd-1643-4cfd-8d5f-ad2311ff9f8c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:07.762197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955edd-1643-4cfd-8d5f-ad2311ff9f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:07.791607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-89955edd-1643-4cfd-8d5f-ad2311ff9f8c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:07.793292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-89955edd-1643-4cfd-8d5f-ad2311ff9f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:07.804180 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4145/8326] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:07.808463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a6db9-d4f5-4c1f-84a6-99ff795c05f9 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:07.824084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a6db9-d4f5-4c1f-84a6-99ff795c05f9 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:07.834937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:07.835842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:07.836552 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:07.837268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:07.845491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:24:07.847509 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:07.850678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:07.896671 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4182/8327] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:24:07.896840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955edd-1643-4cfd-8d5f-ad2311ff9f8c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:07.907810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955edd-1643-4cfd-8d5f-ad2311ff9f8c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:07.912358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:07.913266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:07.913266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:07.913587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:07.914457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:07.916818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:07.970023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:07.984793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4146/8328] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:07 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:07.988030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c4899-ba4d-4488-b562-124166784a1c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:08.094385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:08.095292 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.095802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.096219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.097040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:08.099571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:08.136256 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4183/8329] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:07 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:08.137641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:08.149165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae98a47-b35c-43a1-86ab-8cdb143b92ca admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:08.154985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:08.156139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.156515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.156878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.157737 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:08.159793 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:08.183439 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4147/8330] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:08.187720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:08.201901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1198a9-91e6-48c9-9adf-c4a1fd72bcc4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:08.223001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feba1d0-e24e-469f-8465-34c03c284ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:08.223648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feba1d0-e24e-469f-8465-34c03c284ede admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.223947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feba1d0-e24e-469f-8465-34c03c284ede admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.224214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feba1d0-e24e-469f-8465-34c03c284ede admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.224874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feba1d0-e24e-469f-8465-34c03c284ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:08.226546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feba1d0-e24e-469f-8465-34c03c284ede admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:08.242834 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4184/8331] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:08.244174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:08.259348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a434b-478d-43f2-8525-8f7a6b0e93e0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:08.267020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:08.268083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.268303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.268579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] PATH_INFO: `/v3/projects/93c7626dedfb4acbaec8d2627c570b80/users/57eafbe4f20f4c40b007fb5478205a9f/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.313085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c7626dedfb4acbaec8d2627c570b80, user_id=57eafbe4f20f4c40b007fb5478205a9f, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:08.319343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:08.344848 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4148/8332] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:08 2022] PUT /identity/v3/projects/93c7626dedfb4acbaec8d2627c570b80/users/57eafbe4f20f4c40b007fb5478205a9f/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:08.355243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df63a9-3b70-4713-ae11-ee156b4f1473 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.355512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df63a9-3b70-4713-ae11-ee156b4f1473 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.355805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df63a9-3b70-4713-ae11-ee156b4f1473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.402072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-93df63a9-3b70-4713-ae11-ee156b4f1473 None None] MFA Rules not processed for user `57eafbe4f20f4c40b007fb5478205a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:08.403593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-93df63a9-3b70-4713-ae11-ee156b4f1473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:08.532374 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4185/8333] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:08.533387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:08.715985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c7787-0cd3-45dd-b43b-1afbdffc77c0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:08.732637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed5bb0-6df0-4947-b9e3-be84f1b4f4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:08.733566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed5bb0-6df0-4947-b9e3-be84f1b4f4be admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.733983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed5bb0-6df0-4947-b9e3-be84f1b4f4be admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.734486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed5bb0-6df0-4947-b9e3-be84f1b4f4be admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.735402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed5bb0-6df0-4947-b9e3-be84f1b4f4be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:08.737565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed5bb0-6df0-4947-b9e3-be84f1b4f4be admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:08.779601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:08.812973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df63a9-3b70-4713-ae11-ee156b4f1473 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:08.837701 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4149/8334] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:08 2022] POST /identity/v3/projects => generated 381 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:08.845236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df63a9-3b70-4713-ae11-ee156b4f1473 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:08.870552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.870836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.871113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.896723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:08.915654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] MFA Rules not processed for user `98278fb103484afdbbe3a8fab1ce6519`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:08.917798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:08.936035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:08.936716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.936994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.937254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.938394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:08.940480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:08.969299 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4186/8335] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:08 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:08.973489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:08.984731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac50e5-d944-447c-9b59-3268a805c541 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:08.990143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:08.991277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:08.991625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:08.991970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:08.992730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:08.994479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:09.009255 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4187/8336] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:24:09.010699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:09.021445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d813a-4608-4c97-9398-0f0876a4b833 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:09.026804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:09.027480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:09.027818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:09.028102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] PATH_INFO: `/v3/projects/68203fde4361416081c319be14988e0d/users/ed9c5e9b5d3e4ceeb0bc456c1068742b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:09.037944 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4150/8337] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:09.047042 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:09.049746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:09.050847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68203fde4361416081c319be14988e0d, user_id=ed9c5e9b5d3e4ceeb0bc456c1068742b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:09.054932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:09.079002 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4188/8338] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:09 2022] PUT /identity/v3/projects/68203fde4361416081c319be14988e0d/users/ed9c5e9b5d3e4ceeb0bc456c1068742b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:09.089755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:09.204465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5f332-0344-4b06-8a8f-77d12a5b9a48 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:09.210587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:09.211416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:09.211706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc108ae-ed62-4ac7-9a26-60147e633736 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:09.211994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:09.212400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:09.213523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:09.216521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:09.239829 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4189/8339] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:09.244591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:09.256499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead189d-c1c8-4fd0-9bfa-ea087f1b84bb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:09.262208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91f0e0-eb41-4d9a-9bc0-b5b8078f613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:09.263271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91f0e0-eb41-4d9a-9bc0-b5b8078f613e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:09.263684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91f0e0-eb41-4d9a-9bc0-b5b8078f613e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:09.264230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91f0e0-eb41-4d9a-9bc0-b5b8078f613e admin admin] PATH_INFO: `/v3/projects/68203fde4361416081c319be14988e0d/users/ed9c5e9b5d3e4ceeb0bc456c1068742b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:09.270544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91f0e0-eb41-4d9a-9bc0-b5b8078f613e admin admin] RBAC: Authorizing `identity:create_grant(project_id=68203fde4361416081c319be14988e0d, user_id=ed9c5e9b5d3e4ceeb0bc456c1068742b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:09.273744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91f0e0-eb41-4d9a-9bc0-b5b8078f613e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:09.296349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:09.297353 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4190/8340] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:09 2022] PUT /identity/v3/projects/68203fde4361416081c319be14988e0d/users/ed9c5e9b5d3e4ceeb0bc456c1068742b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:09.298259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:09.298566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:09.298846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:09.299917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7201-e70d-4607-9e6d-6c87d24cb1a5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:09.300449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7201-e70d-4607-9e6d-6c87d24cb1a5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:09.300861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c7201-e70d-4607-9e6d-6c87d24cb1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:09.303986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:09.342428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-777c7201-e70d-4607-9e6d-6c87d24cb1a5 None None] MFA Rules not processed for user `ed9c5e9b5d3e4ceeb0bc456c1068742b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:09.344385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-777c7201-e70d-4607-9e6d-6c87d24cb1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:09.362024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:09.406772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:09.410634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:09.422620 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4151/8341] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:24:09.517605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:09.552956 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4191/8342] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:09.560948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:09.562418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:09.739224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9201b2-88d9-41c4-8263-39e42f751d9a service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:09.837769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:09.838827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:09.841124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:09.841124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:09.854057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:09.855956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:09.870626 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4152/8343] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:10.007493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c7201-e70d-4607-9e6d-6c87d24cb1a5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:10.024015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c7201-e70d-4607-9e6d-6c87d24cb1a5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:10.030740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:10.032087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:10.032556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:10.033008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:10.120880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:10.122476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:10.137626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4192/8344] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:13.973801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:13.975448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:14.078315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2db04-bb20-490d-a2f8-c51e22313c59 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:14.083418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:14.084069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:14.084346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:14.084616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] PATH_INFO: `/v3/users/2c778820854d4636ac34bd86ab7fbaff` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:14.085958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c778820854d4636ac34bd86ab7fbaff)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:14.087983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:14.139184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:14.151949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c778820854d4636ac34bd86ab7fbaff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.164749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c778820854d4636ac34bd86ab7fbaff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.176801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c778820854d4636ac34bd86ab7fbaff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.177885 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c778820854d4636ac34bd86ab7fbaff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.178572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c778820854d4636ac34bd86ab7fbaff'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.197132 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4153/8345] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:13 2022] DELETE /identity/v3/users/2c778820854d4636ac34bd86ab7fbaff => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:14.422990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:14.450961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ca95f-c9ab-4e50-9b0c-2c745651d91e service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:14.542527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:14.543203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:14.543517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:14.543783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] PATH_INFO: `/v3/projects/81468f9a5a5d4b63bf62e9fe4bd19b56` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:14.545615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] RBAC: Authorizing `identity:delete_project(project_id=81468f9a5a5d4b63bf62e9fe4bd19b56)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:14.548009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:14.633492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81468f9a5a5d4b63bf62e9fe4bd19b56', 'request_id': 'req-211872a6-4d20-475c-9e34-73ac39b7845e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.642609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81468f9a5a5d4b63bf62e9fe4bd19b56', 'request_id': 'req-211872a6-4d20-475c-9e34-73ac39b7845e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.657984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81468f9a5a5d4b63bf62e9fe4bd19b56', 'request_id': 'req-211872a6-4d20-475c-9e34-73ac39b7845e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.695966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] The token cache is being invalidate because project 81468f9a5a5d4b63bf62e9fe4bd19b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:14.696901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.698527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-211872a6-4d20-475c-9e34-73ac39b7845e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:14.702885 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4193/8346] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:14 2022] DELETE /identity/v3/projects/81468f9a5a5d4b63bf62e9fe4bd19b56 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:14.706092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:14.709893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:14.730173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30379f9a-cc33-4e33-ab61-8204db10d778 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:14.881637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:14.881929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:14.883018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:14.893843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:14.894631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:14.897891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:14.898281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] PATH_INFO: `/v3/users/2d96106fefe84b9ca41126ec77a25ef2` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:14.899731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d96106fefe84b9ca41126ec77a25ef2)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:14.904194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:14.907867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] MFA Rules not processed for user `342e60103c9b45bcb348b6f8bcd686ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:14.909146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:14.937225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:15.008470 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4194/8347] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:15.027150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d96106fefe84b9ca41126ec77a25ef2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.103908 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d96106fefe84b9ca41126ec77a25ef2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.125799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d96106fefe84b9ca41126ec77a25ef2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.130578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d96106fefe84b9ca41126ec77a25ef2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.131944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d96106fefe84b9ca41126ec77a25ef2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.154700 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4154/8348] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:14 2022] DELETE /identity/v3/users/2d96106fefe84b9ca41126ec77a25ef2 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:15.396398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:15.398354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:15.416485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04079399-c6af-49f0-94eb-9c20f135eb99 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:15.485557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:15.486679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:15.487068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:15.487587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] PATH_INFO: `/v3/projects/c409519bdbbf4783b5184cd196e79626` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:15.489018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c409519bdbbf4783b5184cd196e79626)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:15.491712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:15.558037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c409519bdbbf4783b5184cd196e79626', 'request_id': 'req-13726818-77ab-4d7b-8d61-bbaebf8b4664'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.565703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c409519bdbbf4783b5184cd196e79626', 'request_id': 'req-13726818-77ab-4d7b-8d61-bbaebf8b4664'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.580129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c409519bdbbf4783b5184cd196e79626', 'request_id': 'req-13726818-77ab-4d7b-8d61-bbaebf8b4664'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.611160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] The token cache is being invalidate because project c409519bdbbf4783b5184cd196e79626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:15.612286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.613997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-13726818-77ab-4d7b-8d61-bbaebf8b4664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:15.622020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4195/8349] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:15 2022] DELETE /identity/v3/projects/c409519bdbbf4783b5184cd196e79626 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:17.853925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:17.856333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:17.870668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9517e7f-85b3-4c34-94db-5f2861b8f68f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:17.904176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:17.939745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:17.940395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:17.940673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:17.941024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] PATH_INFO: `/v3/users/57eafbe4f20f4c40b007fb5478205a9f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:17.942170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57eafbe4f20f4c40b007fb5478205a9f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:17.944453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:18.002199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57eafbe4f20f4c40b007fb5478205a9f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:18.017444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57eafbe4f20f4c40b007fb5478205a9f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:18.028502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57eafbe4f20f4c40b007fb5478205a9f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:18.029119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57eafbe4f20f4c40b007fb5478205a9f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:18.029819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57eafbe4f20f4c40b007fb5478205a9f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:18.052234 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4155/8350] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:17 2022] DELETE /identity/v3/users/57eafbe4f20f4c40b007fb5478205a9f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:18.061899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:18.062357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:18.062471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:18.096208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:18.098530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:18.187369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:18.202420 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4196/8351] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:18.225948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:18.229555 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:18.372328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587c93b-ae4f-471a-873d-3068ab083521 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:18.381838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:18.383982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:18.384506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:18.384884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:18.417499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:18.429942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:18.459778 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4156/8352] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:19.140350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:19.141941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:19.161156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf05b2-799d-4152-aea8-6e888804ab00 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:19.166163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:19.167063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:19.167489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:19.167902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] PATH_INFO: `/v3/projects/93c7626dedfb4acbaec8d2627c570b80` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:19.170329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c7626dedfb4acbaec8d2627c570b80)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:19.172772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:19.180655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:19.182771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:19.197805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219d607-f8a4-4386-8d55-f2287a231ff7 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:19.214712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:19.215594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:19.216028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:19.216298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] PATH_INFO: `/v3/users/17e4d830f68543a6a5563ac020f20bd0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:19.217649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e4d830f68543a6a5563ac020f20bd0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:19.219805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:19.255289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c7626dedfb4acbaec8d2627c570b80', 'request_id': 'req-244893b7-4ac2-4f44-9c77-e792d3fa0161'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.264211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c7626dedfb4acbaec8d2627c570b80', 'request_id': 'req-244893b7-4ac2-4f44-9c77-e792d3fa0161'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.274952 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c7626dedfb4acbaec8d2627c570b80', 'request_id': 'req-244893b7-4ac2-4f44-9c77-e792d3fa0161'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.290756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] The token cache is being invalidate because project 93c7626dedfb4acbaec8d2627c570b80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:19.291146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.292037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.295007 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4197/8353] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:19 2022] DELETE /identity/v3/projects/93c7626dedfb4acbaec8d2627c570b80 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:19.299517 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:19.303364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:19.311407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e4d830f68543a6a5563ac020f20bd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.317226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:19.334282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e4d830f68543a6a5563ac020f20bd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.348471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e4d830f68543a6a5563ac020f20bd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.349336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e4d830f68543a6a5563ac020f20bd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.350276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d2a23dbb-1be2-40c1-99b2-19dbc4bc4060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e4d830f68543a6a5563ac020f20bd0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.368623 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4157/8354] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:19 2022] DELETE /identity/v3/users/17e4d830f68543a6a5563ac020f20bd0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:24:19.370467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:19.370826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:19.371290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:19.390413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244893b7-4ac2-4f44-9c77-e792d3fa0161 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:19.392048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:19.393295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:19.450107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:19.451052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:19.451331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:19.451607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] PATH_INFO: `/v3/users/ed9c5e9b5d3e4ceeb0bc456c1068742b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:19.452780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9c5e9b5d3e4ceeb0bc456c1068742b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:19.454784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:19.466460 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4158/8355] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:19.478599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:19.480427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:19.492451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:19.512756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9c5e9b5d3e4ceeb0bc456c1068742b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.513438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9c5e9b5d3e4ceeb0bc456c1068742b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.523355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9c5e9b5d3e4ceeb0bc456c1068742b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.524247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9c5e9b5d3e4ceeb0bc456c1068742b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.538434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9c5e9b5d3e4ceeb0bc456c1068742b'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:19.550519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bba475-84ae-466f-b4bc-f897b8fed9aa None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:19.553752 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4198/8356] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:19 2022] DELETE /identity/v3/users/ed9c5e9b5d3e4ceeb0bc456c1068742b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:24:19.554566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:19.555256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:19.555603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:19.555945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:19.558135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:19.573109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:19.574694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:19.586728 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4159/8357] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:24:20.349223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.350891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:20.434886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be21b-fa3d-4cfc-9b65-614c6d7acb9d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.440628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.441346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.441662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.441940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] PATH_INFO: `/v3/projects/c1e7eb5d14114976a662200156c13bc9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.443123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e7eb5d14114976a662200156c13bc9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.445206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.456951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.458839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:20.477552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccba508-4438-4c09-83d6-995ede0f9e9f service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.486189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.486883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.487157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.487399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] PATH_INFO: `/v3/projects/68203fde4361416081c319be14988e0d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.488655 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68203fde4361416081c319be14988e0d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.491022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.509970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e7eb5d14114976a662200156c13bc9', 'request_id': 'req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.518198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e7eb5d14114976a662200156c13bc9', 'request_id': 'req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.528264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e7eb5d14114976a662200156c13bc9', 'request_id': 'req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.540036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] The token cache is being invalidate because project c1e7eb5d14114976a662200156c13bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:20.540335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.540968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-5e9892a6-3487-4e63-95a0-4f1d7d9aa402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.542632 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4199/8358] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:20 2022] DELETE /identity/v3/projects/c1e7eb5d14114976a662200156c13bc9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:20.547431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e82e0-4afb-4c4c-a2f5-b3a6bf87f354 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.547728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e82e0-4afb-4c4c-a2f5-b3a6bf87f354 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.548004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e82e0-4afb-4c4c-a2f5-b3a6bf87f354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.549002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68203fde4361416081c319be14988e0d', 'request_id': 'req-5810ba52-05c8-4c07-b875-deff76efc5f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.556400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68203fde4361416081c319be14988e0d', 'request_id': 'req-5810ba52-05c8-4c07-b875-deff76efc5f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.562034 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68203fde4361416081c319be14988e0d', 'request_id': 'req-5810ba52-05c8-4c07-b875-deff76efc5f2'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.563111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:20.569000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1f9e82e0-4afb-4c4c-a2f5-b3a6bf87f354 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:20.570282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1f9e82e0-4afb-4c4c-a2f5-b3a6bf87f354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:20.574805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] The token cache is being invalidate because project 68203fde4361416081c319be14988e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:20.575489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.576976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5810ba52-05c8-4c07-b875-deff76efc5f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:20.579065 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4160/8359] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:20 2022] DELETE /identity/v3/projects/68203fde4361416081c319be14988e0d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:20.584269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcdfa6-ffe2-440c-8c9e-41858e101c5a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.584626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcdfa6-ffe2-440c-8c9e-41858e101c5a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.585029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcdfa6-ffe2-440c-8c9e-41858e101c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.604725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-81bcdfa6-ffe2-440c-8c9e-41858e101c5a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:20.605963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-81bcdfa6-ffe2-440c-8c9e-41858e101c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:20.607561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:20.636704 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4200/8360] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:24:20.645790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabd946-0d74-48ef-a8d7-63376e9865e8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.646105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabd946-0d74-48ef-a8d7-63376e9865e8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.646375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabd946-0d74-48ef-a8d7-63376e9865e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.666511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2eabd946-0d74-48ef-a8d7-63376e9865e8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:20.667710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2eabd946-0d74-48ef-a8d7-63376e9865e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:20.675250 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4161/8361] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:24:20.675663 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4201/8362] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:24:20.679775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcdfa6-ffe2-440c-8c9e-41858e101c5a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.683893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988ddda-f074-4cb5-9c27-e8d9c1eba7e5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.684172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988ddda-f074-4cb5-9c27-e8d9c1eba7e5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.684447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988ddda-f074-4cb5-9c27-e8d9c1eba7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.689660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcdfa6-ffe2-440c-8c9e-41858e101c5a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.694350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.694968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.695244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.695517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.696165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:24:20.696604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.698258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.704159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e988ddda-f074-4cb5-9c27-e8d9c1eba7e5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:20.705340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e988ddda-f074-4cb5-9c27-e8d9c1eba7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:20.710195 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4162/8363] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:24:20.714588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.716822 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4202/8364] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:20.720762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988ddda-f074-4cb5-9c27-e8d9c1eba7e5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.724434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89f770-6a61-48fa-86d0-a47fee1f62a5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.729189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.729941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.730245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.730454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988ddda-f074-4cb5-9c27-e8d9c1eba7e5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.730552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.731198 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.733090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.735286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.735869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.736145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.736417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.737064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:24:20.737363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.739042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.751649 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4203/8365] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:24:20.760402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.764033 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4163/8366] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:20.768157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.772601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d62098-efa3-45be-a10e-c12635d9ffb3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.832393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21813c8e-66f0-4ab6-a918-e2b96069f36c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.837506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.837972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.838424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.838593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.838712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.838864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.838957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.839131 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.839589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.839761 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.841715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.841816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.865892 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4164/8367] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:20.870325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.876515 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4204/8368] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:20.880771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.881325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c5b74-256a-4fb3-bb43-b1651bc1e519 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.932921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:20.948097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.948935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.949369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.949861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.950570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.952422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.954968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98947cce-a82f-4c2b-82de-b6a8023c8aeb admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.960208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.960946 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.961145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.961409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.962056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.963990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.964615 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4165/8369] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:24:20.968314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:20.979091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bf090-f1ee-4d88-bb6c-7f21433bbe77 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:20.984354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:20.984963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:20.985301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:20.985612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:20.986315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:20.988039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:20.997494 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4205/8370] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:20 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:20.999130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.003665 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4166/8371] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:24:21.004632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.013530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fd8d5-2b11-4811-8521-c3d876fddaed admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.014844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0636c87-a281-4ab4-a3ce-32207e5da335 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.018402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.019080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.019353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.019621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.019878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bec41-58b3-4dbe-ac8d-1fc28b698a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.020259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.020527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bec41-58b3-4dbe-ac8d-1fc28b698a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.020808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bec41-58b3-4dbe-ac8d-1fc28b698a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.021081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bec41-58b3-4dbe-ac8d-1fc28b698a8c admin admin] PATH_INFO: `/v3/projects/caee650e216e4b80b4440361a57d2f68/users/01a32874941d4a2982a1e7c3aa5ca26a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.022230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.035841 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4206/8372] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:21.037323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.040733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bec41-58b3-4dbe-ac8d-1fc28b698a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=caee650e216e4b80b4440361a57d2f68, user_id=01a32874941d4a2982a1e7c3aa5ca26a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.043958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bec41-58b3-4dbe-ac8d-1fc28b698a8c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.049340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43281ff3-b0bf-42ea-addf-1c3fce3d2bb3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.055977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.056588 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.056894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.057132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.057850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.059453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.063242 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4167/8373] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:21 2022] PUT /identity/v3/projects/caee650e216e4b80b4440361a57d2f68/users/01a32874941d4a2982a1e7c3aa5ca26a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:21.071932 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4207/8374] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:21.073775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.074123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.074447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.078202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.086317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:21.090671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794c4be-1e4c-4090-a72c-e401fb7204d0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.110341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] MFA Rules not processed for user `01a32874941d4a2982a1e7c3aa5ca26a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:21.111561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:21.157842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fed74-a528-44d2-a1ec-6580309f309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.158600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fed74-a528-44d2-a1ec-6580309f309d admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.158883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fed74-a528-44d2-a1ec-6580309f309d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.159158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fed74-a528-44d2-a1ec-6580309f309d admin admin] PATH_INFO: `/v3/projects/b96407f55e0247d3b54586a85ab41a37/users/52ab75708ff84cd891ab516002b10ee8/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.176558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fed74-a528-44d2-a1ec-6580309f309d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96407f55e0247d3b54586a85ab41a37, user_id=52ab75708ff84cd891ab516002b10ee8, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.180793 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4168/8375] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:24:21.180914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fed74-a528-44d2-a1ec-6580309f309d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.188027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.189144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:21.200962 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4208/8376] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:21 2022] PUT /identity/v3/projects/b96407f55e0247d3b54586a85ab41a37/users/52ab75708ff84cd891ab516002b10ee8/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:21.211155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ccfd0-1b31-4bbc-b513-c60b7f6b452a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.211441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ccfd0-1b31-4bbc-b513-c60b7f6b452a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.211718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ccfd0-1b31-4bbc-b513-c60b7f6b452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.244357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0b2ccfd0-1b31-4bbc-b513-c60b7f6b452a None None] MFA Rules not processed for user `52ab75708ff84cd891ab516002b10ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:21.246617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0b2ccfd0-1b31-4bbc-b513-c60b7f6b452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:21.252597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5222f-e86a-4982-bd25-50c80768e8f2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.306455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.306976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.307255 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.307664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.316667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:21.317487 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4209/8377] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:21.321936 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ccfd0-1b31-4bbc-b513-c60b7f6b452a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.374873 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.376355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.386540 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4169/8378] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:24:21.404612 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ccfd0-1b31-4bbc-b513-c60b7f6b452a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.410616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.411244 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.411496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.411771 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.412411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.415293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.445970 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4210/8379] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:21 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:21.450512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.465205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4ad8c-d775-4492-a86a-ea1db69eb068 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.538998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.539791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.540069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.540334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.540964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.543164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.567914 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4170/8380] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:21 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:21.572063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.585391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:21.592843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6618d-7e4d-4e80-935f-6e6233ad43a0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.598392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.599123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.599492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.599913 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.600787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.602546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.618064 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4211/8381] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:21.619696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.631949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9231470-2fc6-4a56-a4bd-0828bb6e491d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.637105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.637797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.638099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.638411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] PATH_INFO: `/v3/projects/24551168ea74478bb1e36f8c82a47b6e/users/50604895d976485c8ebe26d1bb8a87ad/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.671980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=24551168ea74478bb1e36f8c82a47b6e, user_id=50604895d976485c8ebe26d1bb8a87ad, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.678057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.704596 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4171/8382] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:21 2022] PUT /identity/v3/projects/24551168ea74478bb1e36f8c82a47b6e/users/50604895d976485c8ebe26d1bb8a87ad/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:21.705746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.732279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd0390-e9d1-442e-9f2e-1d5e85d94c96 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.805376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b3097-1852-4013-8137-f34c03998ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.805376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b3097-1852-4013-8137-f34c03998ecc admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.805376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b3097-1852-4013-8137-f34c03998ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.805376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b3097-1852-4013-8137-f34c03998ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.811485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b3097-1852-4013-8137-f34c03998ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.811485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b3097-1852-4013-8137-f34c03998ecc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.811485 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4212/8383] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:24:21.817423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:21.836256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17914f6-1fe5-43b2-aa59-28941b0080ab admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:21.841946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:21.842631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.842910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.843593 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] PATH_INFO: `/v3/projects/24551168ea74478bb1e36f8c82a47b6e/users/50604895d976485c8ebe26d1bb8a87ad/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.845875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24551168ea74478bb1e36f8c82a47b6e, user_id=50604895d976485c8ebe26d1bb8a87ad, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:21.849162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:21.867593 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4172/8384] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:21 2022] PUT /identity/v3/projects/24551168ea74478bb1e36f8c82a47b6e/users/50604895d976485c8ebe26d1bb8a87ad/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:21.880012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:21.880417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:21.880697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:21.918898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] MFA Rules not processed for user `50604895d976485c8ebe26d1bb8a87ad`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:21.920314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:22.012892 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4213/8385] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:22.021142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:22.023022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:22.102089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89ce0-ae59-4de2-8b12-a9a5a7c2c124 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:22.107205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:22.108010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:22.108475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:22.108919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:22.122571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:22.125225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:22.137186 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4173/8386] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:22.194881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:22.197748 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:22.221893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafedd3-0431-41a0-885c-d446708ea8b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:22.226851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e213c7-54b4-4b9c-8555-0f4245f1a9cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:22.227645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e213c7-54b4-4b9c-8555-0f4245f1a9cc service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:22.228071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e213c7-54b4-4b9c-8555-0f4245f1a9cc service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:22.228400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e213c7-54b4-4b9c-8555-0f4245f1a9cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:22.305285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e213c7-54b4-4b9c-8555-0f4245f1a9cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:22.306630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e213c7-54b4-4b9c-8555-0f4245f1a9cc service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:22.317408 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4214/8387] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:26.292168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:26.299069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:26.366888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676914e-95e5-47fa-9ef8-3cb15db3d9e6 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:26.373087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:26.373905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:26.374185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:26.374568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] PATH_INFO: `/v3/users/52ab75708ff84cd891ab516002b10ee8` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:26.375786 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ab75708ff84cd891ab516002b10ee8)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:26.377867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:26.437619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ab75708ff84cd891ab516002b10ee8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:26.453566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ab75708ff84cd891ab516002b10ee8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:26.466454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ab75708ff84cd891ab516002b10ee8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:26.467333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ab75708ff84cd891ab516002b10ee8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:26.468098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ab75708ff84cd891ab516002b10ee8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:26.483413 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4174/8388] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:26 2022] DELETE /identity/v3/users/52ab75708ff84cd891ab516002b10ee8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:24:26.489903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:26.490447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:26.491204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:26.511696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:26.513015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:26.518447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:26.609973 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4215/8389] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:26.633706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:26.635852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:26.736477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876e90d-21ad-4391-b72a-a8dc7d653f64 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:26.742716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:26.743264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:26.743540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:26.743805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:26.762382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:26.763874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:26.778035 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4175/8390] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:27.086104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:27.089714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:27.107905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1ff3e-99c6-472a-a9ed-b2a8bca60088 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:27.120279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:27.121191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:27.121490 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:27.121781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] PATH_INFO: `/v3/projects/b96407f55e0247d3b54586a85ab41a37` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:27.123119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96407f55e0247d3b54586a85ab41a37)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:27.125087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:27.179781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96407f55e0247d3b54586a85ab41a37', 'request_id': 'req-ea28e938-c8d2-49ba-b373-e8c128ea010f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.186444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96407f55e0247d3b54586a85ab41a37', 'request_id': 'req-ea28e938-c8d2-49ba-b373-e8c128ea010f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.199370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96407f55e0247d3b54586a85ab41a37', 'request_id': 'req-ea28e938-c8d2-49ba-b373-e8c128ea010f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.212731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] The token cache is being invalidate because project b96407f55e0247d3b54586a85ab41a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:27.212985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.213673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.215531 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4216/8391] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:27 2022] DELETE /identity/v3/projects/b96407f55e0247d3b54586a85ab41a37 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:27.219370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:27.237291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcba2d6-92ac-47de-931a-a8d79cc1e4b3 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:27.277406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:27.323411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:27.324537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:27.324822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:27.325439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] PATH_INFO: `/v3/users/50604895d976485c8ebe26d1bb8a87ad` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:27.326474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=50604895d976485c8ebe26d1bb8a87ad)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:27.329263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:27.403714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50604895d976485c8ebe26d1bb8a87ad'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.421572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50604895d976485c8ebe26d1bb8a87ad'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.436054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50604895d976485c8ebe26d1bb8a87ad'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.436738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50604895d976485c8ebe26d1bb8a87ad'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.437374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5a983465-84fb-4efb-8b9e-debc76b543ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50604895d976485c8ebe26d1bb8a87ad'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:27.463357 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4176/8392] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:27 2022] DELETE /identity/v3/users/50604895d976485c8ebe26d1bb8a87ad => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:27.886013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:27.888642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:27.903289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28e938-c8d2-49ba-b373-e8c128ea010f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:27.968803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:27.969439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:27.969751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:27.970020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] PATH_INFO: `/v3/projects/24551168ea74478bb1e36f8c82a47b6e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:27.971306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24551168ea74478bb1e36f8c82a47b6e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:27.973371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.015306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:28.026012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24551168ea74478bb1e36f8c82a47b6e', 'request_id': 'req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:28.032565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24551168ea74478bb1e36f8c82a47b6e', 'request_id': 'req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:28.041994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24551168ea74478bb1e36f8c82a47b6e', 'request_id': 'req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:28.052675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] The token cache is being invalidate because project 24551168ea74478bb1e36f8c82a47b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:28.052969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:28.053654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-4ebe3324-0a0b-40be-9a97-4a57a140b9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:28.055400 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4217/8393] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:27 2022] DELETE /identity/v3/projects/24551168ea74478bb1e36f8c82a47b6e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:28.060863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c0177-bb9d-47e5-b8bd-0c5bd1c0d25b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.061449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c0177-bb9d-47e5-b8bd-0c5bd1c0d25b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.064446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c0177-bb9d-47e5-b8bd-0c5bd1c0d25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.089952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-050c0177-bb9d-47e5-b8bd-0c5bd1c0d25b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:28.091176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-050c0177-bb9d-47e5-b8bd-0c5bd1c0d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:28.154077 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4177/8394] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:24:28.161979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b06844-1eb4-491c-a158-7eecb82af2be None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.162261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b06844-1eb4-491c-a158-7eecb82af2be None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.162534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b06844-1eb4-491c-a158-7eecb82af2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.182955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a4b06844-1eb4-491c-a158-7eecb82af2be None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:28.184162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a4b06844-1eb4-491c-a158-7eecb82af2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:28.192522 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4218/8395] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:28.196914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c0177-bb9d-47e5-b8bd-0c5bd1c0d25b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.208117 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c0177-bb9d-47e5-b8bd-0c5bd1c0d25b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.213796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.214414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.214694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.214968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.215624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:24:28.215930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.217602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.230079 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4178/8396] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:24:28.234751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b06844-1eb4-491c-a158-7eecb82af2be None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.244602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b06844-1eb4-491c-a158-7eecb82af2be None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.249315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.249993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.250271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.250540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.251290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.253311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.282717 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4219/8397] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:28 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:28.286834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.297672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4546c-4dc3-4f4f-9472-fbaa39965272 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.334671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:28.363428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.364035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.364311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.364612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.365346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.367660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.388926 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4179/8398] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:28 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:28.393334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.404333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ae02-c87c-4c0e-8ce6-fb0b0d14c2d5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.409158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.409910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.410193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.410464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.411108 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.412725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.423540 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4220/8399] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:24:28.427726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.438227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72995a-c2a5-4e86-a411-81f37b9c6549 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.443346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edfff4-ff61-4c09-ae8a-89ecea72b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.443958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edfff4-ff61-4c09-ae8a-89ecea72b33a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.444236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edfff4-ff61-4c09-ae8a-89ecea72b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.444626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edfff4-ff61-4c09-ae8a-89ecea72b33a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.445283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edfff4-ff61-4c09-ae8a-89ecea72b33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.446936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edfff4-ff61-4c09-ae8a-89ecea72b33a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.458269 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4180/8400] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:28.462513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.473326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818bda-ae91-4797-bb09-873b7f5dd4db admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.478396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.479051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.479327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.480083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] PATH_INFO: `/v3/projects/4f4b6b67b148445494265e6ad8651a95/users/1b3966e053ce4c028783294c23cc7560/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.495111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4b6b67b148445494265e6ad8651a95, user_id=1b3966e053ce4c028783294c23cc7560, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.497848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.511823 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4221/8401] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:28 2022] PUT /identity/v3/projects/4f4b6b67b148445494265e6ad8651a95/users/1b3966e053ce4c028783294c23cc7560/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:28.522711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56c777-985b-4d5a-96df-ce5c3033a64c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.523233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56c777-985b-4d5a-96df-ce5c3033a64c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.523509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56c777-985b-4d5a-96df-ce5c3033a64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.554462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2c56c777-985b-4d5a-96df-ce5c3033a64c None None] MFA Rules not processed for user `1b3966e053ce4c028783294c23cc7560`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:28.555723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2c56c777-985b-4d5a-96df-ce5c3033a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:28.638132 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4181/8402] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:28.639672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.725240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d384c3d-124e-45a2-9566-f45191fdf386 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.732720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.734251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.735821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.736168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.737629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.741823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.779835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4222/8403] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:28 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:28.784011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56c777-985b-4d5a-96df-ce5c3033a64c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.795234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56c777-985b-4d5a-96df-ce5c3033a64c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.861498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.862265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.862684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.863014 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.863831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.866661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.897058 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4182/8404] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:28 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:28.898321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.909964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2a7f8-0647-4a5d-83cd-9895613a641a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.914521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.915144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.915398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.915664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.916311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.917965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.929139 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4223/8405] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:28.933610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.944522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47b2a3-19d5-46b8-b1d1-67fe13b99ba8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:28.951535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:28.952784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:28.953068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:28.953338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] PATH_INFO: `/v3/projects/2f2b8282d6b2472ea4ec24239b12f13b/users/72d89be398184dc781db2f3b8714b322/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:28.970914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2b8282d6b2472ea4ec24239b12f13b, user_id=72d89be398184dc781db2f3b8714b322, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:28.975029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:28.998220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:28.998659 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4183/8406] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:28 2022] PUT /identity/v3/projects/2f2b8282d6b2472ea4ec24239b12f13b/users/72d89be398184dc781db2f3b8714b322/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:29.009899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc7797-6183-473d-b960-084c7e375594 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:29.050879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:29.083271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02094feb-6dbf-45af-9410-14c0899fd95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:29.083970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094feb-6dbf-45af-9410-14c0899fd95f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:29.084317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094feb-6dbf-45af-9410-14c0899fd95f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:29.084658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094feb-6dbf-45af-9410-14c0899fd95f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:29.085324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02094feb-6dbf-45af-9410-14c0899fd95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:29.086961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02094feb-6dbf-45af-9410-14c0899fd95f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:29.101280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:29.101697 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4224/8407] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:29.112064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507f74b-80b6-441d-9f0a-1f5d2a7b5513 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:29.117368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:29.118181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:29.118543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:29.118725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] PATH_INFO: `/v3/projects/2f2b8282d6b2472ea4ec24239b12f13b/users/72d89be398184dc781db2f3b8714b322/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:29.120670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2b8282d6b2472ea4ec24239b12f13b, user_id=72d89be398184dc781db2f3b8714b322, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:29.123406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:29.126870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:29.138627 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4184/8408] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:29 2022] PUT /identity/v3/projects/2f2b8282d6b2472ea4ec24239b12f13b/users/72d89be398184dc781db2f3b8714b322/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:24:29.150049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:29.150347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:29.150606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:29.182363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] MFA Rules not processed for user `72d89be398184dc781db2f3b8714b322`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:29.183645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:29.259595 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4225/8409] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:29 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:29.270989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:29.273328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:29.348884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb35ced-586b-4877-92a4-c5b5d0f2872e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:29.354354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:29.355124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:29.355403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:29.355676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:29.366822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:29.368426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:29.380988 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4185/8410] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:29.439030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:29.441195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:29.457406 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df108cf-d379-4c11-b37a-f70cc863ca32 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:29.461369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae4851-2569-498d-a146-8f61d14ed57e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:29.462147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae4851-2569-498d-a146-8f61d14ed57e service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:29.462970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae4851-2569-498d-a146-8f61d14ed57e service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:29.462970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae4851-2569-498d-a146-8f61d14ed57e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:29.530140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae4851-2569-498d-a146-8f61d14ed57e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:29.531838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae4851-2569-498d-a146-8f61d14ed57e service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:29.541592 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4226/8411] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:33.305518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:33.370884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44477b-7a0b-4e74-86d7-38737ef9e1ad service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:33.376981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:33.377654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:33.377929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:33.378226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] PATH_INFO: `/v3/users/1b3966e053ce4c028783294c23cc7560` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:33.379383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3966e053ce4c028783294c23cc7560)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:33.381308 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:33.429965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3966e053ce4c028783294c23cc7560'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:33.442895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3966e053ce4c028783294c23cc7560'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:33.451317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3966e053ce4c028783294c23cc7560'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:33.452040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3966e053ce4c028783294c23cc7560'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:33.452725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3966e053ce4c028783294c23cc7560'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:33.465838 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4186/8412] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:33 2022] DELETE /identity/v3/users/1b3966e053ce4c028783294c23cc7560 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:33.470408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:33.471059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:33.471203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:33.491013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:33.492229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:33.542004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=100 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:33.559439 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4227/8413] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:33.569625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:33.571168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:33.591310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:33.638481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2af801-ebdc-4972-88a4-587f7480822a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:33.642180 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:33.642801 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:33.643079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:33.643349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:33.654209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:33.655787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:33.673714 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4187/8414] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:34.078655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:34.086604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:34.116027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53bc63-7926-43b2-b49c-c94d2e941bfb None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:34.123024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:34.124760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:34.125359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:34.125760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] PATH_INFO: `/v3/projects/4f4b6b67b148445494265e6ad8651a95` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:34.127638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4b6b67b148445494265e6ad8651a95)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:34.131719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:34.194878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4b6b67b148445494265e6ad8651a95', 'request_id': 'req-70885a98-8df0-49bb-beaa-ae80ff730cf7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.201971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4b6b67b148445494265e6ad8651a95', 'request_id': 'req-70885a98-8df0-49bb-beaa-ae80ff730cf7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.210825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4b6b67b148445494265e6ad8651a95', 'request_id': 'req-70885a98-8df0-49bb-beaa-ae80ff730cf7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.221911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] The token cache is being invalidate because project 4f4b6b67b148445494265e6ad8651a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:34.222243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.222924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.224549 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4228/8415] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:34 2022] DELETE /identity/v3/projects/4f4b6b67b148445494265e6ad8651a95 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:34.228462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:34.238823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47941eaf-48b5-483a-8f58-72b04808da0c service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:34.297610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:34.298245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:34.298546 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:34.298815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] PATH_INFO: `/v3/users/72d89be398184dc781db2f3b8714b322` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:34.299952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d89be398184dc781db2f3b8714b322)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:34.301969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:34.349891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d89be398184dc781db2f3b8714b322'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.362996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d89be398184dc781db2f3b8714b322'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.371256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d89be398184dc781db2f3b8714b322'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.371996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d89be398184dc781db2f3b8714b322'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.372667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-60558f57-c85b-4668-bb16-9694a6b16b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d89be398184dc781db2f3b8714b322'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.387847 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4188/8416] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:34 2022] DELETE /identity/v3/users/72d89be398184dc781db2f3b8714b322 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:34.745796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:34.747835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:34.770167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70885a98-8df0-49bb-beaa-ae80ff730cf7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:34.864256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:34.864991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:34.865274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:34.866098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] PATH_INFO: `/v3/projects/2f2b8282d6b2472ea4ec24239b12f13b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:34.867481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f2b8282d6b2472ea4ec24239b12f13b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:34.870456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:34.928742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f2b8282d6b2472ea4ec24239b12f13b', 'request_id': 'req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.936039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f2b8282d6b2472ea4ec24239b12f13b', 'request_id': 'req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.944811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f2b8282d6b2472ea4ec24239b12f13b', 'request_id': 'req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.972560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] The token cache is being invalidate because project 2f2b8282d6b2472ea4ec24239b12f13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:34.972905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.973542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1614a13a-e9bc-405e-933b-1c0d6e6e5a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:34.978857 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4229/8417] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:34 2022] DELETE /identity/v3/projects/2f2b8282d6b2472ea4ec24239b12f13b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:34.983592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c360-7012-4a01-b3e4-ed58dd85dd76 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:34.983874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c360-7012-4a01-b3e4-ed58dd85dd76 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:34.984168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c360-7012-4a01-b3e4-ed58dd85dd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.010429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e5b2c360-7012-4a01-b3e4-ed58dd85dd76 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:35.011881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e5b2c360-7012-4a01-b3e4-ed58dd85dd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:35.060841 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:35.105713 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4189/8418] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:35.111768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e425474-e26b-44ad-86fb-4c87be371e5d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.112288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e425474-e26b-44ad-86fb-4c87be371e5d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.112914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e425474-e26b-44ad-86fb-4c87be371e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.183935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3e425474-e26b-44ad-86fb-4c87be371e5d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:35.185291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3e425474-e26b-44ad-86fb-4c87be371e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:35.216343 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4230/8419] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:35.217018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c360-7012-4a01-b3e4-ed58dd85dd76 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.227337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c360-7012-4a01-b3e4-ed58dd85dd76 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:35.238815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:35.239466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.241125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.241537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.242213 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:24:35.242607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:35.244595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:35.268743 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4190/8420] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:24:35.270014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e425474-e26b-44ad-86fb-4c87be371e5d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.282551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:35.287342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e425474-e26b-44ad-86fb-4c87be371e5d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:35.294479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:35.295534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.296040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.296886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.298878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:35.301149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:35.341901 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4231/8421] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:35 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:35.345936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.369550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5dc29-b691-4503-a35b-41994bf37c08 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:35.460299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:35.461397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.461799 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.462217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.463023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:35.465671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:35.501528 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4191/8422] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:35 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:35.503821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.525413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31d897-f9fc-4669-ac3d-18ef6d978ea2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:35.539191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:35.540298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.541028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.541442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.542565 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:35.546440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:35.567190 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4232/8423] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:35.571416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.585893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ec5ff-8fee-4308-8a53-b4fd168da366 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:35.593539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb0ae5-56a7-4155-b7b7-9f67b501d816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:35.594223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb0ae5-56a7-4155-b7b7-9f67b501d816 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.594611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb0ae5-56a7-4155-b7b7-9f67b501d816 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.594972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb0ae5-56a7-4155-b7b7-9f67b501d816 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.596058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb0ae5-56a7-4155-b7b7-9f67b501d816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:35.599607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb0ae5-56a7-4155-b7b7-9f67b501d816 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:35.625505 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4192/8424] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:24:35.628726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.640366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b379d62-9015-4a61-b346-85f2902f0696 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:35.645790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:35.647395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.647873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.648279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] PATH_INFO: `/v3/projects/e2aae04e106543b5a0b087dbd0aa4782/users/b45ba1feae9840ebb52de3c701c6d784/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.666630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2aae04e106543b5a0b087dbd0aa4782, user_id=b45ba1feae9840ebb52de3c701c6d784, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:35.669621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:35.705741 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4233/8425] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:35 2022] PUT /identity/v3/projects/e2aae04e106543b5a0b087dbd0aa4782/users/b45ba1feae9840ebb52de3c701c6d784/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:35.716639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c27aa-e9fb-4c53-91a0-2408071b3d47 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.717250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c27aa-e9fb-4c53-91a0-2408071b3d47 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.717787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c27aa-e9fb-4c53-91a0-2408071b3d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.758678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2f6c27aa-e9fb-4c53-91a0-2408071b3d47 None None] MFA Rules not processed for user `b45ba1feae9840ebb52de3c701c6d784`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:35.760056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f6c27aa-e9fb-4c53-91a0-2408071b3d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:35.833499 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4193/8426] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:35.837668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.926837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f7b2d-3887-4181-94fc-23e29a8cc5e4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:35.931930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:35.932785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:35.932907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:35.933092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:35.933783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:35.935858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:35.972004 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4234/8427] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:35 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:35.976232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c27aa-e9fb-4c53-91a0-2408071b3d47 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:35.988673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c27aa-e9fb-4c53-91a0-2408071b3d47 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.021615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:36.058963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.059699 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.061346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.061346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.061346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:36.063540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:36.095672 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4194/8428] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:35 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:36.099921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:36.114450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626d182-4f77-4b2c-9dcc-0646634a423a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.119711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.120339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.120614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.120908 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.125495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:36.125495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:36.135963 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4235/8429] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:36.139733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:36.156725 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9834d91-8c6f-4ae6-8adf-b5953864ebc4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.167425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c5b54-438e-4e62-b75c-949d0ce89e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.168149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c5b54-438e-4e62-b75c-949d0ce89e57 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.168420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c5b54-438e-4e62-b75c-949d0ce89e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.168688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c5b54-438e-4e62-b75c-949d0ce89e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.177861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c5b54-438e-4e62-b75c-949d0ce89e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:36.179526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c5b54-438e-4e62-b75c-949d0ce89e57 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:36.233576 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4195/8430] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:36.234000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:36.246229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6ceb0-ca36-40e3-80c5-67058f92a0ce admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.254377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.255114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.255397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.255670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] PATH_INFO: `/v3/projects/fe46950ac4d445ff81b0a3f731062d93/users/10825b84db3c438885aafab2a0222a0a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.276630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe46950ac4d445ff81b0a3f731062d93, user_id=10825b84db3c438885aafab2a0222a0a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:36.279842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:36.341965 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4236/8431] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:36 2022] PUT /identity/v3/projects/fe46950ac4d445ff81b0a3f731062d93/users/10825b84db3c438885aafab2a0222a0a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:36.440672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7c9eb-5341-42d0-ae23-5f25784d7fe6 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.441156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7c9eb-5341-42d0-ae23-5f25784d7fe6 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.441360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7c9eb-5341-42d0-ae23-5f25784d7fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.474263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-07c7c9eb-5341-42d0-ae23-5f25784d7fe6 None None] MFA Rules not processed for user `10825b84db3c438885aafab2a0222a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:36.475898 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-07c7c9eb-5341-42d0-ae23-5f25784d7fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:36.563262 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4196/8432] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:36.564512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:36.647439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:36.655977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae84c-e8ff-4555-8854-36162fff450e admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.666783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.672806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.673358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.673705 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.675652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:36.678087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:36.722766 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4237/8433] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:36 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:36.727063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7c9eb-5341-42d0-ae23-5f25784d7fe6 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:36.744161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7c9eb-5341-42d0-ae23-5f25784d7fe6 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.832054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.838535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.846083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.846545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.847328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:36.849256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:36.883580 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4197/8434] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:36 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:36.888419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:36.915839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643e16-7588-49ae-9fca-4d8d5f824572 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.927163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.928260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.928691 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.929112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:36.929945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:36.931740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:36.947338 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4238/8435] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:36.951292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:36.965163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148dd5e-779e-4bde-86bd-4357554f4d81 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:36.979775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:36.980469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:36.980745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:36.981013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] PATH_INFO: `/v3/projects/47bb8b7c08dc4df99eaa880ccc769d98/users/bd46e53977ed4c56946aa620410ff902/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:37.014892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bb8b7c08dc4df99eaa880ccc769d98, user_id=bd46e53977ed4c56946aa620410ff902, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:37.017741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:37.034310 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4198/8436] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:36 2022] PUT /identity/v3/projects/47bb8b7c08dc4df99eaa880ccc769d98/users/bd46e53977ed4c56946aa620410ff902/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:37.039733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:37.050605 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa18fb-857b-4b53-b44a-cab36653b257 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:37.131294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a2e90-178e-45be-b06b-88e7ee023fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:37.132225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a2e90-178e-45be-b06b-88e7ee023fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:37.132571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a2e90-178e-45be-b06b-88e7ee023fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:37.132918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a2e90-178e-45be-b06b-88e7ee023fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:37.133676 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a2e90-178e-45be-b06b-88e7ee023fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:37.135371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a2e90-178e-45be-b06b-88e7ee023fe1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:37.147465 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4239/8437] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:37.151429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:37.164979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68403bb9-bdcb-4e5d-b8a5-31368775edf6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:37.170449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:37.171597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:37.172017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:37.172483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] PATH_INFO: `/v3/projects/47bb8b7c08dc4df99eaa880ccc769d98/users/bd46e53977ed4c56946aa620410ff902/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:37.174708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bb8b7c08dc4df99eaa880ccc769d98, user_id=bd46e53977ed4c56946aa620410ff902, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:37.178998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:37.182195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:37.205515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4199/8438] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:37 2022] PUT /identity/v3/projects/47bb8b7c08dc4df99eaa880ccc769d98/users/bd46e53977ed4c56946aa620410ff902/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:37.207828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfed7-91d1-4424-86db-9b42e98b0c56 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:37.208195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfed7-91d1-4424-86db-9b42e98b0c56 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:37.209737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfed7-91d1-4424-86db-9b42e98b0c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:37.243178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-441cfed7-91d1-4424-86db-9b42e98b0c56 None None] MFA Rules not processed for user `bd46e53977ed4c56946aa620410ff902`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:37.244428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-441cfed7-91d1-4424-86db-9b42e98b0c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:37.321399 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4240/8439] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:37.333011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:37.334821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:37.418413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25cc0-d418-40c1-806a-8101e537929e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:37.423370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37e32b-dc85-4dd5-8c15-d2ecef619db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:37.424105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37e32b-dc85-4dd5-8c15-d2ecef619db8 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:37.424387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37e32b-dc85-4dd5-8c15-d2ecef619db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:37.424658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37e32b-dc85-4dd5-8c15-d2ecef619db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:37.434974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37e32b-dc85-4dd5-8c15-d2ecef619db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:37.436449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37e32b-dc85-4dd5-8c15-d2ecef619db8 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:37.447390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4200/8440] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:37.501591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cfed7-91d1-4424-86db-9b42e98b0c56 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:37.516236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cfed7-91d1-4424-86db-9b42e98b0c56 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:37.525431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:37.526078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:37.526353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:37.526622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:37.585902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:37.598388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:37.600074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:37.611014 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4241/8441] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:40.117556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759551-aef0-4fdb-a008-67a9a1336794 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:40.118574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759551-aef0-4fdb-a008-67a9a1336794 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:40.118922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759551-aef0-4fdb-a008-67a9a1336794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:40.138774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-97759551-aef0-4fdb-a008-67a9a1336794 None None] MFA Rules not processed for user `342e60103c9b45bcb348b6f8bcd686ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:40.140199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-97759551-aef0-4fdb-a008-67a9a1336794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:40.208218 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4201/8442] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:40.721566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:40.791315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76180c0-60a5-4520-88e2-69fcd25347c0 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:40.798224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:40.801599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:40.801599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:40.801599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] PATH_INFO: `/v3/users/b45ba1feae9840ebb52de3c701c6d784` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:40.802633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b45ba1feae9840ebb52de3c701c6d784)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:40.804576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:40.873569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b45ba1feae9840ebb52de3c701c6d784'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:40.874434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b45ba1feae9840ebb52de3c701c6d784'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:40.884914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b45ba1feae9840ebb52de3c701c6d784'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:40.885765 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b45ba1feae9840ebb52de3c701c6d784'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:40.909153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b45ba1feae9840ebb52de3c701c6d784'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:40.924963 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4242/8443] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:40 2022] DELETE /identity/v3/users/b45ba1feae9840ebb52de3c701c6d784 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:40.932056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:40.932056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:40.932056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:40.962139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:40.963881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:41.076558 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4202/8444] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:41.087456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:41.089078 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:41.190157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf15ace-4ec2-4183-9b7c-896bd88e8d11 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:41.195726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:41.196354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:41.196626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:41.196891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:41.208294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:41.209847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:41.254741 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4243/8445] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:41.606658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:41.608727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:41.634420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0a017-7375-4c59-9508-ff03c8625825 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:41.648622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:41.650077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:41.651244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:41.651730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] PATH_INFO: `/v3/projects/e2aae04e106543b5a0b087dbd0aa4782` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:41.653547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2aae04e106543b5a0b087dbd0aa4782)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:41.658470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:41.740256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:41.759425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2aae04e106543b5a0b087dbd0aa4782', 'request_id': 'req-26b878e3-6f63-4c82-88ce-28a26a077ce3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:41.773031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2aae04e106543b5a0b087dbd0aa4782', 'request_id': 'req-26b878e3-6f63-4c82-88ce-28a26a077ce3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:41.782713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2aae04e106543b5a0b087dbd0aa4782', 'request_id': 'req-26b878e3-6f63-4c82-88ce-28a26a077ce3'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:41.800716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] The token cache is being invalidate because project e2aae04e106543b5a0b087dbd0aa4782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:41.801332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:41.802229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:41.807270 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4203/8446] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:41 2022] DELETE /identity/v3/projects/e2aae04e106543b5a0b087dbd0aa4782 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:41.808444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:41.821305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b65ec-9d86-451e-9c94-2ee7a8c451dc service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:41.840251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:41.941329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:41.942256 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:41.942710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:41.943519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] PATH_INFO: `/v3/users/10825b84db3c438885aafab2a0222a0a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:41.948698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10825b84db3c438885aafab2a0222a0a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:41.951155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:42.026003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10825b84db3c438885aafab2a0222a0a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:42.027122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10825b84db3c438885aafab2a0222a0a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:42.063587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10825b84db3c438885aafab2a0222a0a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:42.064446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10825b84db3c438885aafab2a0222a0a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:42.080883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10825b84db3c438885aafab2a0222a0a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:42.102796 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4244/8447] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:41 2022] DELETE /identity/v3/users/10825b84db3c438885aafab2a0222a0a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:43.076273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:43.078319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:43.109742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b878e3-6f63-4c82-88ce-28a26a077ce3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:43.206846 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:43.207661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:43.207944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:43.208217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] PATH_INFO: `/v3/projects/fe46950ac4d445ff81b0a3f731062d93` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:43.210037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe46950ac4d445ff81b0a3f731062d93)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:43.212008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:43.298839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe46950ac4d445ff81b0a3f731062d93', 'request_id': 'req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.317372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe46950ac4d445ff81b0a3f731062d93', 'request_id': 'req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.324328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe46950ac4d445ff81b0a3f731062d93', 'request_id': 'req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.338596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] The token cache is being invalidate because project fe46950ac4d445ff81b0a3f731062d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:43.338959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.339692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.341507 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4204/8448] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:43 2022] DELETE /identity/v3/projects/fe46950ac4d445ff81b0a3f731062d93 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:43.345793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:43.348312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:43.364647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff15067-1b46-4d1a-b845-851cb2395115 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:43.564217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:43.564794 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:43.565075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:43.565351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] PATH_INFO: `/v3/users/bd46e53977ed4c56946aa620410ff902` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:43.567104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd46e53977ed4c56946aa620410ff902)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:43.569238 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:43.635213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:43.654749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd46e53977ed4c56946aa620410ff902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.655710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd46e53977ed4c56946aa620410ff902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.678697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd46e53977ed4c56946aa620410ff902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.679573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd46e53977ed4c56946aa620410ff902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.704755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-f233f27c-8569-49cf-b44a-8ff51261f04f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd46e53977ed4c56946aa620410ff902'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:43.723371 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4245/8449] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:43 2022] DELETE /identity/v3/users/bd46e53977ed4c56946aa620410ff902 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:44.111178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:44.113239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:44.141358 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfca564-b1bd-45a4-836b-d7ad6278ca2c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:44.274522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:44.292694 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:44.293842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:44.295277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:44.295881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] PATH_INFO: `/v3/projects/47bb8b7c08dc4df99eaa880ccc769d98` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:44.298353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bb8b7c08dc4df99eaa880ccc769d98)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:44.306078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:44.408659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bb8b7c08dc4df99eaa880ccc769d98', 'request_id': 'req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:44.424414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bb8b7c08dc4df99eaa880ccc769d98', 'request_id': 'req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:44.441399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bb8b7c08dc4df99eaa880ccc769d98', 'request_id': 'req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:44.462469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] The token cache is being invalidate because project 47bb8b7c08dc4df99eaa880ccc769d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:44.462795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:44.466509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-15addb6c-04d3-4c0a-a15e-05e7f951ad9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:44.468261 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4205/8450] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:44 2022] DELETE /identity/v3/projects/47bb8b7c08dc4df99eaa880ccc769d98 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:44.474825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fd65-b7ed-454d-906f-0b1fb586e6cb None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:44.476519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fd65-b7ed-454d-906f-0b1fb586e6cb None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:44.477009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fd65-b7ed-454d-906f-0b1fb586e6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:44.529169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5c17fd65-b7ed-454d-906f-0b1fb586e6cb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:44.530814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5c17fd65-b7ed-454d-906f-0b1fb586e6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:44.610359 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4246/8451] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:44.621524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69acb5-2472-423e-b485-f9cfe8aab714 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:44.621955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69acb5-2472-423e-b485-f9cfe8aab714 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:44.622700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69acb5-2472-423e-b485-f9cfe8aab714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:44.679991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-fe69acb5-2472-423e-b485-f9cfe8aab714 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:44.681587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fe69acb5-2472-423e-b485-f9cfe8aab714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:44.690844 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4206/8452] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:44.694872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17fd65-b7ed-454d-906f-0b1fb586e6cb None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:44.711176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17fd65-b7ed-454d-906f-0b1fb586e6cb None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:44.717386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:44.718198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:44.718571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:44.718951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:44.719725 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:24:44.720129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:44.722077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:44.741391 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4247/8453] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:24:44.746468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69acb5-2472-423e-b485-f9cfe8aab714 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:44.757738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69acb5-2472-423e-b485-f9cfe8aab714 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:44.763323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:44.763951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:44.764294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:44.765008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:44.766068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:44.768598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:44.825196 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4207/8454] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:44 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:44.826929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:44.839815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f1e36-36b2-40ec-9b32-7765efa22ad8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:44.923571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:44.924243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:44.924524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:44.924854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:44.925711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:44.927901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:44.969448 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4248/8455] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:44 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:44.970696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:44.983654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23f569-e700-4c99-951e-239afa1f0534 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:44.988518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:44.989133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:44.989411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:44.989741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:44.990741 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:44.992372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.005301 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4208/8456] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:45.008898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.020530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e5236-f26c-4b4b-89d3-3482d34d8e0c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.026598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23b190-fd75-463b-a99a-f772c2240732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.027566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b190-fd75-463b-a99a-f772c2240732 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.028225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b190-fd75-463b-a99a-f772c2240732 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.028225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b190-fd75-463b-a99a-f772c2240732 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.029014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23b190-fd75-463b-a99a-f772c2240732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.031079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23b190-fd75-463b-a99a-f772c2240732 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.043334 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4249/8457] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:45.047849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.060123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828aefa-95fb-4fff-9fef-094b9ed952c4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.067289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.068113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.068886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.069221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] PATH_INFO: `/v3/projects/83353167d11c4133a80f5630e7b24f0a/users/f8a3a7c83aa14e088423d8dfbe39ff19/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.090656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] RBAC: Authorizing `identity:create_grant(project_id=83353167d11c4133a80f5630e7b24f0a, user_id=f8a3a7c83aa14e088423d8dfbe39ff19, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.093718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.130194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec48f69-03e4-472b-8e7c-1c51ca58ac95 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.130763 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec48f69-03e4-472b-8e7c-1c51ca58ac95 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.131185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec48f69-03e4-472b-8e7c-1c51ca58ac95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.139981 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4209/8458] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:45 2022] PUT /identity/v3/projects/83353167d11c4133a80f5630e7b24f0a/users/f8a3a7c83aa14e088423d8dfbe39ff19/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:45.172837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-bec48f69-03e4-472b-8e7c-1c51ca58ac95 None None] MFA Rules not processed for user `f8a3a7c83aa14e088423d8dfbe39ff19`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:45.174162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bec48f69-03e4-472b-8e7c-1c51ca58ac95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:45.180162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:45.278313 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4250/8459] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:45.283314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.393313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1911f5-eeba-4c0f-8f72-3313006c410f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.400120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.400945 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.401316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.402209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.402932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.405429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.444260 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4210/8460] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:45 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:45.445175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec48f69-03e4-472b-8e7c-1c51ca58ac95 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.456265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec48f69-03e4-472b-8e7c-1c51ca58ac95 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.517516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.518161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.518544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.519149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.519867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.522048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.552760 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4251/8461] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:45 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:24:45.557208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.570481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055639d-471b-4d77-9404-292a9d20a936 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.575806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.576465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.576746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.577009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.578168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.580107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.580977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:45.596573 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4211/8462] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:45.598857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.625183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c394f-8f84-4b85-a2ae-f0cf7af7c956 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.631492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.632252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.632594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.632933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] PATH_INFO: `/v3/projects/68a7dfc29e984352ad11519b0e2b69f7/users/92b99b8f12624271b29c9af71272e6ca/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.652996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a7dfc29e984352ad11519b0e2b69f7, user_id=92b99b8f12624271b29c9af71272e6ca, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.655849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.676602 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4252/8463] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:45 2022] PUT /identity/v3/projects/68a7dfc29e984352ad11519b0e2b69f7/users/92b99b8f12624271b29c9af71272e6ca/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:24:45.681063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.705395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f908f-2d22-46a8-ab3a-89aadee06946 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.789950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6be91-5bf1-414f-b89b-1fdeb7c01668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.790612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6be91-5bf1-414f-b89b-1fdeb7c01668 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.790920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6be91-5bf1-414f-b89b-1fdeb7c01668 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.791429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6be91-5bf1-414f-b89b-1fdeb7c01668 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.792159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6be91-5bf1-414f-b89b-1fdeb7c01668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.794312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6be91-5bf1-414f-b89b-1fdeb7c01668 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.814750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:45.815007 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4212/8464] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:45.833848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415bf7f-63cc-43d1-af58-48371d7c6975 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:45.839824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:45.840526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.840801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.841073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] PATH_INFO: `/v3/projects/68a7dfc29e984352ad11519b0e2b69f7/users/92b99b8f12624271b29c9af71272e6ca/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.843704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a7dfc29e984352ad11519b0e2b69f7, user_id=92b99b8f12624271b29c9af71272e6ca, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:45.853604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:45.861287 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4253/8465] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:45 2022] PUT /identity/v3/projects/68a7dfc29e984352ad11519b0e2b69f7/users/92b99b8f12624271b29c9af71272e6ca/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:45.872821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:45.873105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:45.873872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:45.912056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] MFA Rules not processed for user `92b99b8f12624271b29c9af71272e6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:45.913312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:46.016690 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4213/8466] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:46.028997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:46.031789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:46.110202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9d0e3-7774-4e6c-8df6-524af6225929 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:46.114095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c43ca2-a985-4489-8df0-516e219ecd0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:46.114723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43ca2-a985-4489-8df0-516e219ecd0c service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:46.115000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43ca2-a985-4489-8df0-516e219ecd0c service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:46.115620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43ca2-a985-4489-8df0-516e219ecd0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:46.127043 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:46.183206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c43ca2-a985-4489-8df0-516e219ecd0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:46.184386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c43ca2-a985-4489-8df0-516e219ecd0c service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:46.196105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4254/8467] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:50.600231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:50.601917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:50.618482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce63aff-24b5-4ec4-ab72-858b60e4437c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:50.622491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:50.623235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:50.623542 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:50.623784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:50.634291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:50.635831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:50.646962 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4214/8468] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:24:54.895098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e98a90-ea6f-4244-a5d6-9898adb7a1c6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:54.897170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e98a90-ea6f-4244-a5d6-9898adb7a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:54.897647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e98a90-ea6f-4244-a5d6-9898adb7a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:54.927875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-30e98a90-ea6f-4244-a5d6-9898adb7a1c6 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:54.929282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-30e98a90-ea6f-4244-a5d6-9898adb7a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:55.006025 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4255/8469] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:24:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:55.079997 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:55.082141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:55.186823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a917b77-f1fa-4dd8-aae2-ff48478bf715 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:55.195765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1452ff-4ace-4506-8958-ac68a7c65ec2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:55.197965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1452ff-4ace-4506-8958-ac68a7c65ec2 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:55.198500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1452ff-4ace-4506-8958-ac68a7c65ec2 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:55.198961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1452ff-4ace-4506-8958-ac68a7c65ec2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:55.222826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:55.241501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1452ff-4ace-4506-8958-ac68a7c65ec2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:55.243215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1452ff-4ace-4506-8958-ac68a7c65ec2 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:55.267122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4215/8470] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:24:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:24:55.743698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e98a90-ea6f-4244-a5d6-9898adb7a1c6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:55.814797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e98a90-ea6f-4244-a5d6-9898adb7a1c6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:55.819766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:55.820409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:55.820685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:55.820957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] PATH_INFO: `/v3/users/f8a3a7c83aa14e088423d8dfbe39ff19` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:55.822116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a3a7c83aa14e088423d8dfbe39ff19)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:55.824086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:55.873964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a3a7c83aa14e088423d8dfbe39ff19'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:55.874627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a3a7c83aa14e088423d8dfbe39ff19'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:55.882485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a3a7c83aa14e088423d8dfbe39ff19'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:55.883355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a3a7c83aa14e088423d8dfbe39ff19'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:55.888285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:55.898223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a3a7c83aa14e088423d8dfbe39ff19'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:55.911384 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4256/8471] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:55 2022] DELETE /identity/v3/users/f8a3a7c83aa14e088423d8dfbe39ff19 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:55.915812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:55.916096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:55.916363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:55.936844 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:55.938108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:56.001875 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4216/8472] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:56.012330 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:56.013980 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:56.080055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29b1d4-8a49-4199-a02c-5fed03cd5891 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:56.084239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:56.084859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:56.085137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:56.085384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:56.095295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:56.096973 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:56.117684 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4257/8473] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:24:56.407084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:56.408700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:56.430732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78fc9c-0348-4ac8-940b-22c3a614238b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:56.436322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:56.437173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:56.437550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:56.437819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] PATH_INFO: `/v3/projects/83353167d11c4133a80f5630e7b24f0a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:56.438927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] RBAC: Authorizing `identity:delete_project(project_id=83353167d11c4133a80f5630e7b24f0a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:56.440930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:56.508078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83353167d11c4133a80f5630e7b24f0a', 'request_id': 'req-c495e4dc-4acd-4546-9dbe-6f670981611d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.517975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83353167d11c4133a80f5630e7b24f0a', 'request_id': 'req-c495e4dc-4acd-4546-9dbe-6f670981611d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.526100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83353167d11c4133a80f5630e7b24f0a', 'request_id': 'req-c495e4dc-4acd-4546-9dbe-6f670981611d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.542862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] The token cache is being invalidate because project 83353167d11c4133a80f5630e7b24f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:56.543197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.543968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c495e4dc-4acd-4546-9dbe-6f670981611d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.546308 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4217/8474] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:56 2022] DELETE /identity/v3/projects/83353167d11c4133a80f5630e7b24f0a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:56.551057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:56.571427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4666-d067-43fc-af74-68bffc2eda97 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:56.610078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:56.610204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:56.610423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:56.632320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] MFA Rules not processed for user `342e60103c9b45bcb348b6f8bcd686ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:56.633573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:56.663025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:56.663953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:56.665053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:56.665411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] PATH_INFO: `/v3/users/92b99b8f12624271b29c9af71272e6ca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:56.666704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b99b8f12624271b29c9af71272e6ca)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:56.677162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:56.729526 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4218/8475] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:56.761701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b99b8f12624271b29c9af71272e6ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.762228 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b99b8f12624271b29c9af71272e6ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.773003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b99b8f12624271b29c9af71272e6ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.773873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b99b8f12624271b29c9af71272e6ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.799159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-1cbf3f98-2e08-44d2-9458-358f30eeb6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b99b8f12624271b29c9af71272e6ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:56.818250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:56.825693 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4258/8476] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:24:56 2022] DELETE /identity/v3/users/92b99b8f12624271b29c9af71272e6ca => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:57.325639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:57.327113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:57.358048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a281ebf-9999-431d-a1e3-7ac811612d55 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:57.382094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:57.429137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:57.429892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:57.431282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:57.431282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] PATH_INFO: `/v3/projects/68a7dfc29e984352ad11519b0e2b69f7` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:57.432090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a7dfc29e984352ad11519b0e2b69f7)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:57.434066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:57.489109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a7dfc29e984352ad11519b0e2b69f7', 'request_id': 'req-921f5e91-908c-4fba-b5b5-bfc77428f35c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:57.501266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a7dfc29e984352ad11519b0e2b69f7', 'request_id': 'req-921f5e91-908c-4fba-b5b5-bfc77428f35c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:57.509004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a7dfc29e984352ad11519b0e2b69f7', 'request_id': 'req-921f5e91-908c-4fba-b5b5-bfc77428f35c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:57.523611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] The token cache is being invalidate because project 68a7dfc29e984352ad11519b0e2b69f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:24:57.524043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:57.524867 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-921f5e91-908c-4fba-b5b5-bfc77428f35c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:24:57.526674 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4219/8477] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:24:57 2022] DELETE /identity/v3/projects/68a7dfc29e984352ad11519b0e2b69f7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:57.532857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247c3d6-fc54-483c-aec2-2a3cce76f614 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:57.533369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247c3d6-fc54-483c-aec2-2a3cce76f614 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:57.533483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247c3d6-fc54-483c-aec2-2a3cce76f614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:57.557540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8247c3d6-fc54-483c-aec2-2a3cce76f614 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:57.561602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8247c3d6-fc54-483c-aec2-2a3cce76f614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:57.641224 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4259/8478] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:57.651432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6222f-8de2-4a38-8f6d-724f5d917bea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:57.651706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6222f-8de2-4a38-8f6d-724f5d917bea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:57.651979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6222f-8de2-4a38-8f6d-724f5d917bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:57.690488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bea6222f-8de2-4a38-8f6d-724f5d917bea None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:57.692264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bea6222f-8de2-4a38-8f6d-724f5d917bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:57.726310 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4220/8479] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:57.730298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247c3d6-fc54-483c-aec2-2a3cce76f614 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:57.743422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247c3d6-fc54-483c-aec2-2a3cce76f614 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:57.751679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:57.752375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:57.752759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:57.753129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:57.753915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:24:57.754299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:57.756013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:57.769507 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4260/8480] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:24:57.773977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6222f-8de2-4a38-8f6d-724f5d917bea None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:57.784515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6222f-8de2-4a38-8f6d-724f5d917bea None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:57.792466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:57.792466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:57.792466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:57.792466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:57.792906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:57.794925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:57.829396 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4221/8481] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:24:57 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:57.834530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:57.851656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4cdf-a914-4b80-a109-dc92b126131a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:57.930191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:57.931055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:57.931320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:57.931655 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:57.932382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:57.934659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:57.962977 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4261/8482] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:24:57 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:24:57.967716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:57.985427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619af62-a30b-4850-adf7-6777b8f36d15 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:57.991423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:57.992070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:57.992355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:57.992629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:57.993696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:57.995479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:58.007285 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4222/8483] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:24:58.011569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:58.023791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9e05-9332-4e44-a546-0aa6f8f8a332 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:58.029424 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b04da1-869e-4d6f-9296-9a12a932a700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:58.030234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b04da1-869e-4d6f-9296-9a12a932a700 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:58.030549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b04da1-869e-4d6f-9296-9a12a932a700 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:58.030787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b04da1-869e-4d6f-9296-9a12a932a700 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:58.031514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b04da1-869e-4d6f-9296-9a12a932a700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:58.033105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b04da1-869e-4d6f-9296-9a12a932a700 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:58.049121 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4262/8484] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:24:58.050544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:58.064093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0445ec4-580b-4e55-afe4-c99160d8be96 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:58.069044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:58.070265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:58.070265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:58.070488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] PATH_INFO: `/v3/projects/454be92019f0498980bebad9e988f37e/users/0b32996144954c8face21e641875f7a0/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:58.088731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454be92019f0498980bebad9e988f37e, user_id=0b32996144954c8face21e641875f7a0, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:58.092093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:58.111974 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4223/8485] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:24:58 2022] PUT /identity/v3/projects/454be92019f0498980bebad9e988f37e/users/0b32996144954c8face21e641875f7a0/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:24:58.124981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39636142-3637-4f2b-9165-f147dbc8ceec None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:58.125327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39636142-3637-4f2b-9165-f147dbc8ceec None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:58.125713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39636142-3637-4f2b-9165-f147dbc8ceec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:58.159845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-39636142-3637-4f2b-9165-f147dbc8ceec None None] MFA Rules not processed for user `0b32996144954c8face21e641875f7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:24:58.161495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-39636142-3637-4f2b-9165-f147dbc8ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:58.223026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:58.235505 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4263/8486] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:24:58.249437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:24:58.251450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:24:58.328002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:58.335128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861060ad-50b2-477b-bf21-29618eb3a277 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:24:58.343778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298bd6c-4616-46e6-a95f-e9c5c0332acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:24:58.344713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298bd6c-4616-46e6-a95f-e9c5c0332acc service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:24:58.345236 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298bd6c-4616-46e6-a95f-e9c5c0332acc service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:24:58.345990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298bd6c-4616-46e6-a95f-e9c5c0332acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:24:58.370299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298bd6c-4616-46e6-a95f-e9c5c0332acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:24:58.371990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298bd6c-4616-46e6-a95f-e9c5c0332acc service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:24:58.383346 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4224/8487] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:24:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:07.170540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39636142-3637-4f2b-9165-f147dbc8ceec None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:07.308335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39636142-3637-4f2b-9165-f147dbc8ceec None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:07.315769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:07.316584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:07.316862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:07.317132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] PATH_INFO: `/v3/users/0b32996144954c8face21e641875f7a0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:07.318326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b32996144954c8face21e641875f7a0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:07.320264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:07.380013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b32996144954c8face21e641875f7a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:07.380757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b32996144954c8face21e641875f7a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:07.392546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b32996144954c8face21e641875f7a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:07.393307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b32996144954c8face21e641875f7a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:07.410145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b32996144954c8face21e641875f7a0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:07.424985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4264/8488] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:07 2022] DELETE /identity/v3/users/0b32996144954c8face21e641875f7a0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:07.429436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:07.430671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:07.430963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:07.470775 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:07.472368 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:07.548990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4225/8489] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:07.560525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:07.562268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:07.634478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:07.665276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290f0be-cdbe-4420-85e4-4205791d348f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:07.669572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8deb4-6d62-449b-938c-18f27fe89e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:07.670482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8deb4-6d62-449b-938c-18f27fe89e29 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:07.670984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8deb4-6d62-449b-938c-18f27fe89e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:07.671448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8deb4-6d62-449b-938c-18f27fe89e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:07.683145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8deb4-6d62-449b-938c-18f27fe89e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:07.684968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8deb4-6d62-449b-938c-18f27fe89e29 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:07.703086 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4265/8490] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:25:08.088275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.090705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:08.117247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f902e75-f852-4768-b919-83f041b0e706 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:08.129142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:08.129839 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.130122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.130399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] PATH_INFO: `/v3/projects/454be92019f0498980bebad9e988f37e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.131578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=454be92019f0498980bebad9e988f37e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:08.133596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:08.231454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454be92019f0498980bebad9e988f37e', 'request_id': 'req-d6595147-c97f-4485-88c5-7adb2a747791'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:08.259891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454be92019f0498980bebad9e988f37e', 'request_id': 'req-d6595147-c97f-4485-88c5-7adb2a747791'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:08.271654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454be92019f0498980bebad9e988f37e', 'request_id': 'req-d6595147-c97f-4485-88c5-7adb2a747791'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:08.283554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] The token cache is being invalidate because project 454be92019f0498980bebad9e988f37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:08.284600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:08.284764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d6595147-c97f-4485-88c5-7adb2a747791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:08.287683 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4226/8491] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:08 2022] DELETE /identity/v3/projects/454be92019f0498980bebad9e988f37e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:08.292568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815018-3d78-4b20-9f50-10517e8698ac None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.293065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815018-3d78-4b20-9f50-10517e8698ac None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.293163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815018-3d78-4b20-9f50-10517e8698ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.318226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-12815018-3d78-4b20-9f50-10517e8698ac None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:08.319499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-12815018-3d78-4b20-9f50-10517e8698ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:08.394020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4266/8492] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:08.403049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f0558-e9b0-4676-940b-388c2ea4e8e7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.403406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f0558-e9b0-4676-940b-388c2ea4e8e7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.403784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f0558-e9b0-4676-940b-388c2ea4e8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.427061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-469f0558-e9b0-4676-940b-388c2ea4e8e7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:08.429226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-469f0558-e9b0-4676-940b-388c2ea4e8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:08.438320 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4227/8493] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:08.442450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12815018-3d78-4b20-9f50-10517e8698ac None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.455301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12815018-3d78-4b20-9f50-10517e8698ac None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:08.460784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:08.461426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.461743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.462017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.462701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:25:08.463009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:08.464665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:08.483240 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4267/8494] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:25:08.484120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f0558-e9b0-4676-940b-388c2ea4e8e7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.497578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f0558-e9b0-4676-940b-388c2ea4e8e7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:08.502895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:08.503534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.503815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.504084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.504719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:08.506972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:08.539252 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4228/8495] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:08 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:25:08.543614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.555020 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b8731-e010-4c44-bd72-e7d44b16a7a7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:08.634658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:08.635294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.635573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.635845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.636491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:08.638508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:08.666584 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4268/8496] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:08 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:25:08.668688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.679887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7ce4-b222-44f0-a950-a70038198242 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:08.689975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:08.690629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.691045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.691321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.691965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:08.693733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:08.704923 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4229/8497] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:08.709355 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.719904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906048-4cb6-411b-9dc8-bcc9561cf888 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:08.725275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6085a-e224-4e55-8236-45f62708b94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:08.726010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6085a-e224-4e55-8236-45f62708b94a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.726845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6085a-e224-4e55-8236-45f62708b94a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.726845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6085a-e224-4e55-8236-45f62708b94a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.727380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6085a-e224-4e55-8236-45f62708b94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:08.729171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6085a-e224-4e55-8236-45f62708b94a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:08.746554 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4269/8498] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:08.747905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.759135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34f4f-4bd4-4cbc-9488-359ea8ae462f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:08.764564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:08.764823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.765100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.765371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] PATH_INFO: `/v3/projects/120e67d0734d420c90eec58239f7c907/users/677033f05fce4c16bcb573ec22f27fab/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.788578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=120e67d0734d420c90eec58239f7c907, user_id=677033f05fce4c16bcb573ec22f27fab, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:08.791389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:08.807625 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4230/8499] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:08 2022] PUT /identity/v3/projects/120e67d0734d420c90eec58239f7c907/users/677033f05fce4c16bcb573ec22f27fab/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:08.817955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1768ff-75b5-4deb-9f7b-5e4006ce46e6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:08.818240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1768ff-75b5-4deb-9f7b-5e4006ce46e6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:08.818537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1768ff-75b5-4deb-9f7b-5e4006ce46e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:08.851940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fd1768ff-75b5-4deb-9f7b-5e4006ce46e6 None None] MFA Rules not processed for user `677033f05fce4c16bcb573ec22f27fab`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:08.853403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fd1768ff-75b5-4deb-9f7b-5e4006ce46e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:08.888141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:08.928499 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4270/8500] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:25:08.932919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:08.938478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:09.012705 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3754cf-fbc9-4d84-abd8-e2e101a8ac3f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.018763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.019449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.019768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.020155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.020818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.022826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.059319 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4231/8501] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:08 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:09.064721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1768ff-75b5-4deb-9f7b-5e4006ce46e6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.079089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1768ff-75b5-4deb-9f7b-5e4006ce46e6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.166206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.166961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.167683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.167683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.168425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.173045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.207226 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4271/8502] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:09 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:09.207968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.225604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c0840-0d2b-4eb8-8093-6f965a9c9930 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.231763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.232583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.233009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.233429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.235105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.238588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.254549 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4232/8503] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:09.258773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.273250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a4c14-9248-4e5b-bcfb-9cca5349efc8 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.279062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838daa3-d385-4d83-a932-4e03485e8acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.280225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838daa3-d385-4d83-a932-4e03485e8acb admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.280351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838daa3-d385-4d83-a932-4e03485e8acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.280719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838daa3-d385-4d83-a932-4e03485e8acb admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.281510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838daa3-d385-4d83-a932-4e03485e8acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.283582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838daa3-d385-4d83-a932-4e03485e8acb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.301034 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4272/8504] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:25:09.305932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.317080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955f2e1-3f9c-499f-a40f-87ca3b940f5f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.323190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.323872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.324152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.324680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] PATH_INFO: `/v3/projects/1f537a68317a496cbae0de9163393407/users/2e923f015f53425aab0107e6fc5e48c9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.343642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f537a68317a496cbae0de9163393407, user_id=2e923f015f53425aab0107e6fc5e48c9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.348095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.370512 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4233/8505] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:09 2022] PUT /identity/v3/projects/1f537a68317a496cbae0de9163393407/users/2e923f015f53425aab0107e6fc5e48c9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:09.380672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca65b2f-0c5c-4b70-bf8b-9ce6187c3975 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.381603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca65b2f-0c5c-4b70-bf8b-9ce6187c3975 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.381603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca65b2f-0c5c-4b70-bf8b-9ce6187c3975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.414197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-aca65b2f-0c5c-4b70-bf8b-9ce6187c3975 None None] MFA Rules not processed for user `2e923f015f53425aab0107e6fc5e48c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:09.415435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aca65b2f-0c5c-4b70-bf8b-9ce6187c3975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:09.499724 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4273/8506] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:09.500900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.578935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba10a9b-b76b-40af-948e-d1fc5aba1073 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.584604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.585341 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.586105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.586105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.586868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.588902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.624057 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4234/8507] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:09 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:09.625685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca65b2f-0c5c-4b70-bf8b-9ce6187c3975 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.640472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca65b2f-0c5c-4b70-bf8b-9ce6187c3975 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.683856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:09.707390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.708069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.708315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.708697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.709533 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.711378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.746273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.747974 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4274/8508] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:09 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:09.756818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee171bc8-920d-43f8-982e-06cefaf4c37f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.762323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.762686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.762966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.763245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.764063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.765696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.781268 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4235/8509] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:09.785721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.799497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1162d-6636-4175-9429-951038750a65 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.804746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:09.805429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:09.805747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:09.806026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] PATH_INFO: `/v3/projects/5656ef6a029a41bab4356645f13ce58d/users/5d46841b62bb456685d6db7f204d495b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:09.832342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5656ef6a029a41bab4356645f13ce58d, user_id=5d46841b62bb456685d6db7f204d495b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:09.836787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:09.862775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4275/8510] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:09 2022] PUT /identity/v3/projects/5656ef6a029a41bab4356645f13ce58d/users/5d46841b62bb456685d6db7f204d495b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:09.863676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:09.890761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5945f7b-07ad-40b7-a97e-22e06f0d9ace admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:09.947147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:10.026778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e1e3d-907f-4355-a7c6-39cd33063b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:10.027419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1e3d-907f-4355-a7c6-39cd33063b8f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:10.027778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1e3d-907f-4355-a7c6-39cd33063b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:10.028525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1e3d-907f-4355-a7c6-39cd33063b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:10.029277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e1e3d-907f-4355-a7c6-39cd33063b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:10.030469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e1e3d-907f-4355-a7c6-39cd33063b8f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:10.050423 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4236/8511] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:10.051265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:10.069898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fcb9a-1769-46df-8637-16d85d6e56a2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:10.083995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:10.084699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:10.085011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:10.085249 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] PATH_INFO: `/v3/projects/5656ef6a029a41bab4356645f13ce58d/users/5d46841b62bb456685d6db7f204d495b/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:10.087541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5656ef6a029a41bab4356645f13ce58d, user_id=5d46841b62bb456685d6db7f204d495b, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:10.090476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:10.118636 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4276/8512] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:10 2022] PUT /identity/v3/projects/5656ef6a029a41bab4356645f13ce58d/users/5d46841b62bb456685d6db7f204d495b/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:25:10.120362 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:10.120731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:10.121028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:10.166376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] MFA Rules not processed for user `5d46841b62bb456685d6db7f204d495b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:10.167500 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:10.331272 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4237/8513] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:10.342599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:10.345630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:10.458470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9da40-a2ec-4a20-9d6f-57c2b795a7e0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:10.468138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:10.469076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:10.469553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:10.469967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:10.579255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:10.581900 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:10.601347 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4277/8514] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:12.609792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:12.611399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:12.654680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cb83e-0110-4045-9ae3-39d03e51de8b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:12.677160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:12.677865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:12.678324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:12.678610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:12.792373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:12.799158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:12.811209 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4238/8515] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:13.918630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:13.936568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f924b-423e-4eac-bb0b-6f902745d254 service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:13.946724 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011854f-7708-4c2b-ba22-742d320c3f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:13.947373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011854f-7708-4c2b-ba22-742d320c3f2c service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:13.947650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011854f-7708-4c2b-ba22-742d320c3f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:13.947921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011854f-7708-4c2b-ba22-742d320c3f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:13.974849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011854f-7708-4c2b-ba22-742d320c3f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:13.976781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011854f-7708-4c2b-ba22-742d320c3f2c service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:14.003107 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4278/8516] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:18.563328 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:18.662827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:18.718556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff239-c132-4c9f-8603-4bbda5ce2e80 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:18.725398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:18.726099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:18.726415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:18.726656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] PATH_INFO: `/v3/users/677033f05fce4c16bcb573ec22f27fab` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:18.730721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=677033f05fce4c16bcb573ec22f27fab)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:18.733337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:18.857625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677033f05fce4c16bcb573ec22f27fab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:18.874448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677033f05fce4c16bcb573ec22f27fab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:18.884983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677033f05fce4c16bcb573ec22f27fab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:18.885658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677033f05fce4c16bcb573ec22f27fab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:18.886321 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677033f05fce4c16bcb573ec22f27fab'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:18.903890 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4239/8517] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:18 2022] DELETE /identity/v3/users/677033f05fce4c16bcb573ec22f27fab => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:18.908694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:18.909235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:18.909421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:18.968358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:18.970322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:18.983736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:19.062186 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4279/8518] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:19.069150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:19.070978 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:19.175080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe3fb4-4a79-49ad-acab-29bb34d8d712 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:19.181091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:19.181884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:19.182400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:19.182685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:19.206532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:19.208558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:19.231044 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4240/8519] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:19.637354 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:19.639030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:19.663242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e92ab-e15c-4213-9aa2-69420b7789f2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:19.685925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:19.687893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:19.688935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:19.689916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] PATH_INFO: `/v3/projects/120e67d0734d420c90eec58239f7c907` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:19.693496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] RBAC: Authorizing `identity:delete_project(project_id=120e67d0734d420c90eec58239f7c907)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:19.700602 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:19.773583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '120e67d0734d420c90eec58239f7c907', 'request_id': 'req-af37a89f-c525-4335-83d7-56bda626b49f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:19.784103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '120e67d0734d420c90eec58239f7c907', 'request_id': 'req-af37a89f-c525-4335-83d7-56bda626b49f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:19.798626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '120e67d0734d420c90eec58239f7c907', 'request_id': 'req-af37a89f-c525-4335-83d7-56bda626b49f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:19.814275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] The token cache is being invalidate because project 120e67d0734d420c90eec58239f7c907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:19.814275 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:19.815028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:19.816808 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4280/8520] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:19 2022] DELETE /identity/v3/projects/120e67d0734d420c90eec58239f7c907 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:19.821256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:19.832485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06043cf-3758-48af-a5e2-ab870160bad2 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:19.934722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:19.934722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:19.934722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:19.934722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] PATH_INFO: `/v3/users/2e923f015f53425aab0107e6fc5e48c9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:19.935052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e923f015f53425aab0107e6fc5e48c9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:19.935052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:19.937363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:20.013547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e923f015f53425aab0107e6fc5e48c9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.034199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e923f015f53425aab0107e6fc5e48c9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.050270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e923f015f53425aab0107e6fc5e48c9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.053842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e923f015f53425aab0107e6fc5e48c9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.054979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e923f015f53425aab0107e6fc5e48c9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.109909 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4241/8521] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:19 2022] DELETE /identity/v3/users/2e923f015f53425aab0107e6fc5e48c9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:20.494136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:20.496343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:20.517067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37a89f-c525-4335-83d7-56bda626b49f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:20.615603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:20.616469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:20.616982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:20.617661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] PATH_INFO: `/v3/projects/1f537a68317a496cbae0de9163393407` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:20.619231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f537a68317a496cbae0de9163393407)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:20.621404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:20.697621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f537a68317a496cbae0de9163393407', 'request_id': 'req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.700412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:20.707814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f537a68317a496cbae0de9163393407', 'request_id': 'req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.720160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f537a68317a496cbae0de9163393407', 'request_id': 'req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.736219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] The token cache is being invalidate because project 1f537a68317a496cbae0de9163393407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:20.738029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.738520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.741020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4281/8522] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:20 2022] DELETE /identity/v3/projects/1f537a68317a496cbae0de9163393407 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:20.745708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:20.748025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:20.764271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31462e-11d8-4056-bfba-7e924c11310c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:20.852672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:20.853795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:20.854077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:20.854511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] PATH_INFO: `/v3/users/5d46841b62bb456685d6db7f204d495b` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:20.856018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d46841b62bb456685d6db7f204d495b)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:20.858282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:20.924897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d46841b62bb456685d6db7f204d495b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.947031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d46841b62bb456685d6db7f204d495b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.959247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d46841b62bb456685d6db7f204d495b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.960050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d46841b62bb456685d6db7f204d495b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.962398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d46841b62bb456685d6db7f204d495b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:20.975757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:20.989029 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4242/8523] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:20 2022] DELETE /identity/v3/users/5d46841b62bb456685d6db7f204d495b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:21.069291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:21.072880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:21.200285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4c8e9-5d63-4d40-9464-36bb8d6a51a4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:21.340815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5409af-ebc4-4d6e-b6c2-9936fc39f6f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:21.341875 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5409af-ebc4-4d6e-b6c2-9936fc39f6f7 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:21.344845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5409af-ebc4-4d6e-b6c2-9936fc39f6f7 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:21.344845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5409af-ebc4-4d6e-b6c2-9936fc39f6f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:21.344845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ce5409af-ebc4-4d6e-b6c2-9936fc39f6f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:21.362709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5409af-ebc4-4d6e-b6c2-9936fc39f6f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:21.366074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5409af-ebc4-4d6e-b6c2-9936fc39f6f7 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:21.385062 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4282/8524] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:21.716990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:21.719713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:21.832416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606193ca-eebc-488d-beb4-f6b654619370 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:21.838680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:21.839370 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:21.839745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:21.840169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] PATH_INFO: `/v3/projects/5656ef6a029a41bab4356645f13ce58d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:21.841448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5656ef6a029a41bab4356645f13ce58d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:21.843515 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:21.906093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5656ef6a029a41bab4356645f13ce58d', 'request_id': 'req-623f9c13-93ac-4da0-b071-4f20514db09a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:21.917107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5656ef6a029a41bab4356645f13ce58d', 'request_id': 'req-623f9c13-93ac-4da0-b071-4f20514db09a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:21.926904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5656ef6a029a41bab4356645f13ce58d', 'request_id': 'req-623f9c13-93ac-4da0-b071-4f20514db09a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:21.941269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] The token cache is being invalidate because project 5656ef6a029a41bab4356645f13ce58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:21.941607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:21.942277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-623f9c13-93ac-4da0-b071-4f20514db09a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:21.945016 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4243/8525] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:21 2022] DELETE /identity/v3/projects/5656ef6a029a41bab4356645f13ce58d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:21.951202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e6568-2df2-43ca-beef-f79ba1ccef96 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:21.951807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e6568-2df2-43ca-beef-f79ba1ccef96 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:21.952152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e6568-2df2-43ca-beef-f79ba1ccef96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.024779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-4e5e6568-2df2-43ca-beef-f79ba1ccef96 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:22.026229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4e5e6568-2df2-43ca-beef-f79ba1ccef96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:22.065514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:22.102762 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4283/8526] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:22.110975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595e91-4328-4cd8-ad0a-97caf59b3abb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.111290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595e91-4328-4cd8-ad0a-97caf59b3abb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.111598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de595e91-4328-4cd8-ad0a-97caf59b3abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.136987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-de595e91-4328-4cd8-ad0a-97caf59b3abb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:22.138376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-de595e91-4328-4cd8-ad0a-97caf59b3abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:22.148093 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4244/8527] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:25:22.152692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e6568-2df2-43ca-beef-f79ba1ccef96 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.165191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e6568-2df2-43ca-beef-f79ba1ccef96 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.170407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.171232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.171358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.171721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.172437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:25:22.172838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.174664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.189497 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4284/8528] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:25:22.192595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595e91-4328-4cd8-ad0a-97caf59b3abb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.203133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de595e91-4328-4cd8-ad0a-97caf59b3abb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.207991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.208607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.208891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.209179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.209883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.212459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.253034 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4245/8529] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:22 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:22.257394 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.270069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44151cdb-0736-410f-8f9b-720c6628c8ef admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.344919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.345733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.346237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.346630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.347397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.349499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.379626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4285/8530] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:22 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:22.384833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.398166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b70cc-db2e-437c-bab0-546087a7f705 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.404075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.404726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.404987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.405300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.406325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.408016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.421700 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4246/8531] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:22.425961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.439203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e078-1c8e-45dd-8362-fd293bf86e51 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.450600 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28e831-b781-4738-9fbe-336288ea3d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.451347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28e831-b781-4738-9fbe-336288ea3d31 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.451773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28e831-b781-4738-9fbe-336288ea3d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.452547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28e831-b781-4738-9fbe-336288ea3d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.453370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28e831-b781-4738-9fbe-336288ea3d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.455264 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28e831-b781-4738-9fbe-336288ea3d31 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.468617 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4286/8532] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:22.473008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.485244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8abd8-a6c9-439d-a04f-ef815a1d637f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.491350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.492374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.492660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.493030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] PATH_INFO: `/v3/projects/855cb0e44b9b49d1aca1dd77746c18df/users/cfc02184a4bd40f99c1bdf0b9f65955a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.516299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=855cb0e44b9b49d1aca1dd77746c18df, user_id=cfc02184a4bd40f99c1bdf0b9f65955a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.519829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.539231 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4247/8533] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:22 2022] PUT /identity/v3/projects/855cb0e44b9b49d1aca1dd77746c18df/users/cfc02184a4bd40f99c1bdf0b9f65955a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:22.551224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378e8a5-fe5b-498f-af42-0b0eddf24bf6 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.551563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378e8a5-fe5b-498f-af42-0b0eddf24bf6 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.551836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378e8a5-fe5b-498f-af42-0b0eddf24bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.587528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0378e8a5-fe5b-498f-af42-0b0eddf24bf6 None None] MFA Rules not processed for user `cfc02184a4bd40f99c1bdf0b9f65955a`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:22.589180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0378e8a5-fe5b-498f-af42-0b0eddf24bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:22.666507 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4287/8534] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:22.678407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.680442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:22.708813 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:22.770975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e4436-2af1-4765-95ed-b1314a5e28dd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.780541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42752b12-bc8c-4dfe-83dc-5e613bc1c969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.781554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42752b12-bc8c-4dfe-83dc-5e613bc1c969 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.782030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42752b12-bc8c-4dfe-83dc-5e613bc1c969 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.782483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42752b12-bc8c-4dfe-83dc-5e613bc1c969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.793533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42752b12-bc8c-4dfe-83dc-5e613bc1c969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.795944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42752b12-bc8c-4dfe-83dc-5e613bc1c969 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.810562 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4248/8535] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:22.840646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378e8a5-fe5b-498f-af42-0b0eddf24bf6 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:22.884217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:22.921139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378e8a5-fe5b-498f-af42-0b0eddf24bf6 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:22.927687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:22.927860 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:22.928563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:22.928563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] PATH_INFO: `/v3/users/cfc02184a4bd40f99c1bdf0b9f65955a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:22.929607 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc02184a4bd40f99c1bdf0b9f65955a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:22.931687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:22.991831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc02184a4bd40f99c1bdf0b9f65955a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:22.993165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc02184a4bd40f99c1bdf0b9f65955a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:23.003422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc02184a4bd40f99c1bdf0b9f65955a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:23.004506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc02184a4bd40f99c1bdf0b9f65955a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:23.020818 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc02184a4bd40f99c1bdf0b9f65955a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:23.041205 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4288/8536] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:22 2022] DELETE /identity/v3/users/cfc02184a4bd40f99c1bdf0b9f65955a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:23.045954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:23.046478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:23.046478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:23.067294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:23.069197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:23.157886 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4249/8537] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:23.169824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:23.173955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:23.329358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfa0cd-42d8-409e-8972-90438fbd4e34 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:23.333888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a6e65-73cb-45db-931a-e56f7ed1f9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:23.334784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6e65-73cb-45db-931a-e56f7ed1f9db service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:23.335101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6e65-73cb-45db-931a-e56f7ed1f9db service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:23.335593 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6e65-73cb-45db-931a-e56f7ed1f9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:23.348230 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a6e65-73cb-45db-931a-e56f7ed1f9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:23.349759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a6e65-73cb-45db-931a-e56f7ed1f9db service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:23.360862 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4289/8538] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:23.923505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:23.925562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:23.945425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88a1f9-97a4-4e45-ad8c-3088e2ed750f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:23.957933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:23.958660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:23.959001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:23.959340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] PATH_INFO: `/v3/projects/855cb0e44b9b49d1aca1dd77746c18df` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:23.962253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=855cb0e44b9b49d1aca1dd77746c18df)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:23.964352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.171828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '855cb0e44b9b49d1aca1dd77746c18df', 'request_id': 'req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:24.180161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '855cb0e44b9b49d1aca1dd77746c18df', 'request_id': 'req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:24.186578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '855cb0e44b9b49d1aca1dd77746c18df', 'request_id': 'req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:24.199529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] The token cache is being invalidate because project 855cb0e44b9b49d1aca1dd77746c18df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:24.199825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:24.200543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-52bfe401-8be6-40ad-a7ba-742ce0e0d0cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:24.203044 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4250/8539] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:23 2022] DELETE /identity/v3/projects/855cb0e44b9b49d1aca1dd77746c18df => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:24.207969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792aa2e8-3a9d-42bb-a361-da707c0f9017 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.208523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792aa2e8-3a9d-42bb-a361-da707c0f9017 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.208523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792aa2e8-3a9d-42bb-a361-da707c0f9017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.244805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-792aa2e8-3a9d-42bb-a361-da707c0f9017 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:24.252091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-792aa2e8-3a9d-42bb-a361-da707c0f9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:24.309739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:24.323703 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4290/8540] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:24.331787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e25c1-fd5b-4e88-9d81-077b85ec88ef None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.332064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e25c1-fd5b-4e88-9d81-077b85ec88ef None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.332333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e25c1-fd5b-4e88-9d81-077b85ec88ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.352740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e36e25c1-fd5b-4e88-9d81-077b85ec88ef None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:24.354007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e36e25c1-fd5b-4e88-9d81-077b85ec88ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:24.362216 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4251/8541] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:24.367149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792aa2e8-3a9d-42bb-a361-da707c0f9017 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:24.377626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792aa2e8-3a9d-42bb-a361-da707c0f9017 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:24.382787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:24.383432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.383686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.383974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.384634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:25:24.384932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:24.386614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.403506 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4291/8542] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:25:24.404944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e25c1-fd5b-4e88-9d81-077b85ec88ef None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:24.414682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e25c1-fd5b-4e88-9d81-077b85ec88ef None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:24.420276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:24.420900 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.421247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.421541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.422194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:24.424296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.455230 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4252/8543] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:24 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:24.459741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:24.470457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c38b6c-a2e8-460b-a40a-41b7523c5bf6 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:24.533837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:24.534456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.534731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.534998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.535645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:24.537690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.564942 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4292/8544] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:24 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:24.566539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:24.572193 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:24.578155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f59e0f-195d-4ab0-bd91-068eaaa22e32 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:24.584888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:24.585556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.585810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.586076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.586729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:24.588696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.599587 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4253/8545] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:24.603922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:24.614633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b161-b249-4cb2-ac91-e7d56de1e3d4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:24.619637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c6d84-fa7e-40ca-ad7a-4d5cb02e87da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:24.620239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c6d84-fa7e-40ca-ad7a-4d5cb02e87da admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.620513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c6d84-fa7e-40ca-ad7a-4d5cb02e87da admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.620778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c6d84-fa7e-40ca-ad7a-4d5cb02e87da admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.621414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c6d84-fa7e-40ca-ad7a-4d5cb02e87da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:24.623092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c6d84-fa7e-40ca-ad7a-4d5cb02e87da admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.637233 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4293/8546] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:25:24.638569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:24.649721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b601d-6378-4f47-a49f-c35c0399f5f0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:24.655271 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:24.655942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.656212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.656482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] PATH_INFO: `/v3/projects/6db3c501682c4acbb351ad35708c3a4e/users/7cbd7e6ebfab4ec5a923384e1f357fdb/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.672625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db3c501682c4acbb351ad35708c3a4e, user_id=7cbd7e6ebfab4ec5a923384e1f357fdb, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:24.675413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.693254 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4254/8547] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:24 2022] PUT /identity/v3/projects/6db3c501682c4acbb351ad35708c3a4e/users/7cbd7e6ebfab4ec5a923384e1f357fdb/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:24.703708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a61327-0658-4ee9-9a57-54ec21494f53 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.703994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a61327-0658-4ee9-9a57-54ec21494f53 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.704272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a61327-0658-4ee9-9a57-54ec21494f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.733540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e5a61327-0658-4ee9-9a57-54ec21494f53 None None] MFA Rules not processed for user `7cbd7e6ebfab4ec5a923384e1f357fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:24.734729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e5a61327-0658-4ee9-9a57-54ec21494f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:24.802839 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4294/8548] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:24.815935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:24.817643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:24.877176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02e495-03a4-4f95-abc7-996571aba471 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:24.881107 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe94644-a65a-495c-80b8-ab1397408203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:24.881804 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94644-a65a-495c-80b8-ab1397408203 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:24.882049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94644-a65a-495c-80b8-ab1397408203 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:24.883103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe94644-a65a-495c-80b8-ab1397408203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:24.893550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe94644-a65a-495c-80b8-ab1397408203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:24.895062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe94644-a65a-495c-80b8-ab1397408203 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:24.906656 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4255/8549] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:24 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:30.279272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a61327-0658-4ee9-9a57-54ec21494f53 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:30.336664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:30.351716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a61327-0658-4ee9-9a57-54ec21494f53 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:30.357404 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:30.358255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:30.358496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:30.358770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] PATH_INFO: `/v3/users/7cbd7e6ebfab4ec5a923384e1f357fdb` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:30.360042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbd7e6ebfab4ec5a923384e1f357fdb)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:30.362361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:30.419446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbd7e6ebfab4ec5a923384e1f357fdb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:30.420378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbd7e6ebfab4ec5a923384e1f357fdb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:30.429021 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbd7e6ebfab4ec5a923384e1f357fdb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:30.430179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbd7e6ebfab4ec5a923384e1f357fdb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:30.450026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbd7e6ebfab4ec5a923384e1f357fdb'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:30.472254 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4295/8550] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:30 2022] DELETE /identity/v3/users/7cbd7e6ebfab4ec5a923384e1f357fdb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:30.476638 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:30.477148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:30.477148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:30.498136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:30.499649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:30.608889 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4256/8551] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:30.626988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:30.629345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:30.727309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d7b99-6879-40c3-a500-35e5a1e5442d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:30.732850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6033562-1080-4e1f-ab1b-bb0380bef69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:30.733516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6033562-1080-4e1f-ab1b-bb0380bef69e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:30.733805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6033562-1080-4e1f-ab1b-bb0380bef69e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:30.735261 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6033562-1080-4e1f-ab1b-bb0380bef69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:30.747983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6033562-1080-4e1f-ab1b-bb0380bef69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:30.749594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6033562-1080-4e1f-ab1b-bb0380bef69e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:30.762611 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4296/8552] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:25:31.160316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:31.162090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:31.177583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb09c60-b629-4029-b003-45729a1feb69 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:31.185073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:31.185826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.186510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.186510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] PATH_INFO: `/v3/projects/6db3c501682c4acbb351ad35708c3a4e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.187632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db3c501682c4acbb351ad35708c3a4e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:31.189784 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:31.254608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db3c501682c4acbb351ad35708c3a4e', 'request_id': 'req-bf8ba59c-3d9f-4ddb-b100-14275503ae07'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:31.275995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db3c501682c4acbb351ad35708c3a4e', 'request_id': 'req-bf8ba59c-3d9f-4ddb-b100-14275503ae07'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:31.286922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db3c501682c4acbb351ad35708c3a4e', 'request_id': 'req-bf8ba59c-3d9f-4ddb-b100-14275503ae07'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:31.289437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:31.303038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] The token cache is being invalidate because project 6db3c501682c4acbb351ad35708c3a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:31.303421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:31.309010 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-bf8ba59c-3d9f-4ddb-b100-14275503ae07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:31.311047 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4257/8553] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:31 2022] DELETE /identity/v3/projects/6db3c501682c4acbb351ad35708c3a4e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:31.316463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae7d40-8623-4641-b7f9-bbdfaff71d44 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.317887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae7d40-8623-4641-b7f9-bbdfaff71d44 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.318382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae7d40-8623-4641-b7f9-bbdfaff71d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.349262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c4ae7d40-8623-4641-b7f9-bbdfaff71d44 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:31.350934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c4ae7d40-8623-4641-b7f9-bbdfaff71d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:31.432804 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4297/8554] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:31.442313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb19ad-b69f-48cd-9fd0-23f9d367507c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.442778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb19ad-b69f-48cd-9fd0-23f9d367507c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.443350 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb19ad-b69f-48cd-9fd0-23f9d367507c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.469051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-63bb19ad-b69f-48cd-9fd0-23f9d367507c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:31.470187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-63bb19ad-b69f-48cd-9fd0-23f9d367507c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:31.483280 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4258/8555] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:31.487823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae7d40-8623-4641-b7f9-bbdfaff71d44 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:31.502909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae7d40-8623-4641-b7f9-bbdfaff71d44 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:31.509096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:31.509811 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.510135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.510470 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.511203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:25:31.511707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:31.513785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:31.524920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:31.529319 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4298/8556] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:25:31.534102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb19ad-b69f-48cd-9fd0-23f9d367507c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:31.545190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb19ad-b69f-48cd-9fd0-23f9d367507c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:31.550427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:31.551290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.551582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.551852 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.552591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:31.554665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:31.588414 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4259/8557] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:31 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:31.592688 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:31.607006 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440396b3-3f81-4bcf-a67a-f4354b594a90 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:31.704027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:31.704932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.705211 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.705636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.706492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:31.709000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:31.738408 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4299/8558] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:31 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:31.743573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:31.756137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840a71c-22ad-4f53-81cb-60737a64ba2f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:31.761427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:31.762262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.762569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.762869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.763623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:31.765618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:31.778330 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4260/8559] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:31.782932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:31.794787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ae114-2735-47dd-9eb9-c67abce1820c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:31.800869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8065f-ac50-414f-b5c7-d9fe40d11e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:31.801604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8065f-ac50-414f-b5c7-d9fe40d11e38 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.802024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8065f-ac50-414f-b5c7-d9fe40d11e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.802193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8065f-ac50-414f-b5c7-d9fe40d11e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.802880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8065f-ac50-414f-b5c7-d9fe40d11e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:31.804690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8065f-ac50-414f-b5c7-d9fe40d11e38 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:31.818234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4300/8560] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:31.824013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:31.836547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa58f5-8e7c-4ef3-8669-d2e6af59e9ce admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:31.841749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:31.842533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.842777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.843068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] PATH_INFO: `/v3/projects/6e1785b5f4714605b2a35e03c123d54d/users/23df9a93a08b41f9b3573c1e6a85e60e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.866605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1785b5f4714605b2a35e03c123d54d, user_id=23df9a93a08b41f9b3573c1e6a85e60e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:31.869708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:31.885576 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4261/8561] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:31 2022] PUT /identity/v3/projects/6e1785b5f4714605b2a35e03c123d54d/users/23df9a93a08b41f9b3573c1e6a85e60e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:31.891272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859609af-0966-4750-98d7-fec6b614f73c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:31.891698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859609af-0966-4750-98d7-fec6b614f73c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:31.892234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859609af-0966-4750-98d7-fec6b614f73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:31.917614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-859609af-0966-4750-98d7-fec6b614f73c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:31.918880 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-859609af-0966-4750-98d7-fec6b614f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:32.008346 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4301/8562] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:25:32.022240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:32.024789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:32.103636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109376-a445-4791-b605-efceb3e2c3d7 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:32.108469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93549df6-a287-4f24-b585-a068e5a4f7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:32.109057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549df6-a287-4f24-b585-a068e5a4f7bc service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:32.109327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549df6-a287-4f24-b585-a068e5a4f7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:32.109634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549df6-a287-4f24-b585-a068e5a4f7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:32.122406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93549df6-a287-4f24-b585-a068e5a4f7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:32.124038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93549df6-a287-4f24-b585-a068e5a4f7bc service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:32.135785 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4262/8563] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:25:32.458311 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859609af-0966-4750-98d7-fec6b614f73c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:32.459874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-859609af-0966-4750-98d7-fec6b614f73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:32.473308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859609af-0966-4750-98d7-fec6b614f73c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:32.481106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:32.482283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:32.482774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:32.483013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] PATH_INFO: `/v3/users/01a32874941d4a2982a1e7c3aa5ca26a` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:32.484874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a32874941d4a2982a1e7c3aa5ca26a)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:32.494841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:32.551098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a32874941d4a2982a1e7c3aa5ca26a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:32.551907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a32874941d4a2982a1e7c3aa5ca26a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:32.561905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a32874941d4a2982a1e7c3aa5ca26a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:32.562679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a32874941d4a2982a1e7c3aa5ca26a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:32.578664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a32874941d4a2982a1e7c3aa5ca26a'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:32.581726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:32.597722 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4302/8564] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:32 2022] DELETE /identity/v3/users/01a32874941d4a2982a1e7c3aa5ca26a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:32.602187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:32.602675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:32.603002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:32.628689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:32.630733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:32.730205 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4263/8565] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:32.741548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:32.743423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:32.808437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844748-dc2c-41fa-b30c-cc0002c905c5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:32.813223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a79816-c139-43eb-995b-16cd677e42f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:32.813898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79816-c139-43eb-995b-16cd677e42f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:32.814210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79816-c139-43eb-995b-16cd677e42f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:32.814582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a79816-c139-43eb-995b-16cd677e42f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:32.827649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a79816-c139-43eb-995b-16cd677e42f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:32.829272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a79816-c139-43eb-995b-16cd677e42f6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:32.850642 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4303/8566] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:33.133017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:33.136887 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:33.155544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988f61d-d48e-4553-96e3-32ce06f29049 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:33.163128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:33.164005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.164309 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.164764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] PATH_INFO: `/v3/projects/caee650e216e4b80b4440361a57d2f68` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.166682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=caee650e216e4b80b4440361a57d2f68)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:33.169211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:33.221821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:33.247393 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caee650e216e4b80b4440361a57d2f68', 'request_id': 'req-e44f5ece-b497-47ab-9154-d1877e72da6c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:33.258079 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caee650e216e4b80b4440361a57d2f68', 'request_id': 'req-e44f5ece-b497-47ab-9154-d1877e72da6c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:33.267906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caee650e216e4b80b4440361a57d2f68', 'request_id': 'req-e44f5ece-b497-47ab-9154-d1877e72da6c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:33.285476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] The token cache is being invalidate because project caee650e216e4b80b4440361a57d2f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:33.285932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:33.286573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e44f5ece-b497-47ab-9154-d1877e72da6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:33.292072 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4264/8567] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:33 2022] DELETE /identity/v3/projects/caee650e216e4b80b4440361a57d2f68 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:33.294149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a960f-9800-4acc-bcfc-f997e1ef45d2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.294469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a960f-9800-4acc-bcfc-f997e1ef45d2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.294746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a960f-9800-4acc-bcfc-f997e1ef45d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.330241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6f4a960f-9800-4acc-bcfc-f997e1ef45d2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:33.331073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6f4a960f-9800-4acc-bcfc-f997e1ef45d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:33.428187 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4304/8568] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:25:33.436902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cc10b-ec39-426c-b9d0-ff28c1ccecea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.437269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cc10b-ec39-426c-b9d0-ff28c1ccecea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.437718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cc10b-ec39-426c-b9d0-ff28c1ccecea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.468351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2f1cc10b-ec39-426c-b9d0-ff28c1ccecea None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:33.469821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f1cc10b-ec39-426c-b9d0-ff28c1ccecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:33.480910 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4265/8569] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:33.485418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a960f-9800-4acc-bcfc-f997e1ef45d2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:33.501031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a960f-9800-4acc-bcfc-f997e1ef45d2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:33.508786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:33.509804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.509932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.510192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.511053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:25:33.511391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:33.513289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:33.530961 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4305/8570] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:25:33.532713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cc10b-ec39-426c-b9d0-ff28c1ccecea None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:33.549817 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cc10b-ec39-426c-b9d0-ff28c1ccecea None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:33.557390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:33.558181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.558527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.558903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.559739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:33.562545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:33.607962 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4266/8571] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:33 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:33.612416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:33.625065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4c303-7b2e-463a-85f9-bb82d1268f48 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:33.722919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:33.723551 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.723859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.724203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.725542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:33.728492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:33.758252 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4306/8572] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:33 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:33.764750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:33.778701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de7261-7833-4c33-8feb-aafb18ec3c59 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:33.783602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:33.783602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.785029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.785136 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.785740 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:33.787462 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:33.801536 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4267/8573] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:33.807696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:33.823527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49550343-f91c-49e5-a41f-934bfd26f54f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:33.833073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676f030-ea93-4f70-927e-274e1f5d0682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:33.834901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676f030-ea93-4f70-927e-274e1f5d0682 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.835266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676f030-ea93-4f70-927e-274e1f5d0682 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.836147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676f030-ea93-4f70-927e-274e1f5d0682 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.837614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676f030-ea93-4f70-927e-274e1f5d0682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:33.839995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676f030-ea93-4f70-927e-274e1f5d0682 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:33.868067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:33.884074 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4307/8574] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:33.889089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:33.903210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb5091-a35e-44e8-bbba-1ffed9653342 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:33.915953 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:33.916719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.917022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.917405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] PATH_INFO: `/v3/projects/b19a4be0ea3344db9f129ff3d1a8dea5/users/9a5d6c32c3c9482bb9bd12f3c0a921f9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:33.953095 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19a4be0ea3344db9f129ff3d1a8dea5, user_id=9a5d6c32c3c9482bb9bd12f3c0a921f9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:33.957110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:33.979143 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4268/8575] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:33 2022] PUT /identity/v3/projects/b19a4be0ea3344db9f129ff3d1a8dea5/users/9a5d6c32c3c9482bb9bd12f3c0a921f9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:33.993738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e518d3-70f2-4d7b-b7a1-2bbff18d531f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:33.994026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e518d3-70f2-4d7b-b7a1-2bbff18d531f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:33.994304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e518d3-70f2-4d7b-b7a1-2bbff18d531f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:34.034675 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e6e518d3-70f2-4d7b-b7a1-2bbff18d531f None None] MFA Rules not processed for user `9a5d6c32c3c9482bb9bd12f3c0a921f9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:34.036689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e6e518d3-70f2-4d7b-b7a1-2bbff18d531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:34.177827 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4308/8576] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:34.190447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:34.192521 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:34.284687 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb613-013d-4542-8ae1-48b9c5bc2c5c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:34.288600 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:34.289234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:34.290069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:34.290069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:34.302731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:34.305823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:34.321842 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4269/8577] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:37.530166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87561d-810f-4dd9-b3cb-aa1e4a6831ed None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:37.530679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87561d-810f-4dd9-b3cb-aa1e4a6831ed None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:37.530679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87561d-810f-4dd9-b3cb-aa1e4a6831ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:37.576671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2c87561d-810f-4dd9-b3cb-aa1e4a6831ed None None] MFA Rules not processed for user `23df9a93a08b41f9b3573c1e6a85e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:37.578265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2c87561d-810f-4dd9-b3cb-aa1e4a6831ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:37.683302 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4309/8578] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:37.689343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:37.691948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:37.765616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:37.797637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f5021-5115-489b-a8b6-30cee69352ce service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:37.805048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:37.805869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:37.806155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:37.806433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:37.807729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:37.809757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:37.842368 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4270/8579] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:37 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:37.846953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87561d-810f-4dd9-b3cb-aa1e4a6831ed None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:37.859805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87561d-810f-4dd9-b3cb-aa1e4a6831ed None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:37.967062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:37.967689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:37.967963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:37.968229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:37.969508 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:37.970905 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:37.996881 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4310/8580] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:37 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:38.000227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:38.012840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49a5a-a543-45f9-bf76-5d23e893a03b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:38.018994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:38.019757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:38.020092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:38.020484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:38.021371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:38.023389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:38.035387 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4271/8581] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:38.039611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:38.050310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea44fa-d3a7-4ad3-b539-2122089b32c9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:38.059142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:38.059672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:38.060011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:38.060218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] PATH_INFO: `/v3/projects/533ac10ec2da4eb1a344bd80ff5e1a6b/users/3b37d39e38154ea09af13408812291aa/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:38.064056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:38.082443 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533ac10ec2da4eb1a344bd80ff5e1a6b, user_id=3b37d39e38154ea09af13408812291aa, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:38.085242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:38.119599 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4311/8582] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:38 2022] PUT /identity/v3/projects/533ac10ec2da4eb1a344bd80ff5e1a6b/users/3b37d39e38154ea09af13408812291aa/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:38.123791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:38.136668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc04921-94f2-405f-af35-46296668545f admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:38.204283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:38.204999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:38.205346 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:38.205736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:38.206587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:38.208463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:38.219945 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4272/8583] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:38.223919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:38.234983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d835c-600a-4d19-be44-96796bfc26d2 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:38.243684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bf543-fe6b-43db-aa4c-b4a42e5f2a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:38.244497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bf543-fe6b-43db-aa4c-b4a42e5f2a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:38.244869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bf543-fe6b-43db-aa4c-b4a42e5f2a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:38.245215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bf543-fe6b-43db-aa4c-b4a42e5f2a4e admin admin] PATH_INFO: `/v3/projects/533ac10ec2da4eb1a344bd80ff5e1a6b/users/3b37d39e38154ea09af13408812291aa/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:38.247587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bf543-fe6b-43db-aa4c-b4a42e5f2a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=533ac10ec2da4eb1a344bd80ff5e1a6b, user_id=3b37d39e38154ea09af13408812291aa, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:38.250474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bf543-fe6b-43db-aa4c-b4a42e5f2a4e admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:38.263067 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4312/8584] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:38 2022] PUT /identity/v3/projects/533ac10ec2da4eb1a344bd80ff5e1a6b/users/3b37d39e38154ea09af13408812291aa/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:39.943728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:39.964192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544287b-8ac3-4f93-92a6-38492836e1d8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:40.044090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:40.045285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:40.046382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:40.046733 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] PATH_INFO: `/v3/users/9a5d6c32c3c9482bb9bd12f3c0a921f9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:40.048754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5d6c32c3c9482bb9bd12f3c0a921f9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:40.051836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:40.125145 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5d6c32c3c9482bb9bd12f3c0a921f9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.133118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:40.146306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5d6c32c3c9482bb9bd12f3c0a921f9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.160262 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5d6c32c3c9482bb9bd12f3c0a921f9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.160959 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5d6c32c3c9482bb9bd12f3c0a921f9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.162169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5d6c32c3c9482bb9bd12f3c0a921f9'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.179430 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4273/8585] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:39 2022] DELETE /identity/v3/users/9a5d6c32c3c9482bb9bd12f3c0a921f9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:40.184028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:40.184511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:40.184589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:40.212389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:40.215731 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:40.293655 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4313/8586] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:40.304950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:40.306928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:40.391761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ece54d-ee4d-45d2-a59e-6cbdcfc1c137 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:40.396763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e1184-17ae-4406-8ac8-c7e17703dd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:40.397524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1184-17ae-4406-8ac8-c7e17703dd26 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:40.397920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1184-17ae-4406-8ac8-c7e17703dd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:40.398287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1184-17ae-4406-8ac8-c7e17703dd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:40.430054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e1184-17ae-4406-8ac8-c7e17703dd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:40.432584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e1184-17ae-4406-8ac8-c7e17703dd26 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:40.450931 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4274/8587] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:40.700579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:40.703304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:40.720114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c206039-5958-49b0-a231-0cbfff8c2a1c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:40.725837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:40.726892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:40.727377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:40.727886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] PATH_INFO: `/v3/projects/b19a4be0ea3344db9f129ff3d1a8dea5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:40.729520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19a4be0ea3344db9f129ff3d1a8dea5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:40.733098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:40.890789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19a4be0ea3344db9f129ff3d1a8dea5', 'request_id': 'req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.902885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19a4be0ea3344db9f129ff3d1a8dea5', 'request_id': 'req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.913720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19a4be0ea3344db9f129ff3d1a8dea5', 'request_id': 'req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.930840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] The token cache is being invalidate because project b19a4be0ea3344db9f129ff3d1a8dea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:40.931174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.931772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d7e83653-74b3-4cf0-84f1-d1db4463a6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:40.933897 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4314/8588] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:40 2022] DELETE /identity/v3/projects/b19a4be0ea3344db9f129ff3d1a8dea5 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:40.939006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0db4-ebf6-4c81-80a0-a1696f7ac8f3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:40.939302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0db4-ebf6-4c81-80a0-a1696f7ac8f3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:40.939556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0db4-ebf6-4c81-80a0-a1696f7ac8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:40.974552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-43cf0db4-ebf6-4c81-80a0-a1696f7ac8f3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:40.976096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-43cf0db4-ebf6-4c81-80a0-a1696f7ac8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:41.059782 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4275/8589] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:41.069692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9903-eb18-4dde-9d5e-7b73c7a9aadf None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.070035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9903-eb18-4dde-9d5e-7b73c7a9aadf None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.070331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c9903-eb18-4dde-9d5e-7b73c7a9aadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.097342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8c7c9903-eb18-4dde-9d5e-7b73c7a9aadf None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:41.098848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8c7c9903-eb18-4dde-9d5e-7b73c7a9aadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:41.108907 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4315/8590] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:41.113599 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf0db4-ebf6-4c81-80a0-a1696f7ac8f3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:41.128202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf0db4-ebf6-4c81-80a0-a1696f7ac8f3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:41.139948 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:41.140894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.141320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.141766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.142587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:25:41.143073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:41.145024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:41.161107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4276/8591] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:25:41.165893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c9903-eb18-4dde-9d5e-7b73c7a9aadf None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:41.183136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c9903-eb18-4dde-9d5e-7b73c7a9aadf None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:41.190897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:41.191945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.192376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.192781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.194820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:41.196448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:41.257307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:41.263894 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4316/8592] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:41 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:41.271082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3146fe-4900-42b9-b91c-a9b4cab4a143 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:41.417148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:41.418260 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.420163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.420163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.420163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:41.422933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:41.437397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:41.472527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:41.477385 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4277/8593] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:41 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:41.491243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace4c76-96da-4ce6-b568-01a51d3eb236 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:41.498796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:41.500504 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.500867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.501324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.502324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:41.505650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:41.540185 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4317/8594] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:41.541614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:41.561083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde086e3-41fe-45ce-bc17-78b9f441da71 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:41.566877 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f03da2-e92d-4d2e-a1bb-8817f24e8cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:41.567505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f03da2-e92d-4d2e-a1bb-8817f24e8cdf admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.567810 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f03da2-e92d-4d2e-a1bb-8817f24e8cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.568068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f03da2-e92d-4d2e-a1bb-8817f24e8cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.568707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f03da2-e92d-4d2e-a1bb-8817f24e8cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:41.570615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f03da2-e92d-4d2e-a1bb-8817f24e8cdf admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:41.594081 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4278/8595] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:41.595769 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:41.610573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6429dfa-31d9-4351-b0d4-ae73da461fbd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:41.618437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:41.619548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.619831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.620112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] PATH_INFO: `/v3/projects/2a342178ea70484daae59a79fc0af171/users/f1827c31798c485aa67c8edfae035e1a/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.642207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a342178ea70484daae59a79fc0af171, user_id=f1827c31798c485aa67c8edfae035e1a, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:41.647828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:41.648902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:41.688426 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4318/8596] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:41 2022] PUT /identity/v3/projects/2a342178ea70484daae59a79fc0af171/users/f1827c31798c485aa67c8edfae035e1a/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:41.702183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d7bdc-62d5-4c55-97cd-03f3ef3c5cec None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:41.702561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d7bdc-62d5-4c55-97cd-03f3ef3c5cec None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:41.707604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d7bdc-62d5-4c55-97cd-03f3ef3c5cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:41.765347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-942d7bdc-62d5-4c55-97cd-03f3ef3c5cec None None] MFA Rules not processed for user `f1827c31798c485aa67c8edfae035e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:41.766690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-942d7bdc-62d5-4c55-97cd-03f3ef3c5cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:41.921834 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4279/8597] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:41.940710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:41.942561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:42.062779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a89dbf-7e39-4a93-b224-643178eed832 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:42.068659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db107741-753b-4d95-9181-5288e90b9263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:42.069340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db107741-753b-4d95-9181-5288e90b9263 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:42.069792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db107741-753b-4d95-9181-5288e90b9263 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:42.070063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db107741-753b-4d95-9181-5288e90b9263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:42.079363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db107741-753b-4d95-9181-5288e90b9263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:42.081197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db107741-753b-4d95-9181-5288e90b9263 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:42.092818 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4319/8598] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:42.110767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d7bdc-62d5-4c55-97cd-03f3ef3c5cec None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:42.222849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d7bdc-62d5-4c55-97cd-03f3ef3c5cec None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:42.230349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:42.231356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:42.231672 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:42.231947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] PATH_INFO: `/v3/users/f1827c31798c485aa67c8edfae035e1a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:42.233130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1827c31798c485aa67c8edfae035e1a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:42.235446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:42.321108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1827c31798c485aa67c8edfae035e1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:42.338686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1827c31798c485aa67c8edfae035e1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:42.352352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1827c31798c485aa67c8edfae035e1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:42.353361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1827c31798c485aa67c8edfae035e1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:42.354513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1827c31798c485aa67c8edfae035e1a'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:42.360831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:42.394136 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4280/8599] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:25:42 2022] DELETE /identity/v3/users/f1827c31798c485aa67c8edfae035e1a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:42.396543 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:42.397610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:42.398158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:42.443227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:42.447648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:42.578844 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4320/8600] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:42.590264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:42.594349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:42.743495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed43966-0887-4823-9f60-ab4e692e0899 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:42.747814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d82562-85f8-418d-8ff5-034250e9cd3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:42.748559 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d82562-85f8-418d-8ff5-034250e9cd3e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:42.749647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d82562-85f8-418d-8ff5-034250e9cd3e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:42.749647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d82562-85f8-418d-8ff5-034250e9cd3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:42.763365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d82562-85f8-418d-8ff5-034250e9cd3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:42.765064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d82562-85f8-418d-8ff5-034250e9cd3e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:42.782157 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4281/8601] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:43.261957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:43.266707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:43.293643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3929d2-60f5-4d46-921b-f8f5d827dec3 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:43.300134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:43.300927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:43.301276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:43.301666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] PATH_INFO: `/v3/projects/2a342178ea70484daae59a79fc0af171` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:43.302879 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a342178ea70484daae59a79fc0af171)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:43.304984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:43.383972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a342178ea70484daae59a79fc0af171', 'request_id': 'req-0e90342d-174f-40ea-bbe5-daae117b3cef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:43.401498 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a342178ea70484daae59a79fc0af171', 'request_id': 'req-0e90342d-174f-40ea-bbe5-daae117b3cef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:43.412536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a342178ea70484daae59a79fc0af171', 'request_id': 'req-0e90342d-174f-40ea-bbe5-daae117b3cef'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:43.429437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] The token cache is being invalidate because project 2a342178ea70484daae59a79fc0af171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:25:43.429876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:43.430948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-0e90342d-174f-40ea-bbe5-daae117b3cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:25:43.432695 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4321/8602] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:25:43 2022] DELETE /identity/v3/projects/2a342178ea70484daae59a79fc0af171 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:43.438342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602f7c-4ea2-4f98-9e7c-4825b8d37c7d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:43.438777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602f7c-4ea2-4f98-9e7c-4825b8d37c7d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:43.438913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602f7c-4ea2-4f98-9e7c-4825b8d37c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:43.468941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ee602f7c-4ea2-4f98-9e7c-4825b8d37c7d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:43.470840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ee602f7c-4ea2-4f98-9e7c-4825b8d37c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:43.579444 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4282/8603] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:43.581821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441951-697b-4e76-b149-2a318bbf3ea8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:43.582258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441951-697b-4e76-b149-2a318bbf3ea8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:43.582645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441951-697b-4e76-b149-2a318bbf3ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:43.617874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-61441951-697b-4e76-b149-2a318bbf3ea8 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:43.621927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-61441951-697b-4e76-b149-2a318bbf3ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:43.635286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4322/8604] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:43.639347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602f7c-4ea2-4f98-9e7c-4825b8d37c7d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:43.650814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee602f7c-4ea2-4f98-9e7c-4825b8d37c7d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:43.660273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:43.661263 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:43.661567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:43.661838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:43.662545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:25:43.662815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:43.664522 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:43.706212 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4283/8605] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:25:43.710382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61441951-697b-4e76-b149-2a318bbf3ea8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:43.722659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61441951-697b-4e76-b149-2a318bbf3ea8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:43.728640 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:43.729507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:43.729938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:43.730417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:43.731466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:43.733768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:43.755371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:43.801307 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4323/8606] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:43 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:43.806484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:43.822460 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d162d04-8739-4f05-ba7b-a10fd5e2ff60 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:43.924952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:43.925679 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:43.925981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:43.926291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:43.926992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:43.929394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:43.961241 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4284/8607] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:43 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:43.965560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:43.980831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c6c71-3bbe-4940-b37b-a18944a78b63 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:43.987813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:43.988558 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:43.988940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:43.989306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:43.990457 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:43.992547 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:44.039703 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4324/8608] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:44.048717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:44.056320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372b5da-d4a4-4098-b338-15cbd8e261b2 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:44.062024 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a38fd-543c-4f1a-89ed-a22dc940ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:44.063040 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a38fd-543c-4f1a-89ed-a22dc940ef43 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:44.063310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a38fd-543c-4f1a-89ed-a22dc940ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:44.063680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a38fd-543c-4f1a-89ed-a22dc940ef43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:44.064447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a38fd-543c-4f1a-89ed-a22dc940ef43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:44.066364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a38fd-543c-4f1a-89ed-a22dc940ef43 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:44.074244 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:44.082789 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4285/8609] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:25:44.087267 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:44.102167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:44.102167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:44.102426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:44.105342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d979c77-5dcb-49d5-b98b-e33bffdb8494 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:44.112141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abd2e6-1a38-4dbe-8216-4694efee37c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:44.113071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abd2e6-1a38-4dbe-8216-4694efee37c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:44.113494 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abd2e6-1a38-4dbe-8216-4694efee37c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:44.114545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abd2e6-1a38-4dbe-8216-4694efee37c3 admin admin] PATH_INFO: `/v3/projects/8463c9508c0c462f9ed16bea93ac6bbf/users/6985783dd5d04ace8dfe118aaf4877d7/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:44.141585 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] MFA Rules not processed for user `3b37d39e38154ea09af13408812291aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:44.142348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abd2e6-1a38-4dbe-8216-4694efee37c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8463c9508c0c462f9ed16bea93ac6bbf, user_id=6985783dd5d04ace8dfe118aaf4877d7, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:44.143023 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:44.145100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abd2e6-1a38-4dbe-8216-4694efee37c3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:44.162845 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4325/8610] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:44 2022] PUT /identity/v3/projects/8463c9508c0c462f9ed16bea93ac6bbf/users/6985783dd5d04ace8dfe118aaf4877d7/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:44.168720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:44.169185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:44.169596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:44.194567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:44.195736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:44.256317 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4286/8611] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:25:44.274266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:44.277397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:44.280613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4326/8612] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:44.291943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:44.293641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:44.367909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07ab38-639e-4853-b333-90d2a161183b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:44.380379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945057e0-9d81-4d2f-9850-1d0b02421dd4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:44.387975 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725e5f3-9e77-42b2-b2fe-d922099d5f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:44.389398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e5f3-9e77-42b2-b2fe-d922099d5f93 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:44.389398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e5f3-9e77-42b2-b2fe-d922099d5f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:44.389398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725e5f3-9e77-42b2-b2fe-d922099d5f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:44.402954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725e5f3-9e77-42b2-b2fe-d922099d5f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:44.405409 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725e5f3-9e77-42b2-b2fe-d922099d5f93 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:44.420635 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4327/8613] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:44.446526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:44.447324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:44.447650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:44.447972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:44.450503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:44.545505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:44.554257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:44.556022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:44.568033 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4287/8614] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:50.039648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0a33e-8a8e-473c-9643-5605a119d9d2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:50.040169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0a33e-8a8e-473c-9643-5605a119d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:50.040169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0a33e-8a8e-473c-9643-5605a119d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:50.100808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9ec0a33e-8a8e-473c-9643-5605a119d9d2 None None] MFA Rules not processed for user `6985783dd5d04ace8dfe118aaf4877d7`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:50.102293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9ec0a33e-8a8e-473c-9643-5605a119d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:50.212198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:50.235594 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4328/8615] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:50.240688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:50.252097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c7c64-2c26-44a1-a1a5-ca518154d574 service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:50.258275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:50.258960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:50.259942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:50.260232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:50.260884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:50.263201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:50.315406 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4288/8616] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:25:50 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:50.319389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0a33e-8a8e-473c-9643-5605a119d9d2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:50.331347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0a33e-8a8e-473c-9643-5605a119d9d2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:50.451528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:50.452601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:50.453124 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:50.453590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:50.454932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:50.457352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:50.500486 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4329/8617] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:25:50 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:25:50.506714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:50.527632 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f918ebf-6e5c-49c6-a652-32b8ed3a16ab admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:50.538085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:50.539092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:50.540378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:50.541338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:50.546012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:50.550693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:50.597534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:50.612131 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4289/8618] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:25:50.621525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0303fc1-8e15-473e-a649-d4c6e22f33fa admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:50.641907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:50.642757 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:50.643073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:50.643352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] PATH_INFO: `/v3/projects/805eeaa4d97648388104f07f75e9ffb1/users/5c55fb7bb6fc49d0981c73d97fc97cdd/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:50.693235 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805eeaa4d97648388104f07f75e9ffb1, user_id=5c55fb7bb6fc49d0981c73d97fc97cdd, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:50.703755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:50.755335 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4330/8619] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:50 2022] PUT /identity/v3/projects/805eeaa4d97648388104f07f75e9ffb1/users/5c55fb7bb6fc49d0981c73d97fc97cdd/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:25:50.759219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:50.798166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da4e15-697c-4610-930b-88883a30bfbd admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:50.921863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:50.921863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:50.921863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:50.922290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:50.922767 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:50.924551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:50.961494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4290/8620] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:25:50.966512 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:51.015795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991fb18-5a6c-495a-9439-39a66d9ba6f4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:51.028356 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a45f9a-369e-4e2d-9b22-3850c2041b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:51.029168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a45f9a-369e-4e2d-9b22-3850c2041b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:51.029556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a45f9a-369e-4e2d-9b22-3850c2041b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:51.029895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a45f9a-369e-4e2d-9b22-3850c2041b7a admin admin] PATH_INFO: `/v3/projects/805eeaa4d97648388104f07f75e9ffb1/users/5c55fb7bb6fc49d0981c73d97fc97cdd/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:51.038029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a45f9a-369e-4e2d-9b22-3850c2041b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=805eeaa4d97648388104f07f75e9ffb1, user_id=5c55fb7bb6fc49d0981c73d97fc97cdd, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:51.041374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a45f9a-369e-4e2d-9b22-3850c2041b7a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:51.087068 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4331/8621] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:25:50 2022] PUT /identity/v3/projects/805eeaa4d97648388104f07f75e9ffb1/users/5c55fb7bb6fc49d0981c73d97fc97cdd/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:25:54.618070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:54.834750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8eb69-3141-41d7-8dab-08268680130a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:55.011535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:55.012942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:55.013315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:55.013686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:55.016453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:55.243131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:55.357150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:55.358919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:55.401866 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4291/8622] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:58.643221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6a49-8c7a-4c56-a828-57f72f214ffd None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:58.643891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6a49-8c7a-4c56-a828-57f72f214ffd None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:58.644179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6a49-8c7a-4c56-a828-57f72f214ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:58.691680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a54d6a49-8c7a-4c56-a828-57f72f214ffd None None] MFA Rules not processed for user `5c55fb7bb6fc49d0981c73d97fc97cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:58.696534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a54d6a49-8c7a-4c56-a828-57f72f214ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:58.811285 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4332/8623] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:58.825304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:58.829631 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:58.849905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b64fe-b663-4b83-926b-b3b38582cf67 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:58.859135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:58.860190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:58.860534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:58.860922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:58.872484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:58.875013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:58.887247 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4292/8624] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:59.087890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:59.088173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:59.088624 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:59.115429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:25:59.116756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:59.138417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:59.188791 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4333/8625] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:25:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:25:59.213644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:59.215174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:59.258287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:25:59.259863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:59.316223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4033-040e-4897-82dd-42a5249f77cb service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:59.321734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7239d14-ef5f-4a90-b640-02d0da566949 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:59.322487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7239d14-ef5f-4a90-b640-02d0da566949 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:59.322651 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7239d14-ef5f-4a90-b640-02d0da566949 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:59.322870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7239d14-ef5f-4a90-b640-02d0da566949 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:59.324659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7239d14-ef5f-4a90-b640-02d0da566949 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:25:59.341701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7239d14-ef5f-4a90-b640-02d0da566949 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:59.343714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7239d14-ef5f-4a90-b640-02d0da566949 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:59.345004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7598e1-eb2b-47bb-a0b6-95f356339994 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:25:59.350440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:25:59.351221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:25:59.351506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:25:59.351782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:25:59.361071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:25:59.361788 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4293/8626] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:25:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:25:59.362810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:25:59.371990 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4334/8627] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:25:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:26:05.627028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6880b8-d10c-42a3-bd12-a2a6c48030a7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:05.627935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6880b8-d10c-42a3-bd12-a2a6c48030a7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:05.628208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6880b8-d10c-42a3-bd12-a2a6c48030a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:05.652177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6c6880b8-d10c-42a3-bd12-a2a6c48030a7 None None] MFA Rules not processed for user `342e60103c9b45bcb348b6f8bcd686ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:26:05.653872 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6c6880b8-d10c-42a3-bd12-a2a6c48030a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:05.762631 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4294/8628] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:26:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:26:06.768610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:06.782545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c3cec-4558-4ec4-8c6c-caac6d82703f service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:06.788714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:06.789329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:06.789831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:06.790263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:06.793538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:06.812601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:06.814343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:06.830755 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4335/8629] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:08.802759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf318c45-dbce-4cba-a59d-7bf6c14ee72f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:08.803827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf318c45-dbce-4cba-a59d-7bf6c14ee72f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:08.804159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf318c45-dbce-4cba-a59d-7bf6c14ee72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:08.828100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bf318c45-dbce-4cba-a59d-7bf6c14ee72f None None] MFA Rules not processed for user `23df9a93a08b41f9b3573c1e6a85e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:26:08.829685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bf318c45-dbce-4cba-a59d-7bf6c14ee72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:08.867209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:08.923469 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4295/8630] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:26:11.106529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:11.169821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31171491-c75f-44f4-8294-f5620113d542 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:11.175352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:11.175994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:11.176270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:11.176539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] PATH_INFO: `/v3/users/6985783dd5d04ace8dfe118aaf4877d7` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:11.177825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6985783dd5d04ace8dfe118aaf4877d7)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:11.179809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:11.229886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6985783dd5d04ace8dfe118aaf4877d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.230604 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6985783dd5d04ace8dfe118aaf4877d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.232966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:11.242172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6985783dd5d04ace8dfe118aaf4877d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.243247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6985783dd5d04ace8dfe118aaf4877d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.255566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6985783dd5d04ace8dfe118aaf4877d7'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.268491 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4336/8631] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:26:11 2022] DELETE /identity/v3/users/6985783dd5d04ace8dfe118aaf4877d7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:26:11.460568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf318c45-dbce-4cba-a59d-7bf6c14ee72f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:11.476243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf318c45-dbce-4cba-a59d-7bf6c14ee72f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:11.575781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:11.576471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:11.576750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:11.577025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] PATH_INFO: `/v3/projects/8463c9508c0c462f9ed16bea93ac6bbf` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:11.583076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8463c9508c0c462f9ed16bea93ac6bbf)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:11.585105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:11.685399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8463c9508c0c462f9ed16bea93ac6bbf', 'request_id': 'req-a19b3cb9-d659-4658-a176-eff2317b6172'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.700747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8463c9508c0c462f9ed16bea93ac6bbf', 'request_id': 'req-a19b3cb9-d659-4658-a176-eff2317b6172'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.708178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8463c9508c0c462f9ed16bea93ac6bbf', 'request_id': 'req-a19b3cb9-d659-4658-a176-eff2317b6172'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.725509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] The token cache is being invalidate because project 8463c9508c0c462f9ed16bea93ac6bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:26:11.725509 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.729206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.729206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4296/8632] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:26:11 2022] DELETE /identity/v3/projects/8463c9508c0c462f9ed16bea93ac6bbf => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:26:11.733353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:11.735414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:11.752061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0d9d9-fc1c-4c05-95b7-0c452f451e8c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:11.823312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:11.824133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:11.824638 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:11.824868 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] PATH_INFO: `/v3/users/5c55fb7bb6fc49d0981c73d97fc97cdd` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:11.826158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c55fb7bb6fc49d0981c73d97fc97cdd)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:11.828316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:11.908985 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c55fb7bb6fc49d0981c73d97fc97cdd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.909720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c55fb7bb6fc49d0981c73d97fc97cdd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.920874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c55fb7bb6fc49d0981c73d97fc97cdd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.922312 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c55fb7bb6fc49d0981c73d97fc97cdd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.939699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6867487f-07d7-431a-a552-8f5975fab314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c55fb7bb6fc49d0981c73d97fc97cdd'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:11.953829 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4337/8633] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:26:11 2022] DELETE /identity/v3/users/5c55fb7bb6fc49d0981c73d97fc97cdd => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:26:12.231055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:12.233598 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:12.256671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b3cb9-d659-4658-a176-eff2317b6172 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:12.334503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:12.336165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.336674 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.337235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] PATH_INFO: `/v3/projects/805eeaa4d97648388104f07f75e9ffb1` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.339568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805eeaa4d97648388104f07f75e9ffb1)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:12.343080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:12.363043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:12.404374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805eeaa4d97648388104f07f75e9ffb1', 'request_id': 'req-1bffb274-b148-471d-ae0e-0bdab5556636'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:12.416707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805eeaa4d97648388104f07f75e9ffb1', 'request_id': 'req-1bffb274-b148-471d-ae0e-0bdab5556636'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:12.430454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805eeaa4d97648388104f07f75e9ffb1', 'request_id': 'req-1bffb274-b148-471d-ae0e-0bdab5556636'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:12.444329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] The token cache is being invalidate because project 805eeaa4d97648388104f07f75e9ffb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:26:12.444640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:12.445372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1bffb274-b148-471d-ae0e-0bdab5556636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:26:12.446932 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4297/8634] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:26:12 2022] DELETE /identity/v3/projects/805eeaa4d97648388104f07f75e9ffb1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:26:12.452216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.452756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.452756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.475635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:26:12.477007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:12.556782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:12.587594 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4338/8635] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:26:12.595875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4933612-15ec-4073-b1a1-84025a76fd3a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.597126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4933612-15ec-4073-b1a1-84025a76fd3a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.597380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4933612-15ec-4073-b1a1-84025a76fd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.615665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:12.618981 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:12.625278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a4933612-15ec-4073-b1a1-84025a76fd3a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:26:12.626665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a4933612-15ec-4073-b1a1-84025a76fd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:12.636994 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4298/8636] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:26:12.646717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4933612-15ec-4073-b1a1-84025a76fd3a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:12.666714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4933612-15ec-4073-b1a1-84025a76fd3a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:12.694981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:12.699630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.700454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.700803 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.701985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:26:12.703323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:12.714231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:12.715809 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d05e6-b30c-47b0-90ab-8eda8601618c None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:12.730269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:12.731220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.731220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.731440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.736444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:12.745557 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4299/8637] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:26:12.750796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:12.760082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:12.761914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:12.774496 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4339/8638] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:12.779586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c7a88-81ff-4e49-8c64-9d9b4900a46e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:12.787424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:12.793918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.794428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.795157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.796987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:12.799644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:12.845314 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4300/8639] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:26:12 2022] POST /identity/v3/projects => generated 393 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:26:12.846713 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:12.858598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de552644-67ac-4fc7-82d8-c3989384123f service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:12.946707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:12.947682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.948056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.948397 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.949111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:12.951185 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:12.975869 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4340/8640] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:26:12 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:26:12.980059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:12.991746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe72ffa-1a14-49f0-896c-32018d000e74 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:12.996497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:12.997118 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:12.997397 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:12.997696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:12.998497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.000127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.014669 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4301/8641] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:26:13.016050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.027569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dfff8-0422-4f20-b66f-026101953d98 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.033206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260f69e-09c7-481c-a0a9-ac940aa45f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.033902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260f69e-09c7-481c-a0a9-ac940aa45f56 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.034183 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260f69e-09c7-481c-a0a9-ac940aa45f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.034455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260f69e-09c7-481c-a0a9-ac940aa45f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.035123 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260f69e-09c7-481c-a0a9-ac940aa45f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.036776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260f69e-09c7-481c-a0a9-ac940aa45f56 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.048545 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4341/8642] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:26:13.063372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.075415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5f572-a944-4f4b-926d-2d13f63c0abb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.080967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.081668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.081952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.082224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] PATH_INFO: `/v3/projects/b67de414e1aa4fde9ed87336f30c2767/users/3cb55865ecad4c05adc52a1aacd2aa10/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.100750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67de414e1aa4fde9ed87336f30c2767, user_id=3cb55865ecad4c05adc52a1aacd2aa10, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.103422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.121518 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4302/8643] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:26:13 2022] PUT /identity/v3/projects/b67de414e1aa4fde9ed87336f30c2767/users/3cb55865ecad4c05adc52a1aacd2aa10/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:26:13.132346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c470d-0f55-4ae1-b062-961313b77b37 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.132664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c470d-0f55-4ae1-b062-961313b77b37 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.132903 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c470d-0f55-4ae1-b062-961313b77b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.170168 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-660c470d-0f55-4ae1-b062-961313b77b37 None None] MFA Rules not processed for user `3cb55865ecad4c05adc52a1aacd2aa10`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:26:13.171486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-660c470d-0f55-4ae1-b062-961313b77b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:13.247178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:13.262861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4342/8644] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:26:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:26:13.263935 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.329020 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347465a-afa7-4501-9f1e-8011f5a99de3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.334385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.335147 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.335437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.335692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.336486 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.338496 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.374219 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4303/8645] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:26:13 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:26:13.375465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c470d-0f55-4ae1-b062-961313b77b37 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.387097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c470d-0f55-4ae1-b062-961313b77b37 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.449317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.450532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.450849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.451227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.451896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.454241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.478781 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4343/8646] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:26:13 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:26:13.482848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.495535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5535f-65ea-40b7-bbf1-92698b5f3f59 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.501219 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.501988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.502272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.502547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.503203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.504835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.511155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:13.516415 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4304/8647] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:26:13.521306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.533647 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2f6ad-8582-49cb-87de-1a35730a4339 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.538390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.539052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.539326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.539597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] PATH_INFO: `/v3/projects/f94968a8495d447d9adf019219816f9c/users/c2f8e397465f45a08b807d5240c07729/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.558893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94968a8495d447d9adf019219816f9c, user_id=c2f8e397465f45a08b807d5240c07729, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.563822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.585511 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4344/8648] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:26:13 2022] PUT /identity/v3/projects/f94968a8495d447d9adf019219816f9c/users/c2f8e397465f45a08b807d5240c07729/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:26:13.586748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.600132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3e653-03f0-42c4-953f-572c07dba991 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.667573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd6f77-2a3e-4211-9a88-57bcd584a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.668916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd6f77-2a3e-4211-9a88-57bcd584a854 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.669382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd6f77-2a3e-4211-9a88-57bcd584a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.669816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd6f77-2a3e-4211-9a88-57bcd584a854 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.670579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd6f77-2a3e-4211-9a88-57bcd584a854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.673004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd6f77-2a3e-4211-9a88-57bcd584a854 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.688329 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4305/8649] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:26:13.693657 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.708300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccaf79-09a2-473a-aade-ab799a5dc3ea admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.713890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.714634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.714984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.715319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] PATH_INFO: `/v3/projects/f94968a8495d447d9adf019219816f9c/users/c2f8e397465f45a08b807d5240c07729/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.717454 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94968a8495d447d9adf019219816f9c, user_id=c2f8e397465f45a08b807d5240c07729, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:13.720226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:13.743607 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4345/8650] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:26:13 2022] PUT /identity/v3/projects/f94968a8495d447d9adf019219816f9c/users/c2f8e397465f45a08b807d5240c07729/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:26:13.744992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.745256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.745554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:13.778319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] MFA Rules not processed for user `c2f8e397465f45a08b807d5240c07729`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:26:13.779659 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:13.877600 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4306/8651] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:26:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:26:13.889887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:13.891747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:13.977214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6a354-93e7-4707-bd51-05ae500262da admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:13.983456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:13.984375 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:13.984621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:13.984979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:14.053845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:14.072425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:14.081862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:14.095455 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4346/8652] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:17.584149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:17.585664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:17.650570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242c297-272a-49ba-8bac-f0552649bb1a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:17.654319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3f2d8-93a0-48b6-83cd-99c736700618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:17.654952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f2d8-93a0-48b6-83cd-99c736700618 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:17.655215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f2d8-93a0-48b6-83cd-99c736700618 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:17.655553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3f2d8-93a0-48b6-83cd-99c736700618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:17.666119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f2d8-93a0-48b6-83cd-99c736700618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:17.667915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3f2d8-93a0-48b6-83cd-99c736700618 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:17.679403 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4307/8653] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:18.440887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:18.453250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07936d4-7701-46ce-a0c1-ad69c9655686 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:18.457095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:18.458133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:18.458133 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:18.458351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:18.460032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:18.539955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:18.542955 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:18.569747 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4347/8654] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:21.650849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87336d56-6634-446f-b5d3-5261915d7d04 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:21.650849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87336d56-6634-446f-b5d3-5261915d7d04 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:21.650849 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87336d56-6634-446f-b5d3-5261915d7d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:21.711811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-87336d56-6634-446f-b5d3-5261915d7d04 None None] MFA Rules not processed for user `3cb55865ecad4c05adc52a1aacd2aa10`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:26:21.713511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-87336d56-6634-446f-b5d3-5261915d7d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:21.732209 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4308/8655] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:26:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:26:22.149289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:22.161245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586070e3-018a-4061-911f-db80b434ad48 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:22.167906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:22.169247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:22.170398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:22.170695 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:22.173132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:22.188083 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:22.189716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:22.203020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4348/8656] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:22.565707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87336d56-6634-446f-b5d3-5261915d7d04 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:22.580437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87336d56-6634-446f-b5d3-5261915d7d04 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:22.589246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:22.589962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:22.590211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:22.590476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:22.592253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:22.651376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:22.766910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:22.770504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:22.783178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4309/8657] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:25.309478 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:25.311096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:25.332207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683181b1-f635-4e0c-ba05-dc0b29ac0621 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:25.343344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:25.344519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:25.345145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:25.351232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:25.353587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:25.381169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:25.383115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:25.401613 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4349/8658] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:32.776582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=22 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:35.394089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=49 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:42.770259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:26:42.783550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713196fa-91cc-4b82-be48-89abb7ba47a1 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:26:42.787837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:26:42.788667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:26:42.789012 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:26:42.789252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:26:42.791540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:26:42.806290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:26:42.807818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:26:42.820720 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4310/8659] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:26:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:26:52.813378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:11.531886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:11.533562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:11.611915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daaa32-568a-42a6-803e-5fc5d26e9926 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:11.617683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:11.618291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:11.618575 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:11.618845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:11.620682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:11.634845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:11.636426 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:11.649163 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4350/8660] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:27:11.815707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:11.839301 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181d77d-aaab-45e2-b2a3-857fd00b1b18 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:11.843419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:11.844049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:11.844325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:11.844595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:11.846630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:11.864602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:11.864602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:11.873514 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4311/8661] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:27:12.013537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:12.015434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:12.043212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947eb4e-ba27-4429-96ae-5a311f536a69 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:12.049670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:12.050301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:12.050584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:12.051042 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:12.053514 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:12.074169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:12.075893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:12.094924 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4351/8662] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:27:12.317083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:12.330127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a40af-23dd-4f50-9c07-b3b3f2558862 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:12.334134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:12.334735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:12.335008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:12.335278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:12.337016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:12.353028 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:12.354540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:12.368805 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4312/8663] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:27:18.446953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:18.463009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b4d1-95be-4880-8b1f-5dd9b9034a6c service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:18.472089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:18.472773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:18.473052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:18.473323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:18.476631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:18.495144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:18.498893 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:18.515176 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4352/8664] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:27:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:27:22.172773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:22.174520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:22.245154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757e963-8d09-453d-a5f2-891ca344c9b9 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:22.250575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:22.251237 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:22.251517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:22.251787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] PATH_INFO: `/v3/users/23df9a93a08b41f9b3573c1e6a85e60e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:22.252938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23df9a93a08b41f9b3573c1e6a85e60e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:22.255008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:22.308416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23df9a93a08b41f9b3573c1e6a85e60e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.321360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23df9a93a08b41f9b3573c1e6a85e60e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.331206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23df9a93a08b41f9b3573c1e6a85e60e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.332055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23df9a93a08b41f9b3573c1e6a85e60e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.332875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23df9a93a08b41f9b3573c1e6a85e60e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.346742 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4313/8665] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:27:22 2022] DELETE /identity/v3/users/23df9a93a08b41f9b3573c1e6a85e60e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:27:22.522971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:22.536613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5eff8-e7a8-4b8a-9aa9-3e103f39a0db service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:22.599120 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:22.599689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:22.599967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:22.600239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] PATH_INFO: `/v3/projects/6e1785b5f4714605b2a35e03c123d54d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:22.601541 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1785b5f4714605b2a35e03c123d54d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:22.603553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:22.651586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1785b5f4714605b2a35e03c123d54d', 'request_id': 'req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.657838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1785b5f4714605b2a35e03c123d54d', 'request_id': 'req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.667059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1785b5f4714605b2a35e03c123d54d', 'request_id': 'req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.671429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:22.680464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] The token cache is being invalidate because project 6e1785b5f4714605b2a35e03c123d54d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:27:22.681036 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.682208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.683848 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4353/8666] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:27:22 2022] DELETE /identity/v3/projects/6e1785b5f4714605b2a35e03c123d54d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:27:22.688075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:22.689719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:22.704536 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162f0f2-37df-48fb-92e5-4bdf26f0d8a9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:22.752881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:22.763245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:22.763989 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:22.764304 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:22.764637 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] PATH_INFO: `/v3/users/3b37d39e38154ea09af13408812291aa` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:22.766098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b37d39e38154ea09af13408812291aa)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:22.768343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:22.822170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b37d39e38154ea09af13408812291aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.836373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b37d39e38154ea09af13408812291aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.845774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b37d39e38154ea09af13408812291aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.846442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b37d39e38154ea09af13408812291aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.847066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b37d39e38154ea09af13408812291aa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:22.862916 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4314/8667] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:27:22 2022] DELETE /identity/v3/users/3b37d39e38154ea09af13408812291aa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:27:23.168596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:23.170418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:23.186176 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea03bbc-98fe-4afe-aa61-03c6a4c30fc5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:23.249134 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:23.249812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:23.250096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:23.250370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] PATH_INFO: `/v3/projects/533ac10ec2da4eb1a344bd80ff5e1a6b` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:23.251599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533ac10ec2da4eb1a344bd80ff5e1a6b)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:23.253627 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:23.305056 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533ac10ec2da4eb1a344bd80ff5e1a6b', 'request_id': 'req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:23.311393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533ac10ec2da4eb1a344bd80ff5e1a6b', 'request_id': 'req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:23.321110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533ac10ec2da4eb1a344bd80ff5e1a6b', 'request_id': 'req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:23.332349 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] The token cache is being invalidate because project 533ac10ec2da4eb1a344bd80ff5e1a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:27:23.332636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:23.333258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:27:23.334827 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4354/8668] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:27:23 2022] DELETE /identity/v3/projects/533ac10ec2da4eb1a344bd80ff5e1a6b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:27:32.159030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:27:32.160603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:32.240939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173c7b-5a95-4676-a0f4-3c56702c6b90 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:27:32.306597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99907e-a179-4a9f-8bdb-bf0ba2f65a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:27:32.307208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99907e-a179-4a9f-8bdb-bf0ba2f65a5c service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:27:32.307484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99907e-a179-4a9f-8bdb-bf0ba2f65a5c service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:27:32.307753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99907e-a179-4a9f-8bdb-bf0ba2f65a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:27:32.310162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5e99907e-a179-4a9f-8bdb-bf0ba2f65a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:27:32.320258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:32.325579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99907e-a179-4a9f-8bdb-bf0ba2f65a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:27:32.327896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99907e-a179-4a9f-8bdb-bf0ba2f65a5c service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:27:32.342709 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4315/8669] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:27:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:27:33.329294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:42.336979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:03.064011 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:28:03.065439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:03.084106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2eb33-3b52-4d1d-ad98-61c2c8f782b3 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:28:03.091057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:28:03.092034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:28:03.092320 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:28:03.093151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:28:03.095353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:03.112305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:28:03.114210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:28:03.128577 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4355/8670] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:28:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:28:13.122994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:15.224129 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb3855-d543-4cfa-bbe1-c4530298aa85 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:28:15.224627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb3855-d543-4cfa-bbe1-c4530298aa85 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:28:15.224726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb3855-d543-4cfa-bbe1-c4530298aa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:28:15.251753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-69eb3855-d543-4cfa-bbe1-c4530298aa85 None None] MFA Rules not processed for user `3cb55865ecad4c05adc52a1aacd2aa10`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:28:15.252985 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-69eb3855-d543-4cfa-bbe1-c4530298aa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:15.325591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4316/8671] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:28:16.128663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:28:16.130283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:16.224777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e5409-4264-4e24-b7df-86724dc526c1 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:28:16.230615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:28:16.231248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:28:16.231516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:28:16.231789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:28:16.233686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:16.252783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:28:16.254471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:28:16.268569 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4356/8672] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:28:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:28:25.313697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=25 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:26.261440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=32 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:30.014954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:28:30.015435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:28:30.015531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:28:30.037148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] MFA Rules not processed for user `342e60103c9b45bcb348b6f8bcd686ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:28:30.038670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:30.105489 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4317/8673] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:28:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:28:30.442996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:28:30.445055 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:30.511456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c2c68-d1c5-4d97-90af-59e4e29c0a98 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:28:30.517114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:28:30.518034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:28:30.518326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:28:30.518666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:28:30.521789 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:30.540613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:28:30.542560 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:28:30.553936 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4357/8674] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:28:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:28:40.095649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=25 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:40.546989 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=26 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:52.221142 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:28:52.222763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:52.255416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d09899-3e54-4b35-90ad-bef788a9159c None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:28:52.261070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:28:52.261761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:28:52.262064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:28:52.262367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:28:52.264713 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:52.281090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:28:52.282816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:28:52.296030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4318/8675] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:28:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:28:53.102295 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:28:53.117977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376728cf-36ae-400f-88f5-5ac9d53ca124 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:28:53.123549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:28:53.124158 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:28:53.124442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:28:53.124747 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:28:53.126639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:28:53.141894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:28:53.143576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:28:53.157746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4358/8676] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:28:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:29:02.289181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=11 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:29:02.521224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:29:02.534354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17744f-36fe-421c-9c5c-7d00e8c91462 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:29:02.538644 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa533622-7fea-43dc-9879-c932c8ee2126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:29:02.539352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa533622-7fea-43dc-9879-c932c8ee2126 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:29:02.539566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa533622-7fea-43dc-9879-c932c8ee2126 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:29:02.539835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa533622-7fea-43dc-9879-c932c8ee2126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:29:02.541827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa533622-7fea-43dc-9879-c932c8ee2126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:29:02.555030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa533622-7fea-43dc-9879-c932c8ee2126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:29:02.556452 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa533622-7fea-43dc-9879-c932c8ee2126 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:29:02.567552 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4319/8677] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:29:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:29:02.870500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:29:02.883656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051a607-f34b-45ca-a3ae-fbc681daa0a0 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:29:02.888373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:29:02.889037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:29:02.889423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:29:02.889715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:29:02.891753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:29:02.920595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:29:02.922245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:29:02.936787 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4359/8678] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:29:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:29:12.561455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:29:12.929254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=15 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:30:16.471126 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6b652-d7b8-45bf-b038-98ae49219623 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:30:16.471666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6b652-d7b8-45bf-b038-98ae49219623 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:30:16.471666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6b652-d7b8-45bf-b038-98ae49219623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:30:16.491914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f7d6b652-d7b8-45bf-b038-98ae49219623 None None] MFA Rules not processed for user `3cb55865ecad4c05adc52a1aacd2aa10`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:30:16.493165 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f7d6b652-d7b8-45bf-b038-98ae49219623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:30:16.500354 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4320/8679] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:30:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:30:24.646729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:30:24.648323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:30:24.722522 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce89e74-94d3-4c37-b145-c9a8ac954b2b service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:30:24.726549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:30:24.727153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:30:24.727413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:30:24.727685 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:30:24.729493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:30:24.740800 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:30:24.742240 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:30:24.751541 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4360/8680] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:30:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:30:26.486502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:30:27.476881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acef455-3673-4e04-995d-12f2b9997571 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:30:27.477369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acef455-3673-4e04-995d-12f2b9997571 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:30:27.477667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acef455-3673-4e04-995d-12f2b9997571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:30:27.498207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3acef455-3673-4e04-995d-12f2b9997571 None None] MFA Rules not processed for user `342e60103c9b45bcb348b6f8bcd686ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:30:27.499412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3acef455-3673-4e04-995d-12f2b9997571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:30:27.507635 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4321/8681] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:30:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:30:34.748035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=26 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:30:37.492789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:31:06.608684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:06.633282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac8a5-ad95-449e-b436-59f93dbdeb6c service cinder] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:06.637808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:06.638553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:06.638950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:06.639242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:06.641412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:06.675300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:06.677318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:06.706362 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4361/8682] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:31:09.098084 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acef455-3673-4e04-995d-12f2b9997571 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:09.111861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acef455-3673-4e04-995d-12f2b9997571 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:09.115549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:09.116204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:09.116549 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:09.116889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:09.118774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:09.138305 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:09.140719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:09.157010 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4322/8683] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:31:14.370226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:14.371756 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:14.389351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47530b2-c03e-47a1-8645-3f8021469dc3 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:14.396296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:14.396944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:14.397220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:14.397502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:14.400198 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:14.414390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:14.416017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:14.426201 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4362/8684] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:31:14.436669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:14.438812 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:14.455089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d49c4-0f5b-4599-863e-5d51ac19b960 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:14.461357 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:14.462000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:14.462278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:14.463153 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:14.465378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:14.479347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:14.481169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:14.492486 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4323/8685] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:31:19.337177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:19.362537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6cfdf-db9b-413e-9d03-7bc18fa3ab82 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:19.367858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb06dd7-adfb-47f8-bae6-6b7c5b15ed7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:19.368469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb06dd7-adfb-47f8-bae6-6b7c5b15ed7c service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:19.368739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb06dd7-adfb-47f8-bae6-6b7c5b15ed7c service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:19.369004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb06dd7-adfb-47f8-bae6-6b7c5b15ed7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:19.370963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2bb06dd7-adfb-47f8-bae6-6b7c5b15ed7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:19.385348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb06dd7-adfb-47f8-bae6-6b7c5b15ed7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:19.386967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb06dd7-adfb-47f8-bae6-6b7c5b15ed7c service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:19.404985 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4363/8686] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:31:22.309933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:22.323576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858b9ed-2c98-4b56-ac50-045e3c92cd50 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:22.327724 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:22.328443 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:22.328721 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:22.328961 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:22.331021 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:22.393826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:22.395276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:22.405106 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4324/8687] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:22 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:31:25.516768 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce81a-dc23-4bc3-b1d1-48de11135754 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:25.517639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce81a-dc23-4bc3-b1d1-48de11135754 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:25.518065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce81a-dc23-4bc3-b1d1-48de11135754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:25.542170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1c4ce81a-dc23-4bc3-b1d1-48de11135754 None None] MFA Rules not processed for user `3cb55865ecad4c05adc52a1aacd2aa10`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:31:25.543462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1c4ce81a-dc23-4bc3-b1d1-48de11135754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:25.551668 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4364/8688] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:31:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:31:29.007493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:29.023579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c89e9-c3c9-4f62-a7b2-73aa0f472b3a service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:29.030127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:29.030933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:29.031426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:29.031706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:29.033565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:29.049093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:29.050965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:29.065107 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4325/8689] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:31:35.536919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=38 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:31:39.057818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=48 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:31:47.773201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ce81a-dc23-4bc3-b1d1-48de11135754 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:31:47.786372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ce81a-dc23-4bc3-b1d1-48de11135754 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:31:47.790568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f7106-e86c-4f45-b563-71ebfc1a2243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:31:47.791181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f7106-e86c-4f45-b563-71ebfc1a2243 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:31:47.791462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f7106-e86c-4f45-b563-71ebfc1a2243 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:31:47.791738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f7106-e86c-4f45-b563-71ebfc1a2243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:31:47.814861 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-029f7106-e86c-4f45-b563-71ebfc1a2243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:31:47.827703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f7106-e86c-4f45-b563-71ebfc1a2243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:31:47.829151 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f7106-e86c-4f45-b563-71ebfc1a2243 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:31:47.840483 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4365/8690] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:31:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:31:57.834184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:03.529241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:03.531214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:03.640731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef9f0-2755-46fb-b436-d7c7e8f9a7d2 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:03.645578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:03.646247 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:03.646595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:03.646933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] PATH_INFO: `/v3/users/3cb55865ecad4c05adc52a1aacd2aa10` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:03.648115 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb55865ecad4c05adc52a1aacd2aa10)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:03.650181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:03.712904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb55865ecad4c05adc52a1aacd2aa10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:03.726122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb55865ecad4c05adc52a1aacd2aa10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:03.735895 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb55865ecad4c05adc52a1aacd2aa10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:03.737046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb55865ecad4c05adc52a1aacd2aa10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:03.737822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb55865ecad4c05adc52a1aacd2aa10'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:03.755475 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4326/8691] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:33:03 2022] DELETE /identity/v3/users/3cb55865ecad4c05adc52a1aacd2aa10 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:03.760287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:03.760803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:03.760919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:03.784844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:03.786236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:03.849508 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4366/8692] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:33:03.859587 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:03.861086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:03.924100 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98364a-7e7f-4ede-bcd5-0985216562a0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:03.928469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:03.929109 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:03.929380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:03.929704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:03.940164 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:03.941661 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:03.951990 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4327/8693] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:33:04.257351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:04.259437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:04.274399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61e5a9-1d2b-44c5-8aca-e32350e07bd0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:04.279255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:04.279907 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:04.280181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:04.280455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] PATH_INFO: `/v3/projects/b67de414e1aa4fde9ed87336f30c2767` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:04.281658 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67de414e1aa4fde9ed87336f30c2767)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:04.283590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:04.338884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67de414e1aa4fde9ed87336f30c2767', 'request_id': 'req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.345379 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67de414e1aa4fde9ed87336f30c2767', 'request_id': 'req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.356569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67de414e1aa4fde9ed87336f30c2767', 'request_id': 'req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.367181 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] The token cache is being invalidate because project b67de414e1aa4fde9ed87336f30c2767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:33:04.367472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.368113 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.369681 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4367/8694] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:33:04 2022] DELETE /identity/v3/projects/b67de414e1aa4fde9ed87336f30c2767 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:04.374106 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:04.384360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a9b68-66f5-4085-b9d2-e167239235af service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:04.393177 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:04.446893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:04.447534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:04.447811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:04.448729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] PATH_INFO: `/v3/users/c2f8e397465f45a08b807d5240c07729` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:04.449241 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f8e397465f45a08b807d5240c07729)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:04.451435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:04.510035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f8e397465f45a08b807d5240c07729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.521746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f8e397465f45a08b807d5240c07729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.529387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f8e397465f45a08b807d5240c07729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.530049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f8e397465f45a08b807d5240c07729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.530680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f8e397465f45a08b807d5240c07729'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.542962 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4328/8695] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:33:04 2022] DELETE /identity/v3/users/c2f8e397465f45a08b807d5240c07729 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:04.718429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:04.720002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:04.737093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9cfc-81fc-4622-8a2c-1edcb6ed94da admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:04.803750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:04.804422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:04.804654 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:04.804924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] PATH_INFO: `/v3/projects/f94968a8495d447d9adf019219816f9c` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:04.806079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f94968a8495d447d9adf019219816f9c)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:04.808013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:04.837649 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:04.839363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:04.846139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:04.870531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f94968a8495d447d9adf019219816f9c', 'request_id': 'req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.878501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f94968a8495d447d9adf019219816f9c', 'request_id': 'req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.889944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f94968a8495d447d9adf019219816f9c', 'request_id': 'req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.905448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] The token cache is being invalidate because project f94968a8495d447d9adf019219816f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:33:04.905770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.906669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:04.908428 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4368/8696] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:33:04 2022] DELETE /identity/v3/projects/f94968a8495d447d9adf019219816f9c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:04.938795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ab6e-b399-49d9-a45b-946adf0d21da admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:04.977057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:05.024379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:05.025205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:05.025626 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:05.026022 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:05.028008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:05.046183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:05.048267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:05.065206 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4329/8697] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:33:08.362499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:08.364214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:08.427350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6f1ec-f44b-4ec1-b008-c1df2ced5458 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:08.431649 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:08.432258 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:08.432534 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:08.432803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:08.434967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:08.513883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:08.515254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:08.525318 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4369/8698] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:15.055648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=20 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:15.175928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:15.177614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:15.239690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:15.241448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:15.257089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0893db-40ac-4df1-a0be-ae4a67b84112 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:15.260214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc28c32-ac51-4f1b-972b-24e21551b3f4 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:15.262648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:15.263444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:15.263711 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:15.263986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] PATH_INFO: `/v3/users/342e60103c9b45bcb348b6f8bcd686ca` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:15.265174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=342e60103c9b45bcb348b6f8bcd686ca)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:15.265824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:15.266526 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:15.266865 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:15.267195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:15.267789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:15.270058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:15.284683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:15.286281 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:15.300906 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4330/8699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:15.317200 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342e60103c9b45bcb348b6f8bcd686ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.318026 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342e60103c9b45bcb348b6f8bcd686ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.326822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342e60103c9b45bcb348b6f8bcd686ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.327844 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342e60103c9b45bcb348b6f8bcd686ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.332817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:15.343172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342e60103c9b45bcb348b6f8bcd686ca'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.356222 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4370/8700] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:33:15 2022] DELETE /identity/v3/users/342e60103c9b45bcb348b6f8bcd686ca => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:33:15.532077 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:15.543756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3787d-d343-4fee-accf-949f938903da service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:15.607124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:15.607757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:15.608032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:15.608300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] PATH_INFO: `/v3/projects/b99d2a0b4d7e40a38e682fc147e968af` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:15.609511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99d2a0b4d7e40a38e682fc147e968af)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:15.611446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:15.674463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99d2a0b4d7e40a38e682fc147e968af', 'request_id': 'req-5b215720-3506-4a87-88c7-74da4bad0546'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.683635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99d2a0b4d7e40a38e682fc147e968af', 'request_id': 'req-5b215720-3506-4a87-88c7-74da4bad0546'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.694202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99d2a0b4d7e40a38e682fc147e968af', 'request_id': 'req-5b215720-3506-4a87-88c7-74da4bad0546'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.707295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] The token cache is being invalidate because project b99d2a0b4d7e40a38e682fc147e968af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:33:15.707620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.708386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.710291 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4331/8701] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:33:15 2022] DELETE /identity/v3/projects/b99d2a0b4d7e40a38e682fc147e968af => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:15.714863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:15.717550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:15.734969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d40312-8f16-43d6-a81a-01259107f3c0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:15.794803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:15.795469 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:15.795746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:15.796017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] PATH_INFO: `/v3/users/98278fb103484afdbbe3a8fab1ce6519` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:15.797165 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] RBAC: Authorizing `identity:delete_user(user_id=98278fb103484afdbbe3a8fab1ce6519)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:15.799138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:15.853523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98278fb103484afdbbe3a8fab1ce6519'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.854179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98278fb103484afdbbe3a8fab1ce6519'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.862660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98278fb103484afdbbe3a8fab1ce6519'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.864025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98278fb103484afdbbe3a8fab1ce6519'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.876387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-dddc7da6-9de3-48f6-ba3b-47e66f5bdbef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98278fb103484afdbbe3a8fab1ce6519'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:15.889482 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4371/8702] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:33:15 2022] DELETE /identity/v3/users/98278fb103484afdbbe3a8fab1ce6519 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:16.070990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:16.072761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:16.087275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b215720-3506-4a87-88c7-74da4bad0546 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:16.107016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:16.150217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:16.150880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:16.151160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:16.151430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] PATH_INFO: `/v3/projects/b67f323d906a456a9497b7cbe701dfd0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:16.152557 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67f323d906a456a9497b7cbe701dfd0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:16.154535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:16.201267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67f323d906a456a9497b7cbe701dfd0', 'request_id': 'req-e976f03d-6c0b-4029-8b5e-712b11213be4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:16.209913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67f323d906a456a9497b7cbe701dfd0', 'request_id': 'req-e976f03d-6c0b-4029-8b5e-712b11213be4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:16.218469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67f323d906a456a9497b7cbe701dfd0', 'request_id': 'req-e976f03d-6c0b-4029-8b5e-712b11213be4'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:16.230673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] The token cache is being invalidate because project b67f323d906a456a9497b7cbe701dfd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:33:16.230965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:16.231677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-e976f03d-6c0b-4029-8b5e-712b11213be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:16.233279 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4332/8703] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:33:16 2022] DELETE /identity/v3/projects/b67f323d906a456a9497b7cbe701dfd0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:25.885715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:26.227507 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:26.498550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddabc4-6a3f-4f40-998a-2f9a21c93c5d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.498782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddabc4-6a3f-4f40-998a-2f9a21c93c5d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.498997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddabc4-6a3f-4f40-998a-2f9a21c93c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.526323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8bddabc4-6a3f-4f40-998a-2f9a21c93c5d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:26.527248 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8bddabc4-6a3f-4f40-998a-2f9a21c93c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:26.598504 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4372/8704] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:26.612558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd3f6e-441e-499d-bb8b-7d932ef859a7 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.612843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd3f6e-441e-499d-bb8b-7d932ef859a7 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.613139 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd3f6e-441e-499d-bb8b-7d932ef859a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.640057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0ecd3f6e-441e-499d-bb8b-7d932ef859a7 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:26.641272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ecd3f6e-441e-499d-bb8b-7d932ef859a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:26.650222 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4333/8705] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:26.655252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddabc4-6a3f-4f40-998a-2f9a21c93c5d None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:26.665611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddabc4-6a3f-4f40-998a-2f9a21c93c5d None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:26.670475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:26.671088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.671365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.671633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.672343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:33:26.672579 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:26.674227 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:26.686811 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4373/8706] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:33:26.700458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd3f6e-441e-499d-bb8b-7d932ef859a7 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:26.710214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd3f6e-441e-499d-bb8b-7d932ef859a7 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:26.715392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:26.716039 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.716311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.716576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.717184 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:26.719155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:26.747860 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4334/8707] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:33:26 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:33:26.752345 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:26.762884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bb11-c9a7-4793-a4e6-3212a2ed37ca admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:26.820067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:26.820671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.820949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.821223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.821888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:26.823878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:26.849490 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4374/8708] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:33:26 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:33:26.854716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:26.870418 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63768d6d-c350-4c20-9e14-ec7560466237 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:26.875167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:26.875794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.876069 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.876336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.877005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:26.878639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:26.889682 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4335/8709] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:33:26.893896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:26.904313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436e78b-609f-4a77-af95-63354f0898fd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:26.909849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c2ff6-a1e7-4188-876b-b415bffabd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:26.910428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c2ff6-a1e7-4188-876b-b415bffabd11 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.910704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c2ff6-a1e7-4188-876b-b415bffabd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.910969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c2ff6-a1e7-4188-876b-b415bffabd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.911614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c2ff6-a1e7-4188-876b-b415bffabd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:26.913206 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c2ff6-a1e7-4188-876b-b415bffabd11 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:26.924008 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4375/8710] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:33:26.928390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:26.938524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff550309-11e0-45cc-a932-b21d25ffbc55 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:26.943438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:26.944078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.944353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.944624 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] PATH_INFO: `/v3/projects/f0364086523d46e7833d803ba0ee5437/users/5e669e81ae1a4a16b5fc70a3a06e2ccd/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:26.959706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0364086523d46e7833d803ba0ee5437, user_id=5e669e81ae1a4a16b5fc70a3a06e2ccd, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:26.962441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:26.977895 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4336/8711] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:33:26 2022] PUT /identity/v3/projects/f0364086523d46e7833d803ba0ee5437/users/5e669e81ae1a4a16b5fc70a3a06e2ccd/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:26.991703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ad2a2-303d-47ca-949b-31b37014230e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:26.991983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ad2a2-303d-47ca-949b-31b37014230e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:26.992250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ad2a2-303d-47ca-949b-31b37014230e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.022391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-8c0ad2a2-303d-47ca-949b-31b37014230e None None] MFA Rules not processed for user `5e669e81ae1a4a16b5fc70a3a06e2ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:27.023545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-8c0ad2a2-303d-47ca-949b-31b37014230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:27.083007 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:27.092873 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4376/8712] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:27.097434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:27.160091 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23daba4-420d-43d2-b6d7-c6ae3205d992 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:27.165514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:27.166223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.166531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.166814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.167422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:27.169356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:27.198761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4337/8713] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:33:27 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:33:27.203102 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ad2a2-303d-47ca-949b-31b37014230e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:27.214301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ad2a2-303d-47ca-949b-31b37014230e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:27.275587 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:27.276122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.276401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.276673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.277321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:27.279283 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:27.301929 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4377/8714] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:33:27 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:33:27.306097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:27.316730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2805d21-3927-426b-92b2-ab070fc37df8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:27.321823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:27.322425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.322701 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.322968 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.323608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:27.325246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:27.336355 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4338/8715] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:33:27.340591 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:27.351105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34191e-b77d-4695-a909-7615de96df67 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:27.357413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:27.358106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.358383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.358651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] PATH_INFO: `/v3/projects/1de6248b363b4cd186d7de7fc0827f1d/users/0ed3b0bcdf1b4716a39a225c80c000d0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.374816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de6248b363b4cd186d7de7fc0827f1d, user_id=0ed3b0bcdf1b4716a39a225c80c000d0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:27.377553 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:27.392379 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4378/8716] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:33:27 2022] PUT /identity/v3/projects/1de6248b363b4cd186d7de7fc0827f1d/users/0ed3b0bcdf1b4716a39a225c80c000d0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:33:27.396531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:27.407056 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc3fc2-d7f9-404a-b46b-afb6641cd152 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:27.473947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4af3-e37e-456a-b67a-60afa235132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:27.474556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4af3-e37e-456a-b67a-60afa235132c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.474832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4af3-e37e-456a-b67a-60afa235132c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.475094 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4af3-e37e-456a-b67a-60afa235132c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.475730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb4af3-e37e-456a-b67a-60afa235132c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:27.477742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb4af3-e37e-456a-b67a-60afa235132c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:27.487960 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4339/8717] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:33:27.492380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:27.503268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b5f9b-db2c-4d97-9117-345c06d207a4 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:27.508236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e1671-abe4-4e9b-aff3-34ac091b8823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:27.508887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e1671-abe4-4e9b-aff3-34ac091b8823 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.509166 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e1671-abe4-4e9b-aff3-34ac091b8823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.509446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e1671-abe4-4e9b-aff3-34ac091b8823 admin admin] PATH_INFO: `/v3/projects/1de6248b363b4cd186d7de7fc0827f1d/users/0ed3b0bcdf1b4716a39a225c80c000d0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.511635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e1671-abe4-4e9b-aff3-34ac091b8823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de6248b363b4cd186d7de7fc0827f1d, user_id=0ed3b0bcdf1b4716a39a225c80c000d0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:27.514352 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e1671-abe4-4e9b-aff3-34ac091b8823 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:27.525105 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4379/8718] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:33:27 2022] PUT /identity/v3/projects/1de6248b363b4cd186d7de7fc0827f1d/users/0ed3b0bcdf1b4716a39a225c80c000d0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:27.540779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.540971 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.541349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.557572 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:27.573715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] MFA Rules not processed for user `0ed3b0bcdf1b4716a39a225c80c000d0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:27.574922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:27.645655 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4340/8719] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:27.659260 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.659549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.659799 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.682825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] MFA Rules not processed for user `5e669e81ae1a4a16b5fc70a3a06e2ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:27.684050 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:27.747445 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4380/8720] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:27.762894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:27.764951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:27.847586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7484f-9f7b-4860-9b9b-45fa5d0a1229 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:27.854692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:27.855302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:27.855582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:27.855853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:27.863778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:27.865394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:27.874489 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4341/8721] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:28.350922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:28.352940 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:28.408164 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:28.431613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0d540-1db3-4f24-bb79-940dc1415d26 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:28.437115 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:28.437770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:28.438048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:28.438321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:28.440149 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:28.454342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:28.455930 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:28.468289 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4381/8722] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:37.864116 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:37.865686 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:37.885621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8327b3-e092-4b61-ad75-014091f0e419 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:37.891190 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308ba16-f079-4771-9b55-976389d2b3be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:37.891794 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308ba16-f079-4771-9b55-976389d2b3be service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:37.892068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308ba16-f079-4771-9b55-976389d2b3be service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:37.892340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308ba16-f079-4771-9b55-976389d2b3be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:37.894192 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1308ba16-f079-4771-9b55-976389d2b3be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:37.909874 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308ba16-f079-4771-9b55-976389d2b3be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:37.911437 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308ba16-f079-4771-9b55-976389d2b3be service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:37.925972 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4342/8723] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:38.461966 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=8 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:40.220662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:40.237782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ae12d-717e-4495-900c-c4e3689cc7b5 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:40.244000 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:40.244592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:40.244870 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:40.245140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:40.254019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:40.255646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:40.265616 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4382/8724] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:40.281083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67e00-0b5e-45c6-98f9-ad7799e57e7f None None] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:40.281567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67e00-0b5e-45c6-98f9-ad7799e57e7f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:40.281675 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67e00-0b5e-45c6-98f9-ad7799e57e7f None None] PATH_INFO: `/` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:40.283085 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4343/8725] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:33:40.287921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] Authenticating service token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 21:33:40.302180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:40.310866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19aec8-5480-4e25-b81c-c78f3fe8c101 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:40.316141 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97062fcc-9ec0-4e35-8ddf-1b3be78e14ca tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed3b0bcdf1b4716a39a225c80c000d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1de6248b363b4cd186d7de7fc0827f1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:40.316728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97062fcc-9ec0-4e35-8ddf-1b3be78e14ca tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:40.317016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97062fcc-9ec0-4e35-8ddf-1b3be78e14ca tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:40.317300 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97062fcc-9ec0-4e35-8ddf-1b3be78e14ca tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] PATH_INFO: `/v3/projects/1de6248b363b4cd186d7de7fc0827f1d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:40.318507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97062fcc-9ec0-4e35-8ddf-1b3be78e14ca tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] RBAC: Authorizing `identity:get_project(project_id=1de6248b363b4cd186d7de7fc0827f1d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:40.320310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97062fcc-9ec0-4e35-8ddf-1b3be78e14ca tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:40.321975 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4383/8726] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 21:33:40 2022] GET /identity/v3/projects/1de6248b363b4cd186d7de7fc0827f1d => generated 393 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:33:40.484847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f67e00-0b5e-45c6-98f9-ad7799e57e7f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:40.487854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-02f67e00-0b5e-45c6-98f9-ad7799e57e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:40.553324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f67e00-0b5e-45c6-98f9-ad7799e57e7f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:40.558249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:40.558942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:40.559280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:40.559553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:40.561692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:40.574591 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:40.576043 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:40.586660 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4344/8727] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:40.747999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa35bf-b2eb-4458-bf99-303b2f8b1cd8 None None] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:40.748280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa35bf-b2eb-4458-bf99-303b2f8b1cd8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:40.748555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa35bf-b2eb-4458-bf99-303b2f8b1cd8 None None] PATH_INFO: `/` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:40.749990 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4384/8728] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 17 21:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:33:40.754781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] Authenticating service token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 17 21:33:40.771650 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:40.776354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=100 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:40.782426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b1eb-a0bd-4748-83d2-8566a15661dd service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:40.787830 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed3b0bcdf1b4716a39a225c80c000d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1de6248b363b4cd186d7de7fc0827f1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:40.788459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:40.788738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:40.789027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] PATH_INFO: `/v3/projects/1de6248b363b4cd186d7de7fc0827f1d` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:40.790238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] RBAC: Authorizing `identity:get_project(project_id=1de6248b363b4cd186d7de7fc0827f1d)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:40.792179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:40.793880 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4345/8729] 10.222.0.26 () {58 vars in 1443 bytes} [Sat Sep 17 21:33:40 2022] GET /identity/v3/projects/1de6248b363b4cd186d7de7fc0827f1d => generated 393 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:33:40.952204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa35bf-b2eb-4458-bf99-303b2f8b1cd8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:40.971207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa35bf-b2eb-4458-bf99-303b2f8b1cd8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:40.975491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac964e-37ff-44cf-9670-e32165e11a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:40.976093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac964e-37ff-44cf-9670-e32165e11a9d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:40.976366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac964e-37ff-44cf-9670-e32165e11a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:40.977002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac964e-37ff-44cf-9670-e32165e11a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:40.978804 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-22ac964e-37ff-44cf-9670-e32165e11a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:40.991471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac964e-37ff-44cf-9670-e32165e11a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:40.992867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac964e-37ff-44cf-9670-e32165e11a9d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:41.003835 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4385/8730] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:43.670364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:43.737303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f64be4-5e75-4af4-b9e1-465775f57225 tempest-ServersAdminNegativeTestJSON-1371865185 tempest-ServersAdminNegativeTestJSON-1371865185-project-admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:43.742332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:43.742965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:43.743243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:43.743513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] PATH_INFO: `/v3/users/5e669e81ae1a4a16b5fc70a3a06e2ccd` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:43.744677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e669e81ae1a4a16b5fc70a3a06e2ccd)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:43.746667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:43.795712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e669e81ae1a4a16b5fc70a3a06e2ccd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:43.810035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e669e81ae1a4a16b5fc70a3a06e2ccd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:43.820584 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e669e81ae1a4a16b5fc70a3a06e2ccd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:43.821226 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e669e81ae1a4a16b5fc70a3a06e2ccd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:43.821829 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e669e81ae1a4a16b5fc70a3a06e2ccd'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:43.834648 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4346/8731] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:33:43 2022] DELETE /identity/v3/users/5e669e81ae1a4a16b5fc70a3a06e2ccd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:43.839383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:43.839901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:43.839901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:43.862184 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:43.863400 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:43.929873 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4386/8732] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:33:43.939977 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:43.941643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:44.006166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f483c97-befa-4905-a1d6-b2c082023e57 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:44.010327 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:44.010931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:44.011208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:44.011475 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:44.021232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:44.022763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:44.032571 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4347/8733] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:44.213723 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:44.214906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:44.237606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490c53-6b66-4813-aebe-4153592b7532 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:44.243035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:44.243684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:44.243984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:44.244265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] PATH_INFO: `/v3/projects/f0364086523d46e7833d803ba0ee5437` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:44.245540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0364086523d46e7833d803ba0ee5437)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:44.247485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:44.297496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0364086523d46e7833d803ba0ee5437', 'request_id': 'req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.303760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0364086523d46e7833d803ba0ee5437', 'request_id': 'req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.313157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0364086523d46e7833d803ba0ee5437', 'request_id': 'req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.323719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] The token cache is being invalidate because project f0364086523d46e7833d803ba0ee5437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:33:44.324001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.324611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.326177 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4387/8734] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:33:44 2022] DELETE /identity/v3/projects/f0364086523d46e7833d803ba0ee5437 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:44.330595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:44.340957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9cc7-1ebd-41c4-bbb2-f61d3164cbe1 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:44.359684 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:44.403298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:44.403949 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:44.404224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:44.404506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] PATH_INFO: `/v3/users/0ed3b0bcdf1b4716a39a225c80c000d0` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:44.405776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed3b0bcdf1b4716a39a225c80c000d0)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:44.407772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:44.457269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed3b0bcdf1b4716a39a225c80c000d0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.469280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed3b0bcdf1b4716a39a225c80c000d0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.485421 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed3b0bcdf1b4716a39a225c80c000d0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.486174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed3b0bcdf1b4716a39a225c80c000d0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.486902 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-47b11231-4b08-4000-bfd5-0afd7331df0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed3b0bcdf1b4716a39a225c80c000d0'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.500030 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4348/8735] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:33:44 2022] DELETE /identity/v3/users/0ed3b0bcdf1b4716a39a225c80c000d0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:33:44.784928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:44.786515 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:44.800666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae1e0c-f23d-47b4-8247-d586ad0c8e47 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:44.856205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:44.869189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:44.869862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:44.870142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:44.870416 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] PATH_INFO: `/v3/projects/1de6248b363b4cd186d7de7fc0827f1d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:44.871557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de6248b363b4cd186d7de7fc0827f1d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:44.873537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:44.922092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de6248b363b4cd186d7de7fc0827f1d', 'request_id': 'req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.928707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de6248b363b4cd186d7de7fc0827f1d', 'request_id': 'req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.937681 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de6248b363b4cd186d7de7fc0827f1d', 'request_id': 'req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.948506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] The token cache is being invalidate because project 1de6248b363b4cd186d7de7fc0827f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:33:44.948998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.949506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-22caa1ac-3fc8-4c3f-b00c-b87db27d19f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:33:44.951011 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4388/8736] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:33:44 2022] DELETE /identity/v3/projects/1de6248b363b4cd186d7de7fc0827f1d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:33:44.957105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5a25-80df-4182-af4d-4830ae244346 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:44.957628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5a25-80df-4182-af4d-4830ae244346 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:44.957787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5a25-80df-4182-af4d-4830ae244346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:44.977640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4cfb5a25-80df-4182-af4d-4830ae244346 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:44.978833 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4cfb5a25-80df-4182-af4d-4830ae244346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:45.040567 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4349/8737] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:45.050803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78268-98e6-49e2-8bf6-0d77138d67e4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.051082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78268-98e6-49e2-8bf6-0d77138d67e4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.051353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78268-98e6-49e2-8bf6-0d77138d67e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.071755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9eb78268-98e6-49e2-8bf6-0d77138d67e4 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:45.072960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9eb78268-98e6-49e2-8bf6-0d77138d67e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:45.082631 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4389/8738] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:33:45.087096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb5a25-80df-4182-af4d-4830ae244346 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:45.097133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb5a25-80df-4182-af4d-4830ae244346 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:45.102667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:45.103231 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.103511 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.103781 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.104458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:33:45.104761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:45.106417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:45.118116 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4350/8739] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:33:45.122620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb78268-98e6-49e2-8bf6-0d77138d67e4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:45.132216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb78268-98e6-49e2-8bf6-0d77138d67e4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:45.138130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:45.138745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.139023 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.139289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.139910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:45.141910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:45.175239 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4390/8740] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:33:45 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:33:45.179765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:45.190517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b9dc4-7cf3-41be-84ae-d2e9aeda9ab4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:45.221319 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:45.255581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:45.256178 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.256449 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.256716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.257337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:45.259323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:45.281898 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4351/8741] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:33:45 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:33:45.286408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:45.297090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576b162-6df5-4f30-a808-57cedec29c7b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:45.302467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:45.303067 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.303336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.303603 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.304243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:45.305897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:45.316232 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4391/8742] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:33:45.320188 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:45.330747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92dc41e-0be4-421f-9254-1f122f9db2e9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:45.336141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceb460-f13d-4fef-b6de-99bc5b6b2b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:45.336746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceb460-f13d-4fef-b6de-99bc5b6b2b3c admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.337029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceb460-f13d-4fef-b6de-99bc5b6b2b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.337312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceb460-f13d-4fef-b6de-99bc5b6b2b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.338002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceb460-f13d-4fef-b6de-99bc5b6b2b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:45.339620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceb460-f13d-4fef-b6de-99bc5b6b2b3c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:45.351122 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4352/8743] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:33:45.355693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:45.366362 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67a330-eed1-4026-904b-5df9c86b82d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:45.371410 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:45.372144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.372415 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.372692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] PATH_INFO: `/v3/projects/dd3d716a01034337b002a97e8c430162/users/22aaf576a5134ec4a09550ccf2c98235/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.388801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd3d716a01034337b002a97e8c430162, user_id=22aaf576a5134ec4a09550ccf2c98235, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:45.391495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:45.405918 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4392/8744] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:33:45 2022] PUT /identity/v3/projects/dd3d716a01034337b002a97e8c430162/users/22aaf576a5134ec4a09550ccf2c98235/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:33:45.411416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d2f2-08df-4986-8333-97fbff0d07e1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.411416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d2f2-08df-4986-8333-97fbff0d07e1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.411730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28d2f2-08df-4986-8333-97fbff0d07e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.431678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6c28d2f2-08df-4986-8333-97fbff0d07e1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:45.432916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6c28d2f2-08df-4986-8333-97fbff0d07e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:45.501634 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4353/8745] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:45.511750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:45.513643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:45.575342 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801af708-7ce3-4fa5-bbf8-24175aa15f18 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:45.579281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77bb022-9504-4062-bb3b-c922bb496d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:45.579926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bb022-9504-4062-bb3b-c922bb496d9f service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:45.580197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bb022-9504-4062-bb3b-c922bb496d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:45.580467 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bb022-9504-4062-bb3b-c922bb496d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:45.590632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77bb022-9504-4062-bb3b-c922bb496d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:45.592077 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77bb022-9504-4062-bb3b-c922bb496d9f service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:45.602657 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4393/8746] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:49.416534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:49.417097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:49.417097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:49.446764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] MFA Rules not processed for user `22aaf576a5134ec4a09550ccf2c98235`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:49.447982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:49.536168 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4354/8747] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:49.539632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:49.540112 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:49.540216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:49.560589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] MFA Rules not processed for user `22aaf576a5134ec4a09550ccf2c98235`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:33:49.561859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:49.570182 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4394/8748] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:33:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:33:49.582869 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:49.584353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:49.629120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:49.672270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6383eb-ca7d-48dc-92af-3acbb4eb6495 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:49.677688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:49.678336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:49.678770 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:49.679076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:49.688345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:49.690068 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:49.700112 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4355/8749] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:54.235686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:54.237223 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:54.252017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:54.317224 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1546e-22ec-496c-980a-4d8fa50ef281 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:54.323096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:54.323720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:54.324032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:54.324305 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:54.326328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:54.343255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:54.344914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:54.359526 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4395/8750] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:33:58.028322 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:33:58.043025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5082b28-4882-44b1-a86f-1b7299c512d3 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:33:58.049560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:33:58.050249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:33:58.050505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:33:58.050797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:33:58.052878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:33:58.068378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:33:58.070050 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:33:58.083270 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4356/8751] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:33:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:34:03.645764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:03.658536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be83c42-5e2d-4003-b437-7aa9ab0477d0 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:03.662218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:03.663019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:03.663303 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:03.663576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:03.665328 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:03.678840 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:03.680285 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:03.692814 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4396/8752] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:34:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:34:03.800408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:03.814628 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132a1a7-4a22-41c7-acd5-467e84150bf7 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:03.820755 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:03.821360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:03.821678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:03.821946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:03.823956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:03.838516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:03.840112 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:03.853439 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4357/8753] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:34:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:34:04.041719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:04.043358 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:04.058170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2f27c-1cd4-491c-9959-c0e3f148f37e service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:04.062845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:04.063482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:04.063755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:04.064025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:04.065967 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:04.082687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:04.084122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:04.098977 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4397/8754] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:34:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:34:13.846212 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=30 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:14.090335 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=39 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:44.555662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:44.570604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb8d8d-ee7f-4faf-9051-f4d2b3874b5b service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:44.575611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:44.576254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:44.576532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:44.576837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] PATH_INFO: `/v3/users/22aaf576a5134ec4a09550ccf2c98235` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:44.578015 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=22aaf576a5134ec4a09550ccf2c98235)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:44.579975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:44.630313 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22aaf576a5134ec4a09550ccf2c98235'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.642445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22aaf576a5134ec4a09550ccf2c98235'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.651467 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22aaf576a5134ec4a09550ccf2c98235'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.652098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22aaf576a5134ec4a09550ccf2c98235'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.652896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-deb8bcbc-c924-4cdb-9235-f5c13eeee21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22aaf576a5134ec4a09550ccf2c98235'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.666845 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4358/8755] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:34:44 2022] DELETE /identity/v3/users/22aaf576a5134ec4a09550ccf2c98235 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:34:44.849299 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:44.862438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ee36-e86e-4edf-a281-00f68889be12 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:44.921389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:44.922090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:44.922363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:44.922629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] PATH_INFO: `/v3/projects/dd3d716a01034337b002a97e8c430162` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:44.923738 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd3d716a01034337b002a97e8c430162)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:44.925714 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:44.973096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd3d716a01034337b002a97e8c430162', 'request_id': 'req-a42c8110-8493-4181-861c-3a8d83416cfa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.979477 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd3d716a01034337b002a97e8c430162', 'request_id': 'req-a42c8110-8493-4181-861c-3a8d83416cfa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.987736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd3d716a01034337b002a97e8c430162', 'request_id': 'req-a42c8110-8493-4181-861c-3a8d83416cfa'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.998129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] The token cache is being invalidate because project dd3d716a01034337b002a97e8c430162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:34:44.998418 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:44.999054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a42c8110-8493-4181-861c-3a8d83416cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:34:45.000560 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4398/8756] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:34:44 2022] DELETE /identity/v3/projects/dd3d716a01034337b002a97e8c430162 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:34:45.005788 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ffee3-b091-4210-8078-822e048d4109 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.006093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ffee3-b091-4210-8078-822e048d4109 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.006364 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ffee3-b091-4210-8078-822e048d4109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.026422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e47ffee3-b091-4210-8078-822e048d4109 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:34:45.027379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e47ffee3-b091-4210-8078-822e048d4109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:45.090033 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4359/8757] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:34:45.098937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb6fbc-f7bf-452c-9684-6ea744b9023a None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.099221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb6fbc-f7bf-452c-9684-6ea744b9023a None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.099492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb6fbc-f7bf-452c-9684-6ea744b9023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.120068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9ecb6fbc-f7bf-452c-9684-6ea744b9023a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:34:45.121282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9ecb6fbc-f7bf-452c-9684-6ea744b9023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:45.129801 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4399/8758] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:34:45.134140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ffee3-b091-4210-8078-822e048d4109 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:45.145133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ffee3-b091-4210-8078-822e048d4109 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:45.151352 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:45.151955 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.152222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.152470 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.153162 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:34:45.153487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:45.155127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:45.167525 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4360/8759] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:34:45.172076 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb6fbc-f7bf-452c-9684-6ea744b9023a None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:45.182014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb6fbc-f7bf-452c-9684-6ea744b9023a None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:45.187956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:45.188557 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.188808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.189069 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.189728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:45.191679 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:45.221853 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4400/8760] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:34:45 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:34:45.228539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:45.238974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51887bc4-33e4-4793-92f4-97525432c7a4 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:45.296325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:45.296834 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.297131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.297378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.298037 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:45.300049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:45.308250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:45.323779 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4361/8761] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:34:45 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:34:45.328106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:45.338857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e86efd-9dde-48a3-b2b2-9ce355255338 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:45.343971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:45.344572 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.344850 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.345125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.345791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:45.347429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:45.357941 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4401/8762] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:34:45.362063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:45.372950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7377-c13c-4da1-9558-7cf84a256404 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:45.378272 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc795e-f519-4c36-8b7c-76aa6d9a23e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:45.378866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc795e-f519-4c36-8b7c-76aa6d9a23e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.379130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc795e-f519-4c36-8b7c-76aa6d9a23e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.379401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc795e-f519-4c36-8b7c-76aa6d9a23e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.380048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc795e-f519-4c36-8b7c-76aa6d9a23e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:45.381700 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc795e-f519-4c36-8b7c-76aa6d9a23e9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:45.392551 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4362/8763] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:34:45.396932 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:45.407732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc78fa-a04e-4b08-bdad-8eb6f79fac5d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:45.413087 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:45.413781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.414061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.414334 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] PATH_INFO: `/v3/projects/2412712d1e76496a892826131215475f/users/da549e8947e445fea05a3969c98c732e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.430790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2412712d1e76496a892826131215475f, user_id=da549e8947e445fea05a3969c98c732e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:45.433484 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:45.448382 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4402/8764] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:34:45 2022] PUT /identity/v3/projects/2412712d1e76496a892826131215475f/users/da549e8947e445fea05a3969c98c732e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:34:45.453759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf159f80-bbbf-468a-9256-7047e5b344fe None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.454047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf159f80-bbbf-468a-9256-7047e5b344fe None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.454318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf159f80-bbbf-468a-9256-7047e5b344fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.473731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bf159f80-bbbf-468a-9256-7047e5b344fe None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:34:45.475302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bf159f80-bbbf-468a-9256-7047e5b344fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:45.536320 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4363/8765] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:34:45.547524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:45.549399 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:45.582075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:45.614155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63504fe8-fbad-40ab-a39f-b67cbd082c37 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:45.617914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1863c-28cf-4e4d-904f-4a5443f846c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:45.618528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1863c-28cf-4e4d-904f-4a5443f846c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:45.618805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1863c-28cf-4e4d-904f-4a5443f846c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:45.619079 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1863c-28cf-4e4d-904f-4a5443f846c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:45.628739 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1863c-28cf-4e4d-904f-4a5443f846c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:45.630259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1863c-28cf-4e4d-904f-4a5443f846c7 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:45.640626 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4403/8766] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:34:49.693711 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:49.694182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:49.694282 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:49.723430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] MFA Rules not processed for user `da549e8947e445fea05a3969c98c732e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:34:49.724583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:49.791805 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4364/8767] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:34:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:34:49.803346 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e187dc3-fcfb-443b-95f5-7cd28c9005df None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:49.803897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e187dc3-fcfb-443b-95f5-7cd28c9005df None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:49.803897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e187dc3-fcfb-443b-95f5-7cd28c9005df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:49.823667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9e187dc3-fcfb-443b-95f5-7cd28c9005df None None] MFA Rules not processed for user `da549e8947e445fea05a3969c98c732e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:34:49.824906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e187dc3-fcfb-443b-95f5-7cd28c9005df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:49.836148 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4404/8768] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:34:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:34:49.844125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:34:49.845780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:34:49.928864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f7f4a-8e71-469e-ba6b-ae0e2409c612 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:34:49.934489 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:34:49.935104 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:34:49.935394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:34:49.935663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:34:49.940897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:49.946798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:34:49.948544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:34:49.959365 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4365/8769] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:34:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:34:59.818095 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=17 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:59.955032 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:35:36.682340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1d0bf-8666-431e-bb98-f7debb0a2c6e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:35:36.682958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1d0bf-8666-431e-bb98-f7debb0a2c6e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:35:36.682958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1d0bf-8666-431e-bb98-f7debb0a2c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:35:36.713437 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-2bd1d0bf-8666-431e-bb98-f7debb0a2c6e None None] MFA Rules not processed for user `da549e8947e445fea05a3969c98c732e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:35:36.714700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2bd1d0bf-8666-431e-bb98-f7debb0a2c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:35:36.740213 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4405/8770] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:35:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:35:46.738182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=15 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:08.592859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:08.594519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:08.643815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879bf5e-0873-48bb-b781-2ec4bb753c8f service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:08.647718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:08.648344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:08.648613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:08.648879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:08.650677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:08.676578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:08.678089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:08.693489 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4366/8771] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:09.898092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1d0bf-8666-431e-bb98-f7debb0a2c6e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:09.910958 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1d0bf-8666-431e-bb98-f7debb0a2c6e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:09.915038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:09.915641 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:09.915911 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:09.916182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:09.918061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:09.935137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:09.936555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:09.950608 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4406/8772] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:16.689825 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:16.701658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebb42e-134a-46ad-9abe-98ddcb87b53f service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:16.706566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:16.707344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:16.707664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:16.707914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:16.709859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:16.723664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:16.725173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:16.736500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4367/8773] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:19.942912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:26.729988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=29 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:29.727037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:29.729790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:29.791245 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:29.820916 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1964e9a-003e-42a5-a1c3-a00805116f0f service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:29.821276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2a109-c2c3-42b0-9bc6-48f0d8a400b9 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:29.825431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:29.826103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:29.826375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:29.826499 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:29.826647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] PATH_INFO: `/v3/users/da549e8947e445fea05a3969c98c732e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:29.826984 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:29.827263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:29.827556 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:29.827792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da549e8947e445fea05a3969c98c732e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:29.829290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:29.829826 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:29.845630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:29.847234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:29.865533 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4407/8774] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:29.883962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da549e8947e445fea05a3969c98c732e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:29.897590 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da549e8947e445fea05a3969c98c732e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:29.908158 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da549e8947e445fea05a3969c98c732e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:29.908831 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da549e8947e445fea05a3969c98c732e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:29.909440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3265c8a4-107f-4f22-b54b-7359f769fa65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da549e8947e445fea05a3969c98c732e'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:29.921893 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4368/8775] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:36:29 2022] DELETE /identity/v3/users/da549e8947e445fea05a3969c98c732e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:36:30.102091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.113146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab4ac8-8d58-459e-90b6-84fd041e8d55 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.186872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.187524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.187810 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.188101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] PATH_INFO: `/v3/projects/2412712d1e76496a892826131215475f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.189247 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2412712d1e76496a892826131215475f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.191245 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.240646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2412712d1e76496a892826131215475f', 'request_id': 'req-e710177d-0e1a-4546-866e-f1b903a9fff8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:30.246916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2412712d1e76496a892826131215475f', 'request_id': 'req-e710177d-0e1a-4546-866e-f1b903a9fff8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:30.254828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2412712d1e76496a892826131215475f', 'request_id': 'req-e710177d-0e1a-4546-866e-f1b903a9fff8'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:30.275815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:30.278877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] The token cache is being invalidate because project 2412712d1e76496a892826131215475f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:36:30.279174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:30.279926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-e710177d-0e1a-4546-866e-f1b903a9fff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:36:30.281583 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4408/8776] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:36:30 2022] DELETE /identity/v3/projects/2412712d1e76496a892826131215475f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:36:30.286965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e353-7794-42ce-97fa-1cac86565275 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.287254 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e353-7794-42ce-97fa-1cac86565275 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.287531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e353-7794-42ce-97fa-1cac86565275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.348771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a898e353-7794-42ce-97fa-1cac86565275 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:36:30.349970 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a898e353-7794-42ce-97fa-1cac86565275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:30.410370 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4369/8777] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:36:30.419306 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e589c6-50d6-48ff-8bb1-afa36a2ebf01 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.419594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e589c6-50d6-48ff-8bb1-afa36a2ebf01 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.419867 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e589c6-50d6-48ff-8bb1-afa36a2ebf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.441754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-93e589c6-50d6-48ff-8bb1-afa36a2ebf01 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:36:30.442972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-93e589c6-50d6-48ff-8bb1-afa36a2ebf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:30.451768 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4409/8778] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:36:30.456290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898e353-7794-42ce-97fa-1cac86565275 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.466325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898e353-7794-42ce-97fa-1cac86565275 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.471238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.471835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.472110 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.472735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] PATH_INFO: `/v3/domains` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.473377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:36:30.473707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.475311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.487027 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4370/8779] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:36:30.491372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e589c6-50d6-48ff-8bb1-afa36a2ebf01 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.500820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e589c6-50d6-48ff-8bb1-afa36a2ebf01 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.505753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.506340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.506611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.506877 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] PATH_INFO: `/v3/projects` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.507507 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.509405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.552602 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4410/8780] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:36:30 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:36:30.558927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.570938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba105a-8e39-4e19-bc3c-f1ea60cf3310 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.597057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:30.629820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.630425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.630703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.630972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] PATH_INFO: `/v3/users` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.631639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.633612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.656679 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4371/8781] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:36:30 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:36:30.660998 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.671653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7de09-c090-45b8-90c3-ac3490562147 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.676568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.677207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.677488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.677782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.678450 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.680084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.690980 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4411/8782] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:36:30.695183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.705837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a65af-5b81-41d7-b25b-addf72fd0696 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.711097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa6c3a-597d-493b-9238-8a19dc8fe15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.711697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa6c3a-597d-493b-9238-8a19dc8fe15e admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.711972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa6c3a-597d-493b-9238-8a19dc8fe15e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.712265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa6c3a-597d-493b-9238-8a19dc8fe15e admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.712906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa6c3a-597d-493b-9238-8a19dc8fe15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.714547 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa6c3a-597d-493b-9238-8a19dc8fe15e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.725194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4372/8783] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:36:30.729449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.739806 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88a3ee-895b-4b90-b192-19f525453b6a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.744987 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.745614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.745892 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.746162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] PATH_INFO: `/v3/projects/ad243abc13a14aa08c717d4105e24f4f/users/d20f8598dfec4de7b496a64e023e7a15/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.761307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad243abc13a14aa08c717d4105e24f4f, user_id=d20f8598dfec4de7b496a64e023e7a15, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.763997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.779672 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4412/8784] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:36:30 2022] PUT /identity/v3/projects/ad243abc13a14aa08c717d4105e24f4f/users/d20f8598dfec4de7b496a64e023e7a15/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:36:30.784915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36877b-db41-4705-a3ed-b4cddb8816f5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.785195 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36877b-db41-4705-a3ed-b4cddb8816f5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.785481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36877b-db41-4705-a3ed-b4cddb8816f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.804548 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0f36877b-db41-4705-a3ed-b4cddb8816f5 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:36:30.805761 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0f36877b-db41-4705-a3ed-b4cddb8816f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:30.871257 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4373/8785] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:36:30.881496 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:30.883372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:30.945428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9718d-ffd9-4bcb-9b36-184603f96020 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:30.949506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:30.950117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:30.950389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:30.950656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:30.960179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:30.961650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:30.971356 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4413/8786] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:35.175419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de6249-4121-496e-a843-ee08248ac235 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:35.176006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de6249-4121-496e-a843-ee08248ac235 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:35.176133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de6249-4121-496e-a843-ee08248ac235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:35.207196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c8de6249-4121-496e-a843-ee08248ac235 None None] MFA Rules not processed for user `d20f8598dfec4de7b496a64e023e7a15`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:36:35.208512 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c8de6249-4121-496e-a843-ee08248ac235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:35.279565 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4374/8787] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:36:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:36:35.292315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:35.293906 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:35.373187 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad02df7-d718-4d67-9d3c-f9357d59b584 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:35.378934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:35.379567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:35.379843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:35.380110 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:35.389645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:35.391458 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:35.401699 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4414/8788] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:36.712664 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91239f9-d79b-4e30-ac0e-3455afc77537 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:36.713171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91239f9-d79b-4e30-ac0e-3455afc77537 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:36.713324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91239f9-d79b-4e30-ac0e-3455afc77537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:36.733780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a91239f9-d79b-4e30-ac0e-3455afc77537 None None] MFA Rules not processed for user `d20f8598dfec4de7b496a64e023e7a15`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:36:36.735105 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a91239f9-d79b-4e30-ac0e-3455afc77537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:36.742608 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4375/8789] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:36:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:36:40.398680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:40.400448 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:40.428971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5e9b1-372b-40b5-acb7-d276d25489d1 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:40.437779 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c48637-625a-47af-b7d6-c91c885a7348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:40.438510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c48637-625a-47af-b7d6-c91c885a7348 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:40.438785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c48637-625a-47af-b7d6-c91c885a7348 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:40.439053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c48637-625a-47af-b7d6-c91c885a7348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:40.440999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-01c48637-625a-47af-b7d6-c91c885a7348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:40.457730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c48637-625a-47af-b7d6-c91c885a7348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:40.459462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c48637-625a-47af-b7d6-c91c885a7348 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:40.474541 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4415/8790] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:46.727697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:48.170349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91239f9-d79b-4e30-ac0e-3455afc77537 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:36:48.183445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91239f9-d79b-4e30-ac0e-3455afc77537 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:36:48.187879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e40658-3109-425a-8a12-e8ce560b8446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:36:48.188560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40658-3109-425a-8a12-e8ce560b8446 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:36:48.188840 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40658-3109-425a-8a12-e8ce560b8446 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:36:48.189108 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e40658-3109-425a-8a12-e8ce560b8446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:36:48.191130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-35e40658-3109-425a-8a12-e8ce560b8446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:36:48.204771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e40658-3109-425a-8a12-e8ce560b8446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:36:48.206371 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e40658-3109-425a-8a12-e8ce560b8446 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:36:48.217197 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4376/8791] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:36:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:36:50.466639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:58.211044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:37:49.151979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:37:49.152492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:37:49.152698 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:37:49.172755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] MFA Rules not processed for user `d20f8598dfec4de7b496a64e023e7a15`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:37:49.174002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:37:49.180842 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4416/8792] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:37:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:37:59.168154 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:38:44.720943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05804af-e8f9-45fb-957f-9a9cedbc5ff8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:44.721621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05804af-e8f9-45fb-957f-9a9cedbc5ff8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:44.721621 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05804af-e8f9-45fb-957f-9a9cedbc5ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:44.778520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c05804af-e8f9-45fb-957f-9a9cedbc5ff8 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:38:44.779917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c05804af-e8f9-45fb-957f-9a9cedbc5ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:44.846182 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4377/8793] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:38:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:38:44.858063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:38:44.859670 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:44.878407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eadabd-f713-4f05-b695-1dd55a47191e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:38:44.884480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:38:44.885080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:44.885369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:44.885653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:44.894898 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:38:44.896480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:38:44.907554 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4417/8794] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:38:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:38:45.142618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6bf73-46a2-48ea-af33-87bf452484c2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:45.142936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6bf73-46a2-48ea-af33-87bf452484c2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:45.143209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6bf73-46a2-48ea-af33-87bf452484c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:45.166149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-e8b6bf73-46a2-48ea-af33-87bf452484c2 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:38:45.167372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-e8b6bf73-46a2-48ea-af33-87bf452484c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:45.177417 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4378/8795] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:38:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:38:45.189532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:38:45.203666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541793e7-887c-46fd-8acb-de353ce48a8a service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:38:45.209869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:38:45.210471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:45.210744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:45.211014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:45.220827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:38:45.222462 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:38:45.232501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4418/8796] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:38:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:38:48.909488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6bf73-46a2-48ea-af33-87bf452484c2 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:38:48.926029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6bf73-46a2-48ea-af33-87bf452484c2 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:38:48.931914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:38:48.932529 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:48.932808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:48.933078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:48.934888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:48.950159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:38:48.951879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:38:48.965658 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4379/8797] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:38:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:38:55.228487 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=12 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:38:55.476637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:38:55.496601 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518184-2ecd-46c6-9008-2efc38e3b361 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:38:55.503100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad8ba6-5668-436e-8d30-075282f7e78b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:38:55.503717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8ba6-5668-436e-8d30-075282f7e78b service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:55.503988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8ba6-5668-436e-8d30-075282f7e78b service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:55.504251 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8ba6-5668-436e-8d30-075282f7e78b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:55.506037 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7fad8ba6-5668-436e-8d30-075282f7e78b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:55.521664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad8ba6-5668-436e-8d30-075282f7e78b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:38:55.523538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad8ba6-5668-436e-8d30-075282f7e78b service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:38:55.543300 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4419/8798] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:38:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:38:57.516742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:38:57.518252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:57.551634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c92763-185b-4e6e-88e5-78d5b977efd9 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:38:57.557210 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1e965-ddb8-4bc9-af99-c1693c70bb33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:38:57.558013 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e965-ddb8-4bc9-af99-c1693c70bb33 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:57.558275 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e965-ddb8-4bc9-af99-c1693c70bb33 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:57.558537 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e965-ddb8-4bc9-af99-c1693c70bb33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:57.560356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8db1e965-ddb8-4bc9-af99-c1693c70bb33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:57.575988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1e965-ddb8-4bc9-af99-c1693c70bb33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:38:57.577691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1e965-ddb8-4bc9-af99-c1693c70bb33 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:38:57.591539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4380/8799] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:38:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:38:58.833451 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80852034-068b-4437-8d6c-e691ea2752cc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:58.833951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80852034-068b-4437-8d6c-e691ea2752cc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:58.834047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80852034-068b-4437-8d6c-e691ea2752cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:58.855045 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-80852034-068b-4437-8d6c-e691ea2752cc None None] MFA Rules not processed for user `d20f8598dfec4de7b496a64e023e7a15`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:38:58.856266 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-80852034-068b-4437-8d6c-e691ea2752cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:58.863708 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4420/8800] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:38:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:38:59.204491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d609859-724c-4a4c-acc0-734284806273 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:59.205088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d609859-724c-4a4c-acc0-734284806273 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:59.205088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d609859-724c-4a4c-acc0-734284806273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:59.243706 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3d609859-724c-4a4c-acc0-734284806273 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:38:59.244940 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d609859-724c-4a4c-acc0-734284806273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:59.252149 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4381/8801] 10.222.0.26 () {58 vars in 1060 bytes} [Sat Sep 17 21:38:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:38:59.258764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80852034-068b-4437-8d6c-e691ea2752cc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:38:59.269882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80852034-068b-4437-8d6c-e691ea2752cc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:38:59.274236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:38:59.274841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:59.275117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:59.275391 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:59.277127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:59.290180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:38:59.291614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:38:59.302642 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4421/8802] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:38:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:38:59.626754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b116b9e-e55f-4c6b-9fa7-54c7148130e0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:59.627036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b116b9e-e55f-4c6b-9fa7-54c7148130e0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:59.627310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b116b9e-e55f-4c6b-9fa7-54c7148130e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:59.648981 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6b116b9e-e55f-4c6b-9fa7-54c7148130e0 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:38:59.650242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6b116b9e-e55f-4c6b-9fa7-54c7148130e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:38:59.658963 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4382/8803] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:38:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:38:59.671357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:38:59.686787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c30707-cd38-428d-8e5b-70c1e00cb8e4 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:38:59.693831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:38:59.694453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:38:59.694776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:38:59.695022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:38:59.703853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:38:59.705442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:38:59.715206 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4422/8804] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:38:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:39:00.237819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b116b9e-e55f-4c6b-9fa7-54c7148130e0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:39:00.254073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b116b9e-e55f-4c6b-9fa7-54c7148130e0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:39:00.260517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234dce7-15f4-473e-9e4d-a384be6c075b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:39:00.261120 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234dce7-15f4-473e-9e4d-a384be6c075b service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:39:00.261391 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234dce7-15f4-473e-9e4d-a384be6c075b service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:39:00.261688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234dce7-15f4-473e-9e4d-a384be6c075b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:39:00.263691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1234dce7-15f4-473e-9e4d-a384be6c075b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:39:00.278155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234dce7-15f4-473e-9e4d-a384be6c075b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:39:00.279762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234dce7-15f4-473e-9e4d-a384be6c075b service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:39:00.292392 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4383/8805] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:39:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:39:09.711142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=21 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:39:10.285570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=77 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:39:43.597678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:39:43.610008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cda59-00fa-4ba5-a3cb-df6375bda7a8 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:39:43.613977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:39:43.614611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:39:43.614888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:39:43.615150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:39:43.616845 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:39:43.629929 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:39:43.631368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:39:43.642306 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4423/8806] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:39:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:39:43.949427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408492e4-1c02-402e-ad93-b8e362b30785 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:39:43.949909 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408492e4-1c02-402e-ad93-b8e362b30785 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:39:43.950018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408492e4-1c02-402e-ad93-b8e362b30785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:39:43.970811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-408492e4-1c02-402e-ad93-b8e362b30785 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:39:43.972031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-408492e4-1c02-402e-ad93-b8e362b30785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:39:43.978978 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4384/8807] 10.222.0.26 () {58 vars in 1060 bytes} [Sat Sep 17 21:39:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:39:43.985225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:39:43.994368 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fe4e4-bdee-42d4-8180-e483ab670631 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:39:43.998528 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:39:43.999130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:39:43.999408 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:39:43.999678 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:39:44.001693 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:39:44.025583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:39:44.027094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:39:44.040977 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4424/8808] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:39:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:39:53.965280 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:39:54.033351 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=15 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:40:18.296123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be6f94-f1ee-47e0-b4e9-f9134e11707b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:40:18.296614 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be6f94-f1ee-47e0-b4e9-f9134e11707b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:40:18.296715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be6f94-f1ee-47e0-b4e9-f9134e11707b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:40:18.317150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-64be6f94-f1ee-47e0-b4e9-f9134e11707b None None] MFA Rules not processed for user `d20f8598dfec4de7b496a64e023e7a15`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:40:18.318405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-64be6f94-f1ee-47e0-b4e9-f9134e11707b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:40:18.326988 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4385/8809] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:40:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:40:28.311152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:08.487616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:08.507795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae74d7-0e70-4843-8e77-abd7996e5116 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:08.511895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:08.514274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:08.514274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:08.514274 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:08.517081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:08.545854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:08.547316 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:08.576127 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4425/8810] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:09.759986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be6f94-f1ee-47e0-b4e9-f9134e11707b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:09.769759 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be6f94-f1ee-47e0-b4e9-f9134e11707b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:09.773963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:09.774569 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:09.774845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:09.775114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:09.776994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:09.794818 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:09.796258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:09.810236 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4386/8811] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:18.573383 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=19 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:19.802620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:22.734439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:22.744204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6891bf-1ef7-4dce-bc0b-df3cd5e8a753 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:22.748841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:22.749476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:22.749774 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:22.750054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:22.752175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:22.765502 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:22.767048 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:22.778369 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4426/8812] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:32.771992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:35.222553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:35.224194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:35.284207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:35.304116 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0164c0-8b34-4e77-94de-61d18a431f60 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:35.311473 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:35.312161 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:35.312653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:35.312947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:35.317636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e9e47-bc75-47f9-ac05-00e3d316bef3 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:35.321986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:35.322517 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:35.323616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:35.324973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:35.325278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:35.325580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:35.334318 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4427/8813] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:35.340933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:35.342592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:35.357616 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4387/8814] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:45.330401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:45.351294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=35 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:46.496147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:46.505853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69d212-b696-4e4b-aa60-a8d1a73fa934 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:46.509829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:46.510425 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:46.510701 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:46.510982 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:46.521118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:46.522643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:46.532947 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4428/8815] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:49.749215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:49.778612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b811037-5c5c-4744-a804-1e2fc8e4f777 service cinder] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:49.784914 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:49.785716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:49.786000 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:49.786270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:49.788296 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:49.804149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:49.810524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:49.815944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:49.826744 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4388/8816] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:49.827816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9eb70-70c6-4e53-b71e-f8c18751e3ae service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:49.833846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:49.834171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:49.834460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:49.834732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] PATH_INFO: `/v3/users/d20f8598dfec4de7b496a64e023e7a15` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:49.835902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20f8598dfec4de7b496a64e023e7a15)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:49.838041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:49.902986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20f8598dfec4de7b496a64e023e7a15'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:49.903687 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20f8598dfec4de7b496a64e023e7a15'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:49.913033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20f8598dfec4de7b496a64e023e7a15'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:49.913722 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20f8598dfec4de7b496a64e023e7a15'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:49.926072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-181a3c95-5f8c-4736-827c-11f558b5af7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20f8598dfec4de7b496a64e023e7a15'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:49.938467 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4429/8817] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:41:49 2022] DELETE /identity/v3/users/d20f8598dfec4de7b496a64e023e7a15 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:41:50.125988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.139866 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4384b2-fb2f-4f66-b50e-79e170464fe0 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.201807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.202442 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.202720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.202988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] PATH_INFO: `/v3/projects/ad243abc13a14aa08c717d4105e24f4f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.204150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad243abc13a14aa08c717d4105e24f4f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.206122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.265347 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad243abc13a14aa08c717d4105e24f4f', 'request_id': 'req-0d5bd853-e02e-4d53-b3ec-705010bd479c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:50.283343 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad243abc13a14aa08c717d4105e24f4f', 'request_id': 'req-0d5bd853-e02e-4d53-b3ec-705010bd479c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:50.292207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad243abc13a14aa08c717d4105e24f4f', 'request_id': 'req-0d5bd853-e02e-4d53-b3ec-705010bd479c'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:50.304541 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] The token cache is being invalidate because project ad243abc13a14aa08c717d4105e24f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:41:50.304836 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:50.305773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-0d5bd853-e02e-4d53-b3ec-705010bd479c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:41:50.307480 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4389/8818] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:41:50 2022] DELETE /identity/v3/projects/ad243abc13a14aa08c717d4105e24f4f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:41:50.312577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6476b-183b-4c0b-8483-4a5fe54f0d94 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.312887 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6476b-183b-4c0b-8483-4a5fe54f0d94 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.313160 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6476b-183b-4c0b-8483-4a5fe54f0d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.334231 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-fed6476b-183b-4c0b-8483-4a5fe54f0d94 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:41:50.335548 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fed6476b-183b-4c0b-8483-4a5fe54f0d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:50.396886 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4430/8819] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:41:50.405984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a845-4845-463c-b604-e19a793d194f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.406359 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a845-4845-463c-b604-e19a793d194f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.406739 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a845-4845-463c-b604-e19a793d194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.432677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-79c0a845-4845-463c-b604-e19a793d194f None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:41:50.433923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-79c0a845-4845-463c-b604-e19a793d194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:50.442942 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4390/8820] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:41:50.447259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6476b-183b-4c0b-8483-4a5fe54f0d94 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.457273 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6476b-183b-4c0b-8483-4a5fe54f0d94 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.462824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.463432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.463708 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.463977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.464629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:41:50.464933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.466956 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.479072 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4431/8821] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:41:50.483498 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a845-4845-463c-b604-e19a793d194f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.493487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a845-4845-463c-b604-e19a793d194f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.499530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.500125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.500425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.500702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.501349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.503324 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.533224 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4391/8822] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:41:50 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:41:50.537788 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.548018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef63da-8ffb-4e68-8142-08cdf4c1cd34 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.601344 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:50.624380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.625013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.625290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.625614 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.626291 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.628441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.655171 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4432/8823] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:41:50 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:41:50.659586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.670868 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6923c93-b306-4986-94dd-56d8189314d8 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.676501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.677113 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.677385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.677680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.678325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.679951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.690743 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4392/8824] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:41:50.695068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.705562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72b9a-d370-435d-a681-bf61272bb94f admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.710884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aaa0cc-f698-41f8-bb34-449a663ed435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.711481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaa0cc-f698-41f8-bb34-449a663ed435 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.711760 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaa0cc-f698-41f8-bb34-449a663ed435 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.712034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaa0cc-f698-41f8-bb34-449a663ed435 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.712683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aaa0cc-f698-41f8-bb34-449a663ed435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.714339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aaa0cc-f698-41f8-bb34-449a663ed435 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.725574 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4433/8825] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:41:50.729658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.739892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d2ff4-bbb8-4be8-a41b-afe3546a44cb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.744863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.745595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.745861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.746141 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] PATH_INFO: `/v3/projects/41989440d1c94d3590210e79e8e5303a/users/59b52fba1a3a42338b828c5d0313343e/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.762554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41989440d1c94d3590210e79e8e5303a, user_id=59b52fba1a3a42338b828c5d0313343e, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.765298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.779394 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4393/8826] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:41:50 2022] PUT /identity/v3/projects/41989440d1c94d3590210e79e8e5303a/users/59b52fba1a3a42338b828c5d0313343e/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:41:50.784667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ad43-b9d9-411b-b42b-e05045179b97 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.785002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ad43-b9d9-411b-b42b-e05045179b97 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.785313 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6ad43-b9d9-411b-b42b-e05045179b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.804755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-84d6ad43-b9d9-411b-b42b-e05045179b97 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:41:50.806004 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-84d6ad43-b9d9-411b-b42b-e05045179b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:50.870714 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4434/8827] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:41:50.880963 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:50.882827 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:50.890149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:50.950082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ef594-3921-4a74-9089-18bf49888344 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:50.954288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ecc76-a4f6-460a-a1e0-85f94728545b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:50.954896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecc76-a4f6-460a-a1e0-85f94728545b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:50.955170 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecc76-a4f6-460a-a1e0-85f94728545b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:50.955435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ecc76-a4f6-460a-a1e0-85f94728545b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:50.969269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ecc76-a4f6-460a-a1e0-85f94728545b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:50.971289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ecc76-a4f6-460a-a1e0-85f94728545b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:50.981651 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4394/8828] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:41:55.423294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:55.423933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:55.423933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:55.455205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] MFA Rules not processed for user `59b52fba1a3a42338b828c5d0313343e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:41:55.456463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:55.525331 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4435/8829] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:41:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:41:55.535429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492e16-3a45-43cb-9cb1-89e65a8614e0 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:55.535980 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492e16-3a45-43cb-9cb1-89e65a8614e0 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:55.536082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b492e16-3a45-43cb-9cb1-89e65a8614e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:55.556204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1b492e16-3a45-43cb-9cb1-89e65a8614e0 None None] MFA Rules not processed for user `59b52fba1a3a42338b828c5d0313343e`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:41:55.557433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1b492e16-3a45-43cb-9cb1-89e65a8614e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:55.564761 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4395/8830] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:41:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:41:55.578315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:55.580466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:55.638481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:55.663460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e152b8-0917-4918-a745-e0e91f8040af None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:55.673179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14935ad3-ddb3-4315-bd5b-fcd4325e258c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:55.674059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14935ad3-ddb3-4315-bd5b-fcd4325e258c service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:55.674357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14935ad3-ddb3-4315-bd5b-fcd4325e258c service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:55.674636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14935ad3-ddb3-4315-bd5b-fcd4325e258c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:55.683631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14935ad3-ddb3-4315-bd5b-fcd4325e258c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:55.685259 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14935ad3-ddb3-4315-bd5b-fcd4325e258c service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:55.696946 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4436/8831] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:41:55.962746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492e16-3a45-43cb-9cb1-89e65a8614e0 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:41:55.973665 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b492e16-3a45-43cb-9cb1-89e65a8614e0 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:41:55.977253 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:41:55.977976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:41:55.978524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:41:55.978652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:41:55.980366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:41:55.993187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:41:55.994703 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:41:56.010134 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4396/8832] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:41:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:05.691785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:05.895540 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4fda-e858-47f8-b30c-17442170ace7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:05.895814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4fda-e858-47f8-b30c-17442170ace7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:05.896081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4fda-e858-47f8-b30c-17442170ace7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:05.921019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-3b9e4fda-e858-47f8-b30c-17442170ace7 None None] MFA Rules not processed for user `59b52fba1a3a42338b828c5d0313343e`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:05.922298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3b9e4fda-e858-47f8-b30c-17442170ace7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:05.930830 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4437/8833] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:05.999420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=33 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:08.484202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:08.494568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfe84e-7a8a-49d9-98ef-2e3034c2902d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:08.499816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:08.501070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:08.501606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:08.502085 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:08.504194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:08.518366 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:08.520018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:08.533071 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4397/8834] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:09.272242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4fda-e858-47f8-b30c-17442170ace7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:09.290163 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4fda-e858-47f8-b30c-17442170ace7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:09.298957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:09.299730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:09.300322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:09.300417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:09.302622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:09.318501 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:09.320766 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:09.344926 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4438/8835] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:18.525742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:19.329135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:26.042748 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.045348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:26.103386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.121883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444285d-32fb-49d3-94c7-aad148c66860 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.127034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.128035 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.128319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.128595 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] PATH_INFO: `/v3/users/59b52fba1a3a42338b828c5d0313343e` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.129816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b52fba1a3a42338b828c5d0313343e)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.131808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.138133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63412a3-b655-4543-b9fa-3562dccc76bd service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.143835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.144642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.145316 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.145754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.148947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:26.167375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.169223 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.186421 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4398/8836] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:26.193970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b52fba1a3a42338b828c5d0313343e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.194864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b52fba1a3a42338b828c5d0313343e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.204497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b52fba1a3a42338b828c5d0313343e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.205265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b52fba1a3a42338b828c5d0313343e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.221843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-69653642-70b4-4805-bf58-64b74a8377cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b52fba1a3a42338b828c5d0313343e'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.235045 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4439/8837] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:42:26 2022] DELETE /identity/v3/users/59b52fba1a3a42338b828c5d0313343e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:42:26.415435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.426992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659dc68-db9c-4644-b782-bec9c4069687 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.484919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.485629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.485906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.486171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] PATH_INFO: `/v3/projects/41989440d1c94d3590210e79e8e5303a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.487298 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41989440d1c94d3590210e79e8e5303a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.489208 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.538176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41989440d1c94d3590210e79e8e5303a', 'request_id': 'req-37f6eb09-713f-4593-9547-66f48fd1c9c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.546720 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41989440d1c94d3590210e79e8e5303a', 'request_id': 'req-37f6eb09-713f-4593-9547-66f48fd1c9c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.552385 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41989440d1c94d3590210e79e8e5303a', 'request_id': 'req-37f6eb09-713f-4593-9547-66f48fd1c9c8'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.563146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] The token cache is being invalidate because project 41989440d1c94d3590210e79e8e5303a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:42:26.563432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.564078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-37f6eb09-713f-4593-9547-66f48fd1c9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:26.565574 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4399/8838] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:42:26 2022] DELETE /identity/v3/projects/41989440d1c94d3590210e79e8e5303a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:26.570660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711401a2-201d-4a6a-b5c7-fbf395fea1b1 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.570944 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711401a2-201d-4a6a-b5c7-fbf395fea1b1 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.571215 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711401a2-201d-4a6a-b5c7-fbf395fea1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.592568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-711401a2-201d-4a6a-b5c7-fbf395fea1b1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:26.593859 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-711401a2-201d-4a6a-b5c7-fbf395fea1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:26.653842 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4440/8839] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:26.662063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb368c2-f4e9-448b-99ba-b2a85157f075 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.662351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb368c2-f4e9-448b-99ba-b2a85157f075 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.662652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb368c2-f4e9-448b-99ba-b2a85157f075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.686074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1bb368c2-f4e9-448b-99ba-b2a85157f075 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:26.687338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1bb368c2-f4e9-448b-99ba-b2a85157f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:26.696944 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4400/8840] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:26.701730 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711401a2-201d-4a6a-b5c7-fbf395fea1b1 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.711947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711401a2-201d-4a6a-b5c7-fbf395fea1b1 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.717005 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.717668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.717937 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.718210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.718869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:42:26.719172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.720803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.732619 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4441/8841] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:42:26.739057 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb368c2-f4e9-448b-99ba-b2a85157f075 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.751797 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb368c2-f4e9-448b-99ba-b2a85157f075 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.757613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.758216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.758493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.758760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.759390 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.761315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.773300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:26.791661 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4401/8842] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:42:26 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:42:26.796070 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.806428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3d55e-f3a5-4234-94ee-4d0a7c2f7bf0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.866390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.867088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.867372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.867642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.868270 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.870294 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.878881 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:26.894424 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4442/8843] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:42:26 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:42:26.898682 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.909441 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb437a-2503-4059-a596-e735351905aa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.915411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.916064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.916348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.916744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.917399 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.919103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.930171 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4402/8844] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:26.935265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.945692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9863d1-a63f-45eb-a42c-c8b0073ff475 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.951015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7ebb8-cb9c-4b11-a1bf-936589a0b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.951616 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ebb8-cb9c-4b11-a1bf-936589a0b50f admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.951894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ebb8-cb9c-4b11-a1bf-936589a0b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.952169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ebb8-cb9c-4b11-a1bf-936589a0b50f admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:26.953091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7ebb8-cb9c-4b11-a1bf-936589a0b50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:26.954753 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7ebb8-cb9c-4b11-a1bf-936589a0b50f admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:26.965945 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4443/8845] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:26.972220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:26.987581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044206bf-84a1-417e-9803-3ede8953717a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:26.995929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:26.996567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:26.996845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:26.997114 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] PATH_INFO: `/v3/projects/e023f41e8bc143f589b6a3a94456fcd9/users/bbbdbabc84d641e2a4069740d51a4655/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:27.013122 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e023f41e8bc143f589b6a3a94456fcd9, user_id=bbbdbabc84d641e2a4069740d51a4655, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:27.015931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:27.034974 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4403/8846] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:42:26 2022] PUT /identity/v3/projects/e023f41e8bc143f589b6a3a94456fcd9/users/bbbdbabc84d641e2a4069740d51a4655/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:27.045744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dfb11-717f-4cce-bdea-6e21254aee07 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:27.046033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dfb11-717f-4cce-bdea-6e21254aee07 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:27.046307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dfb11-717f-4cce-bdea-6e21254aee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:27.076559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-194dfb11-717f-4cce-bdea-6e21254aee07 None None] MFA Rules not processed for user `bbbdbabc84d641e2a4069740d51a4655`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:27.077854 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-194dfb11-717f-4cce-bdea-6e21254aee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:27.148022 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4444/8847] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:27.158588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:27.160503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:27.217033 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9543f5-321c-41a3-bb94-b3d4d3236088 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:27.221090 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28085f-c491-4540-9dfe-66cbf74e9298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:27.221727 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28085f-c491-4540-9dfe-66cbf74e9298 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:27.222008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28085f-c491-4540-9dfe-66cbf74e9298 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:27.222279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28085f-c491-4540-9dfe-66cbf74e9298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:27.231009 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28085f-c491-4540-9dfe-66cbf74e9298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:27.232453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28085f-c491-4540-9dfe-66cbf74e9298 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:27.241611 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4404/8848] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:37.144933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:37.237882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:37.796085 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dfb11-717f-4cce-bdea-6e21254aee07 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:37.859948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dfb11-717f-4cce-bdea-6e21254aee07 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:37.864582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:37.865252 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:37.865562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:37.865837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] PATH_INFO: `/v3/users/bbbdbabc84d641e2a4069740d51a4655` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:37.867336 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbdbabc84d641e2a4069740d51a4655)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:37.868957 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:37.917669 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbdbabc84d641e2a4069740d51a4655'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:37.918460 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbdbabc84d641e2a4069740d51a4655'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:37.925521 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbdbabc84d641e2a4069740d51a4655'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:37.926138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbdbabc84d641e2a4069740d51a4655'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:37.938172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbdbabc84d641e2a4069740d51a4655'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:37.951294 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4445/8849] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:42:37 2022] DELETE /identity/v3/users/bbbdbabc84d641e2a4069740d51a4655 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:37.956168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:37.956525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:37.956809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:37.977344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:37.978612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:38.044060 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4405/8850] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:38.053452 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.054941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:38.113369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c467b40-7314-4e4e-8cfb-42451a866c27 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.117144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1f1cf-99ba-4c93-943b-796a542af287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.117776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1f1cf-99ba-4c93-943b-796a542af287 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.118057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1f1cf-99ba-4c93-943b-796a542af287 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.118304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1f1cf-99ba-4c93-943b-796a542af287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.127665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1f1cf-99ba-4c93-943b-796a542af287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.129101 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1f1cf-99ba-4c93-943b-796a542af287 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.138730 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4446/8851] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:38.315070 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.316566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:38.334886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b74e7-619e-405d-bc14-c119a935c6a1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.343173 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.344051 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.344134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.344405 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] PATH_INFO: `/v3/projects/e023f41e8bc143f589b6a3a94456fcd9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.345602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e023f41e8bc143f589b6a3a94456fcd9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.347558 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.401273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e023f41e8bc143f589b6a3a94456fcd9', 'request_id': 'req-7659bb79-eec9-4976-96d6-bda81f9aa874'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:38.409848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e023f41e8bc143f589b6a3a94456fcd9', 'request_id': 'req-7659bb79-eec9-4976-96d6-bda81f9aa874'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:38.415285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e023f41e8bc143f589b6a3a94456fcd9', 'request_id': 'req-7659bb79-eec9-4976-96d6-bda81f9aa874'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:38.426384 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] The token cache is being invalidate because project e023f41e8bc143f589b6a3a94456fcd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:42:38.426670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:38.427299 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-7659bb79-eec9-4976-96d6-bda81f9aa874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:38.428944 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4406/8852] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:42:38 2022] DELETE /identity/v3/projects/e023f41e8bc143f589b6a3a94456fcd9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:38.434414 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f7da9-d7c0-467e-91e7-b66a33fd3862 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.434935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f7da9-d7c0-467e-91e7-b66a33fd3862 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.435038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f7da9-d7c0-467e-91e7-b66a33fd3862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.455162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-104f7da9-d7c0-467e-91e7-b66a33fd3862 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:38.456359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-104f7da9-d7c0-467e-91e7-b66a33fd3862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:38.474159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:38.518411 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4447/8853] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:38.526576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094c98-8e4c-4b52-8f9f-db399c13c80d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.526881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094c98-8e4c-4b52-8f9f-db399c13c80d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.527124 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094c98-8e4c-4b52-8f9f-db399c13c80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.548627 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-6b094c98-8e4c-4b52-8f9f-db399c13c80d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:38.549863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-6b094c98-8e4c-4b52-8f9f-db399c13c80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:38.559457 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4407/8854] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:38.563858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f7da9-d7c0-467e-91e7-b66a33fd3862 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.574193 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f7da9-d7c0-467e-91e7-b66a33fd3862 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.580210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.580823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.581099 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.581372 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.582054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:42:38.582359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.583962 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.596792 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4448/8855] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:42:38.601134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b094c98-8e4c-4b52-8f9f-db399c13c80d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.610957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b094c98-8e4c-4b52-8f9f-db399c13c80d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.616717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.617331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.617653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.617923 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.618565 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.620482 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.651269 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4408/8856] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:42:38 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:42:38.656147 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.670178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a4c19-5a93-467a-91b3-91c8bbcc27a9 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.730272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.730883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.731162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.731434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.732074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.734062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.758212 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4449/8857] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:42:38 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:42:38.763729 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.774658 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24cc88-071b-47ea-bc14-4d7d1529cf6e admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.780348 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.780943 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.781222 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.781520 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.782161 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.783771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.794606 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4409/8858] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:38.798839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.809297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4ba5-d687-4dee-9f2f-31ba7c3397a0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.814805 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01346c4a-f0b2-407f-b6eb-928e4d5ff500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.815373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346c4a-f0b2-407f-b6eb-928e4d5ff500 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.815651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346c4a-f0b2-407f-b6eb-928e4d5ff500 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.815922 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346c4a-f0b2-407f-b6eb-928e4d5ff500 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.816568 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01346c4a-f0b2-407f-b6eb-928e4d5ff500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.818209 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01346c4a-f0b2-407f-b6eb-928e4d5ff500 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.829262 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4450/8859] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:38.833471 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:38.844379 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5c1ca-399c-4c8a-8a20-89ca2bbe4f4d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:38.849932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:38.850582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.850859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.851155 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] PATH_INFO: `/v3/projects/63f48df0a8d940ac9ff4d385a5b74519/users/10346616193546dcb457fd7f90d8b167/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.866714 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f48df0a8d940ac9ff4d385a5b74519, user_id=10346616193546dcb457fd7f90d8b167, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:38.869388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:38.885505 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4410/8860] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:42:38 2022] PUT /identity/v3/projects/63f48df0a8d940ac9ff4d385a5b74519/users/10346616193546dcb457fd7f90d8b167/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:38.896495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e7395-107a-4363-95f8-712af084aced None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:38.896784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e7395-107a-4363-95f8-712af084aced None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:38.897053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e7395-107a-4363-95f8-712af084aced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:38.927820 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-ee5e7395-107a-4363-95f8-712af084aced None None] MFA Rules not processed for user `10346616193546dcb457fd7f90d8b167`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:38.929032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee5e7395-107a-4363-95f8-712af084aced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:38.993763 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4451/8861] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:39.004552 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:39.006409 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:39.028408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:39.064463 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea5229-b517-4239-8fdd-59333d607914 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:39.068745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2a95b-89dd-41f0-9ee3-af8bb2bb9777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:39.069277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a95b-89dd-41f0-9ee3-af8bb2bb9777 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:39.069581 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a95b-89dd-41f0-9ee3-af8bb2bb9777 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:39.069854 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a95b-89dd-41f0-9ee3-af8bb2bb9777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:39.078639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2a95b-89dd-41f0-9ee3-af8bb2bb9777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:39.080081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2a95b-89dd-41f0-9ee3-af8bb2bb9777 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:39.089235 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4411/8862] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:43.564662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e7395-107a-4363-95f8-712af084aced None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:43.584428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:43.631288 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e7395-107a-4363-95f8-712af084aced None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:43.636242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:43.636869 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:43.637142 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:43.637412 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] PATH_INFO: `/v3/users/10346616193546dcb457fd7f90d8b167` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:43.638619 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10346616193546dcb457fd7f90d8b167)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:43.640555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:43.689157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10346616193546dcb457fd7f90d8b167'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:43.690098 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10346616193546dcb457fd7f90d8b167'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:43.698770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10346616193546dcb457fd7f90d8b167'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:43.699441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10346616193546dcb457fd7f90d8b167'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:43.711775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10346616193546dcb457fd7f90d8b167'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:43.724388 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4452/8863] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:42:43 2022] DELETE /identity/v3/users/10346616193546dcb457fd7f90d8b167 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:43.729134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:43.729663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:43.729763 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:43.748750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:43.749960 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:43.813615 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4412/8864] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:43.823871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:43.825480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:43.884904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec845ff-3ac2-4fed-8a04-edbc5b1b42a5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:43.889019 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c203ca-8d79-42ea-85f7-c5f76becb9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:43.889668 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c203ca-8d79-42ea-85f7-c5f76becb9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:43.889948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c203ca-8d79-42ea-85f7-c5f76becb9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:43.890220 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c203ca-8d79-42ea-85f7-c5f76becb9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:43.900179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c203ca-8d79-42ea-85f7-c5f76becb9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:43.901633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c203ca-8d79-42ea-85f7-c5f76becb9f6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:43.911417 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4453/8865] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:44.099111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.100982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:44.115450 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9f5e1-0600-40c9-9c26-6dd68c8a16da None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.121276 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.122059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.122406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.122750 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] PATH_INFO: `/v3/projects/63f48df0a8d940ac9ff4d385a5b74519` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.124058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f48df0a8d940ac9ff4d385a5b74519)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.126159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.175333 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f48df0a8d940ac9ff4d385a5b74519', 'request_id': 'req-1112b053-03b0-4927-9101-e7ea1128ab40'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:44.183691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f48df0a8d940ac9ff4d385a5b74519', 'request_id': 'req-1112b053-03b0-4927-9101-e7ea1128ab40'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:44.189446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f48df0a8d940ac9ff4d385a5b74519', 'request_id': 'req-1112b053-03b0-4927-9101-e7ea1128ab40'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:44.200805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] The token cache is being invalidate because project 63f48df0a8d940ac9ff4d385a5b74519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:42:44.201076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:44.201816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-1112b053-03b0-4927-9101-e7ea1128ab40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:42:44.203446 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4413/8866] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:42:44 2022] DELETE /identity/v3/projects/63f48df0a8d940ac9ff4d385a5b74519 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:44.209644 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1c17a-9230-46eb-b66f-b58410a187a2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.209931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1c17a-9230-46eb-b66f-b58410a187a2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.210204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1c17a-9230-46eb-b66f-b58410a187a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.232803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-62c1c17a-9230-46eb-b66f-b58410a187a2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:44.234075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-62c1c17a-9230-46eb-b66f-b58410a187a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:44.269523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:44.300035 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4454/8867] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:44.308406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5a453-1f1b-4048-9159-d987ecaf4d7d None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.308708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5a453-1f1b-4048-9159-d987ecaf4d7d None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.308986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5a453-1f1b-4048-9159-d987ecaf4d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.330238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-34a5a453-1f1b-4048-9159-d987ecaf4d7d None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:44.331453 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-34a5a453-1f1b-4048-9159-d987ecaf4d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:44.343174 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4414/8868] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:44.344673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1c17a-9230-46eb-b66f-b58410a187a2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.354947 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1c17a-9230-46eb-b66f-b58410a187a2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.359996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.360615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.360894 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.361167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.361871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:42:44.362170 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.363790 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.376530 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4455/8869] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:42:44.380848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5a453-1f1b-4048-9159-d987ecaf4d7d None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.390816 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5a453-1f1b-4048-9159-d987ecaf4d7d None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.395984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.396595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.396929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.397182 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.397858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.399745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.428321 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4415/8870] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:42:44 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:42:44.432537 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.443402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e26f7-aeb9-411c-b74a-fd4034bef87c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.501752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.501752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.502219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.502219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.502919 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.504915 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.526887 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4456/8871] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:42:44 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:42:44.531310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.542604 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed7ca5-c8d9-4afb-a042-89e894c504ae admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.547824 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.548423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.548704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.549003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.549689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.551312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.561348 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4416/8872] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:44.565529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.575732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb711390-ad17-415d-a1d2-538b917b21d1 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.580726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e885726-c318-4c8a-bf46-33714ca7b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.581325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e885726-c318-4c8a-bf46-33714ca7b428 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.581639 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e885726-c318-4c8a-bf46-33714ca7b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.581909 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e885726-c318-4c8a-bf46-33714ca7b428 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.582554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e885726-c318-4c8a-bf46-33714ca7b428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.584159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e885726-c318-4c8a-bf46-33714ca7b428 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.596291 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4457/8873] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:44.600688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.611864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44651-bf36-440e-8981-c1fa47792e8a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.616967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.617660 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.617932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.618206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] PATH_INFO: `/v3/projects/e3015a88384447ce8ee07537d685f3a3/users/806819c3594d48fe9bf3d0cce5fee164/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.620458 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:44.635993 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3015a88384447ce8ee07537d685f3a3, user_id=806819c3594d48fe9bf3d0cce5fee164, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.638820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.652767 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4417/8874] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:42:44 2022] PUT /identity/v3/projects/e3015a88384447ce8ee07537d685f3a3/users/806819c3594d48fe9bf3d0cce5fee164/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:42:44.658268 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54400-c317-403f-aec0-85da736e1e47 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.658570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54400-c317-403f-aec0-85da736e1e47 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.658862 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54400-c317-403f-aec0-85da736e1e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.678550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c3f54400-c317-403f-aec0-85da736e1e47 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:44.679690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c3f54400-c317-403f-aec0-85da736e1e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:44.744198 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4458/8875] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:44.754351 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:44.756380 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:44.815828 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2405d-b00f-4221-ad22-8704340e2554 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:44.819941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:44.820544 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:44.820823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:44.821097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:44.830715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:44.832167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:44.842304 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4418/8876] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:48.667222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858643-8d69-44c2-952f-b1806f98aab0 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:48.667754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858643-8d69-44c2-952f-b1806f98aab0 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:48.667851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858643-8d69-44c2-952f-b1806f98aab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:48.700219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-87858643-8d69-44c2-952f-b1806f98aab0 None None] MFA Rules not processed for user `806819c3594d48fe9bf3d0cce5fee164`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:48.701775 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-87858643-8d69-44c2-952f-b1806f98aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:48.739990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:48.772802 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4459/8877] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:48.777577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:48.788072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9587-fde1-4291-a0d2-9ed35ee7a1fe service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:48.793563 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:48.794415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:48.794697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:48.794962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:48.795596 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:48.797535 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:48.827539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4419/8878] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:42:48 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:42:48.831851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87858643-8d69-44c2-952f-b1806f98aab0 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:48.846398 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87858643-8d69-44c2-952f-b1806f98aab0 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:48.907752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:48.908376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:48.908651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:48.908921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:48.909594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:48.911523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:48.941141 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4460/8879] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:42:48 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:42:48.942434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:48.953365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea5b2-03e9-459a-89cb-9428f9d26566 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:48.958074 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:48.958688 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:48.959217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:48.959217 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:48.959848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:48.961454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:48.974772 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4420/8880] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:48.979180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:48.990073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f81b-194a-4e6c-a895-b45bbee157fc admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:48.994752 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:48.995423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:48.995741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:48.996014 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] PATH_INFO: `/v3/projects/cdfc8554448b4472b2fc0da4dc4dbf1c/users/75e0fcc96f3943a4bec356ed88e30bc0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:49.013874 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdfc8554448b4472b2fc0da4dc4dbf1c, user_id=75e0fcc96f3943a4bec356ed88e30bc0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:49.016707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:49.031765 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4461/8881] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:42:48 2022] PUT /identity/v3/projects/cdfc8554448b4472b2fc0da4dc4dbf1c/users/75e0fcc96f3943a4bec356ed88e30bc0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:49.035843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:49.046481 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ab1ba-446a-4f11-ba68-81666bd190d3 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:49.105268 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c1624-dfd0-4f1d-baaf-0ad2291def42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:49.105922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c1624-dfd0-4f1d-baaf-0ad2291def42 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:49.106197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c1624-dfd0-4f1d-baaf-0ad2291def42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:49.106474 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c1624-dfd0-4f1d-baaf-0ad2291def42 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:49.107206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c1624-dfd0-4f1d-baaf-0ad2291def42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:49.108728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c1624-dfd0-4f1d-baaf-0ad2291def42 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:49.119522 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4421/8882] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:42:49.123927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:49.135195 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64cbbb-4c73-4fa1-8dd0-2af1b014bc8a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:49.140361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c4530-a6fe-4bd2-8559-09f3d6b73fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:49.141034 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c4530-a6fe-4bd2-8559-09f3d6b73fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:49.141310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c4530-a6fe-4bd2-8559-09f3d6b73fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:49.141623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c4530-a6fe-4bd2-8559-09f3d6b73fb1 admin admin] PATH_INFO: `/v3/projects/cdfc8554448b4472b2fc0da4dc4dbf1c/users/75e0fcc96f3943a4bec356ed88e30bc0/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:49.143666 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c4530-a6fe-4bd2-8559-09f3d6b73fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdfc8554448b4472b2fc0da4dc4dbf1c, user_id=75e0fcc96f3943a4bec356ed88e30bc0, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:49.146389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c4530-a6fe-4bd2-8559-09f3d6b73fb1 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:49.157074 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4462/8883] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:42:49 2022] PUT /identity/v3/projects/cdfc8554448b4472b2fc0da4dc4dbf1c/users/75e0fcc96f3943a4bec356ed88e30bc0/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:42:54.251878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0c9a0-31cc-44fb-bd64-e44e1f44cd76 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:54.252431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0c9a0-31cc-44fb-bd64-e44e1f44cd76 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:54.252431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0c9a0-31cc-44fb-bd64-e44e1f44cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:54.282404 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7fb0c9a0-31cc-44fb-bd64-e44e1f44cd76 None None] MFA Rules not processed for user `75e0fcc96f3943a4bec356ed88e30bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:54.284005 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7fb0c9a0-31cc-44fb-bd64-e44e1f44cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:54.338958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:54.372634 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4422/8884] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:42:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:54.410673 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef010-8c25-485a-95d6-efa4e6ae5688 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:54.411217 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef010-8c25-485a-95d6-efa4e6ae5688 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:54.411729 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef010-8c25-485a-95d6-efa4e6ae5688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:54.474942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d6aef010-8c25-485a-95d6-efa4e6ae5688 None None] MFA Rules not processed for user `94ce9c64a0bb478485c47ebedd5a29f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:54.476175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d6aef010-8c25-485a-95d6-efa4e6ae5688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:54.544363 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4463/8885] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Sep 17 21:42:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:54.551815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0c9a0-31cc-44fb-bd64-e44e1f44cd76 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:54.563383 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0c9a0-31cc-44fb-bd64-e44e1f44cd76 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:54.568787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca97add-379e-43ec-acf1-4e879fdc638a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:54.569597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97add-379e-43ec-acf1-4e879fdc638a service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:54.569976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97add-379e-43ec-acf1-4e879fdc638a service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:54.570871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97add-379e-43ec-acf1-4e879fdc638a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:54.633648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca97add-379e-43ec-acf1-4e879fdc638a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:54.635092 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca97add-379e-43ec-acf1-4e879fdc638a service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:54.644310 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4423/8886] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:42:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:42:54.799286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb2c1-5a19-439c-9f27-2b1ad363d648 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:54.799527 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb2c1-5a19-439c-9f27-2b1ad363d648 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:54.799821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadb2c1-5a19-439c-9f27-2b1ad363d648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:54.808828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:54.855783 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-adadb2c1-5a19-439c-9f27-2b1ad363d648 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:54.857015 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-adadb2c1-5a19-439c-9f27-2b1ad363d648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:54.927746 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4464/8887] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:42:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:55.040412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2c83b-f268-4e3b-aff6-7686b119f947 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:55.040636 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2c83b-f268-4e3b-aff6-7686b119f947 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:55.040889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2c83b-f268-4e3b-aff6-7686b119f947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:55.097412 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-a7a2c83b-f268-4e3b-aff6-7686b119f947 None None] MFA Rules not processed for user `8eea4641bfd940fe8a94ec79484ef4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:42:55.098692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a7a2c83b-f268-4e3b-aff6-7686b119f947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:42:55.171516 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4424/8888] 10.222.0.26 () {58 vars in 1040 bytes} [Sat Sep 17 21:42:55 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:42:55.177895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadb2c1-5a19-439c-9f27-2b1ad363d648 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:42:55.189697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadb2c1-5a19-439c-9f27-2b1ad363d648 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:42:55.194599 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee44186-d6c8-471c-810a-84b6b4d5620e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:42:55.195205 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44186-d6c8-471c-810a-84b6b4d5620e service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:42:55.195480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44186-d6c8-471c-810a-84b6b4d5620e service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:42:55.195750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee44186-d6c8-471c-810a-84b6b4d5620e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:42:55.203963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee44186-d6c8-471c-810a-84b6b4d5620e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:42:55.205577 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee44186-d6c8-471c-810a-84b6b4d5620e service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:42:55.213758 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4465/8889] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:42:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:43:04.001295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87458f-1d7b-4952-8cc9-6bd52dd93341 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:43:04.001954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87458f-1d7b-4952-8cc9-6bd52dd93341 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:43:04.001954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87458f-1d7b-4952-8cc9-6bd52dd93341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:43:04.022063 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3e87458f-1d7b-4952-8cc9-6bd52dd93341 None None] MFA Rules not processed for user `806819c3594d48fe9bf3d0cce5fee164`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:43:04.023251 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3e87458f-1d7b-4952-8cc9-6bd52dd93341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:43:04.031539 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4425/8890] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:43:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:43:05.210382 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=44 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:43:05.882072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:43:05.882361 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:43:05.882632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:43:05.903510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:43:05.905262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:43:05.913020 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4466/8891] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:43:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:43:05.925718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87458f-1d7b-4952-8cc9-6bd52dd93341 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:43:05.937931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87458f-1d7b-4952-8cc9-6bd52dd93341 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:43:05.943266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:43:05.943889 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:43:05.944235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:43:05.944573 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:43:05.953269 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:43:05.955031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:43:05.963604 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4426/8892] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:43:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:43:15.897945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:43:15.960423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:43:30.613317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:43:30.619444 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:43:30.678709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df2bd9-aa1e-42c4-85be-8d265146bd35 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:43:30.683243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:43:30.683896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:43:30.684167 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:43:30.684432 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:43:30.694421 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:43:30.695951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:43:30.705799 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4467/8893] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:43:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:43:40.701447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=26 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:44:08.028933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:44:08.030641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:44:08.107386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb5ec8-b6a9-407d-b6cd-f5a6c19bd46c service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:44:08.113302 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33e482-2562-4056-b981-ec2ea1f7632c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:44:08.113939 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33e482-2562-4056-b981-ec2ea1f7632c service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:44:08.114216 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33e482-2562-4056-b981-ec2ea1f7632c service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:44:08.114480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33e482-2562-4056-b981-ec2ea1f7632c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:44:08.116533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3d33e482-2562-4056-b981-ec2ea1f7632c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:44:08.131751 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33e482-2562-4056-b981-ec2ea1f7632c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:44:08.133335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33e482-2562-4056-b981-ec2ea1f7632c service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:44:08.146436 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4427/8894] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:44:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:44:10.194539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:44:10.211136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b642ee-c50c-43d5-883d-3cbf2e3c5f77 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:44:10.217920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:44:10.218518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:44:10.218786 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:44:10.219053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:44:10.221140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:44:10.236106 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:44:10.237712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:44:10.251036 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4468/8895] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:44:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:44:18.139608 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=32 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:44:20.243781 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:46:09.986312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef8365-8f31-4b1f-bd93-6abcc5d502a5 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:46:09.986951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef8365-8f31-4b1f-bd93-6abcc5d502a5 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:46:09.986951 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef8365-8f31-4b1f-bd93-6abcc5d502a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:46:10.022125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-89ef8365-8f31-4b1f-bd93-6abcc5d502a5 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:46:10.023438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-89ef8365-8f31-4b1f-bd93-6abcc5d502a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:46:10.094467 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4428/8896] 10.222.0.60 () {58 vars in 1015 bytes} [Sat Sep 17 21:46:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:46:10.107546 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:46:10.153866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5db55-a18e-468f-a8ea-a442e409ffb3 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:46:10.160138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:46:10.160751 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:46:10.161027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:46:10.161297 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:46:10.170431 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:46:10.172028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:46:10.182789 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4469/8897] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:46:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:46:11.228690 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed540cea-cd00-4ac8-909a-f85a49f539a3 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:46:11.229285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed540cea-cd00-4ac8-909a-f85a49f539a3 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:46:11.229285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed540cea-cd00-4ac8-909a-f85a49f539a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:46:11.251808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-ed540cea-cd00-4ac8-909a-f85a49f539a3 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:46:11.253004 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ed540cea-cd00-4ac8-909a-f85a49f539a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:46:11.263269 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4429/8898] 10.222.0.50 () {58 vars in 1015 bytes} [Sat Sep 17 21:46:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:46:11.277319 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:46:11.294254 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2613c-c509-4da8-853b-6cc5db662e58 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:46:11.300921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:46:11.301526 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:46:11.301832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:46:11.302096 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:46:11.311044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:46:11.312628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:46:11.322775 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4470/8899] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:46:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:46:21.244566 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=28 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:46:21.318589 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=26 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:03.462768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed540cea-cd00-4ac8-909a-f85a49f539a3 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:03.478734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed540cea-cd00-4ac8-909a-f85a49f539a3 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:03.484550 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328d100-e27c-43f5-8ffc-337e2520b499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:03.485154 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328d100-e27c-43f5-8ffc-337e2520b499 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:03.485432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328d100-e27c-43f5-8ffc-337e2520b499 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:03.485853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328d100-e27c-43f5-8ffc-337e2520b499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:03.487982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5328d100-e27c-43f5-8ffc-337e2520b499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:03.570233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328d100-e27c-43f5-8ffc-337e2520b499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:03.571691 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328d100-e27c-43f5-8ffc-337e2520b499 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:03.585913 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4430/8900] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:47:04.112963 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:04.122961 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b35f0b-bd4d-41c0-876c-637321e6be0d service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:04.127229 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:04.127934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:04.128216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:04.128488 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:04.130549 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:04.162171 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:04.163663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:04.177566 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4471/8901] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:47:08.119159 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:08.119601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:08.119749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:08.170731 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] MFA Rules not processed for user `d45ab41902864586811c81948540fdac`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:47:08.171947 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:08.224101 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4431/8902] 10.222.0.26 () {58 vars in 1061 bytes} [Sat Sep 17 21:47:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:47:08.230773 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:08.240516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d084b-630e-433c-8824-6a9f5078795e service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:08.245197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71989fc7-d66f-4d68-9373-2ce8f911fdfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45ab41902864586811c81948540fdac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:08.245891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71989fc7-d66f-4d68-9373-2ce8f911fdfe service cinder] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:08.246208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71989fc7-d66f-4d68-9373-2ce8f911fdfe service cinder] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:08.246479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71989fc7-d66f-4d68-9373-2ce8f911fdfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:08.248318 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-71989fc7-d66f-4d68-9373-2ce8f911fdfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:08.264896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71989fc7-d66f-4d68-9373-2ce8f911fdfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:08.266366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71989fc7-d66f-4d68-9373-2ce8f911fdfe service cinder] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:08.279893 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4472/8903] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:47:11.770707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:11.772424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:11.793176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec25b80-8cf9-4a8e-90fc-5ecef15f94a9 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:11.799607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:11.800218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:11.800494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:11.800791 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:11.802601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:11.819174 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:11.820789 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:11.834450 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4432/8904] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:47:14.269483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5c08-8988-4ff7-8fbf-3b83d83658d8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:14.269974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5c08-8988-4ff7-8fbf-3b83d83658d8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:14.270073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac5c08-8988-4ff7-8fbf-3b83d83658d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:14.290960 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-eeac5c08-8988-4ff7-8fbf-3b83d83658d8 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:47:14.292661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-eeac5c08-8988-4ff7-8fbf-3b83d83658d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:14.301672 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4473/8905] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:47:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:47:14.345785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:14.357929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddff58-e430-4f56-bf11-4caaa9913c80 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:14.363744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:14.364344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:14.364617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:14.364881 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:14.373340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:14.374965 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:14.383444 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4433/8906] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:47:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:47:14.902895 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6676a-eb28-4884-8c39-806b511586a3 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:14.903189 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6676a-eb28-4884-8c39-806b511586a3 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:14.903555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6676a-eb28-4884-8c39-806b511586a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:14.924463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-81b6676a-eb28-4884-8c39-806b511586a3 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:47:14.925743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-81b6676a-eb28-4884-8c39-806b511586a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:14.934031 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4474/8907] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:47:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:47:14.946513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:14.957367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105206b-024c-4b7e-b73a-a6eefa28155e service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:14.963266 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:14.963875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:14.964152 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:14.964423 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:14.975041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:14.976656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:14.984804 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4434/8908] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:47:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:47:23.636339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6078-a8b3-4145-b6ff-4cd926f99e25 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:23.636912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6078-a8b3-4145-b6ff-4cd926f99e25 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:23.636912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face6078-a8b3-4145-b6ff-4cd926f99e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:23.664815 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-face6078-a8b3-4145-b6ff-4cd926f99e25 None None] MFA Rules not processed for user `806819c3594d48fe9bf3d0cce5fee164`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:47:23.666064 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-face6078-a8b3-4145-b6ff-4cd926f99e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:23.674428 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4475/8909] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:47:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:47:23.941378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:23.952131 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac210f-8fa3-45c3-adc4-8619bc5f30ec service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:23.956586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb4aeb-00bf-41bf-b036-762461726cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:23.957202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb4aeb-00bf-41bf-b036-762461726cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:23.957494 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb4aeb-00bf-41bf-b036-762461726cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:23.957776 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb4aeb-00bf-41bf-b036-762461726cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:23.959811 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8fcb4aeb-00bf-41bf-b036-762461726cb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:23.973445 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb4aeb-00bf-41bf-b036-762461726cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:23.974922 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb4aeb-00bf-41bf-b036-762461726cb7 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:23.986003 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4435/8910] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:47:24.336559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face6078-a8b3-4145-b6ff-4cd926f99e25 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:24.352091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face6078-a8b3-4145-b6ff-4cd926f99e25 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:24.357097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:24.357759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:24.358074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:24.358350 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:24.360062 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:24.371899 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:24.373686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:24.389289 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4476/8911] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:47:25.578388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:25.578875 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:25.579098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:25.600354 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:47:25.601778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:25.609543 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4436/8912] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:47:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:47:25.622506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:25.634721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12279c-e75b-46ed-899a-317c971e841c service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:25.640114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:25.640728 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:25.641009 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:25.641281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:25.649791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:25.651465 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:25.659801 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4477/8913] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:47:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:47:35.594879 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=94 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:35.656449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=38 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:39.832199 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:39.834103 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:39.855205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1cc34-0fd0-4e4e-965e-33dbe7d04882 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:39.861369 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:39.862072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:39.862374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:39.862643 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:39.864588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:39.880394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:39.882002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:39.897000 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4437/8914] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:47:49.890191 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:49.954726 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:47:49.970632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aba42-9710-4db7-a0b3-8c8fc66bd93a service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:47:49.977038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b9d69-2866-4d12-8692-2d7c28763686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:47:49.977778 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b9d69-2866-4d12-8692-2d7c28763686 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:47:49.978094 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b9d69-2866-4d12-8692-2d7c28763686 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:47:49.978370 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b9d69-2866-4d12-8692-2d7c28763686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:47:49.980573 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b29b9d69-2866-4d12-8692-2d7c28763686 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:47:49.996608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b9d69-2866-4d12-8692-2d7c28763686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:47:49.998263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b9d69-2866-4d12-8692-2d7c28763686 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:47:50.013620 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4478/8915] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:47:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:48:00.008119 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:48:13.383407 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:48:13.397906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62a8a0-0c59-4952-bd2b-729fbfff0173 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:48:13.404279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd97b90-c623-40a9-9d4d-c99e5333f6f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:48:13.404882 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd97b90-c623-40a9-9d4d-c99e5333f6f1 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:48:13.405176 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd97b90-c623-40a9-9d4d-c99e5333f6f1 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:48:13.405444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd97b90-c623-40a9-9d4d-c99e5333f6f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:48:13.413975 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd97b90-c623-40a9-9d4d-c99e5333f6f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:48:13.415930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd97b90-c623-40a9-9d4d-c99e5333f6f1 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:48:13.425515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4438/8916] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:48:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:48:23.420950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:48:57.967492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ec605-8613-40d1-89a0-9b4a6d8324d5 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:48:57.968127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ec605-8613-40d1-89a0-9b4a6d8324d5 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:48:57.968127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ec605-8613-40d1-89a0-9b4a6d8324d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:48:58.017889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-a36ec605-8613-40d1-89a0-9b4a6d8324d5 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:48:58.019089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-a36ec605-8613-40d1-89a0-9b4a6d8324d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:48:58.029477 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4479/8917] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 17 21:48:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:48:58.036457 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9580-ffa8-4756-b2a1-dc3118275cce None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:48:58.037002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9580-ffa8-4756-b2a1-dc3118275cce None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:48:58.037002 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb9580-ffa8-4756-b2a1-dc3118275cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:48:58.058128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c9eb9580-ffa8-4756-b2a1-dc3118275cce None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:48:58.059332 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c9eb9580-ffa8-4756-b2a1-dc3118275cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:48:58.070246 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4439/8918] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:48:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:48:58.082128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ec605-8613-40d1-89a0-9b4a6d8324d5 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:48:58.096707 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ec605-8613-40d1-89a0-9b4a6d8324d5 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:48:58.102041 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:48:58.102642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:48:58.102924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:48:58.103190 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:48:58.114999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:48:58.116493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:48:58.128191 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4480/8919] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:48:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:48:58.626692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727179e5-4a8a-46e7-a8b7-8ff58d7ddd03 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:48:58.626957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727179e5-4a8a-46e7-a8b7-8ff58d7ddd03 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:48:58.627224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727179e5-4a8a-46e7-a8b7-8ff58d7ddd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:48:58.648732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-727179e5-4a8a-46e7-a8b7-8ff58d7ddd03 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:48:58.650007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-727179e5-4a8a-46e7-a8b7-8ff58d7ddd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:48:58.656919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4440/8920] 10.222.0.50 () {58 vars in 1015 bytes} [Sat Sep 17 21:48:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:48:58.671495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:48:58.678996 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3216b-ea0c-4b1e-8397-5905f2dd3208 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:48:58.684073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fd550-34c3-4cda-bbd6-cb55192f25d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:48:58.684710 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fd550-34c3-4cda-bbd6-cb55192f25d6 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:48:58.685053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fd550-34c3-4cda-bbd6-cb55192f25d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:48:58.685554 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fd550-34c3-4cda-bbd6-cb55192f25d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:48:58.708369 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fd550-34c3-4cda-bbd6-cb55192f25d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:48:58.711481 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fd550-34c3-4cda-bbd6-cb55192f25d6 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:48:58.723504 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4481/8921] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:48:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:03.618950 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5fca-870f-4d99-94eb-3e1c2149c874 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:03.619532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5fca-870f-4d99-94eb-3e1c2149c874 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:03.619532 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5fca-870f-4d99-94eb-3e1c2149c874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:03.639782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4aec5fca-870f-4d99-94eb-3e1c2149c874 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:03.640982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4aec5fca-870f-4d99-94eb-3e1c2149c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:03.651214 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4441/8922] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 17 21:49:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:03.657226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d12fd8-4643-4c45-918f-ea383ce2005d None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:03.657776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d12fd8-4643-4c45-918f-ea383ce2005d None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:03.657776 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d12fd8-4643-4c45-918f-ea383ce2005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:03.678479 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-27d12fd8-4643-4c45-918f-ea383ce2005d None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:03.679721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-27d12fd8-4643-4c45-918f-ea383ce2005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:03.690396 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4482/8923] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:49:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:03.701764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec5fca-870f-4d99-94eb-3e1c2149c874 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:03.711883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec5fca-870f-4d99-94eb-3e1c2149c874 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:03.715871 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:03.716465 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:03.716742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:03.717011 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:03.728716 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:03.730214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:03.741546 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4442/8924] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:04.476623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae3cbe-1034-4bdf-836d-73ddad0797f2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:04.476924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae3cbe-1034-4bdf-836d-73ddad0797f2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:04.477177 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae3cbe-1034-4bdf-836d-73ddad0797f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:04.498447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f0ae3cbe-1034-4bdf-836d-73ddad0797f2 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:04.498864 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f0ae3cbe-1034-4bdf-836d-73ddad0797f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:04.509193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4483/8925] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:49:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:04.523242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:04.538605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057c696-71d8-4363-ab82-56f81ef47a2e service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:04.544207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:04.544806 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:04.545080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:04.545349 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:04.555473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:04.557329 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:04.569160 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4443/8926] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:06.111622 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854d2c2-0e32-4243-9bff-e4ed9175bd4e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:06.112199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854d2c2-0e32-4243-9bff-e4ed9175bd4e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:06.112199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854d2c2-0e32-4243-9bff-e4ed9175bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:06.132702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-5854d2c2-0e32-4243-9bff-e4ed9175bd4e None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:06.133991 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5854d2c2-0e32-4243-9bff-e4ed9175bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:06.142013 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4484/8927] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:49:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:06.184007 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:06.196734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7736a0-4d64-4bac-8471-a079813a30f6 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:06.203455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b065cae-735b-49cc-8747-2ac6c41681ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:06.204059 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065cae-735b-49cc-8747-2ac6c41681ac service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:06.204338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065cae-735b-49cc-8747-2ac6c41681ac service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:06.204610 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065cae-735b-49cc-8747-2ac6c41681ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:06.213815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b065cae-735b-49cc-8747-2ac6c41681ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:06.215417 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b065cae-735b-49cc-8747-2ac6c41681ac service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:06.224447 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4444/8928] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:49:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:49:06.605822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854d2c2-0e32-4243-9bff-e4ed9175bd4e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:06.615613 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854d2c2-0e32-4243-9bff-e4ed9175bd4e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:06.619842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:06.620441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:06.620718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:06.620988 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:06.695340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:06.696831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:06.717046 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4485/8929] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:09.996102 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9342f0-041a-4281-b594-dfc0ad8d3456 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:09.996597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9342f0-041a-4281-b594-dfc0ad8d3456 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:09.996723 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9342f0-041a-4281-b594-dfc0ad8d3456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:10.054904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-0d9342f0-041a-4281-b594-dfc0ad8d3456 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:10.056258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-0d9342f0-041a-4281-b594-dfc0ad8d3456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:10.066132 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4445/8930] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Sep 17 21:49:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:10.072392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:10.084277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129bf51-defe-4d50-8121-abf778ada4fd service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:10.089762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:10.090376 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:10.090652 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:10.090927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:10.092690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:10.107148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:10.108759 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:10.121911 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4486/8931] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:13.011808 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9342f0-041a-4281-b594-dfc0ad8d3456 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:13.025027 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9342f0-041a-4281-b594-dfc0ad8d3456 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:13.032382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:13.033648 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:13.033931 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:13.034207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:13.036340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:13.052921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:13.054708 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:13.071351 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4446/8932] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:13.092814 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:13.104186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef2db3-be19-411a-b5fa-92c30883f97e service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:13.109878 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:13.110525 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:13.110802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:13.111074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] PATH_INFO: `/v3/users/806819c3594d48fe9bf3d0cce5fee164` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:13.112216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806819c3594d48fe9bf3d0cce5fee164)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:13.114201 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:13.163828 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:13.166910 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806819c3594d48fe9bf3d0cce5fee164'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.167598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806819c3594d48fe9bf3d0cce5fee164'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.176135 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806819c3594d48fe9bf3d0cce5fee164'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.176848 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806819c3594d48fe9bf3d0cce5fee164'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.190445 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806819c3594d48fe9bf3d0cce5fee164'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.203850 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4487/8933] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:49:13 2022] DELETE /identity/v3/users/806819c3594d48fe9bf3d0cce5fee164 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:49:13.392654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:13.407374 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb913bf7-7a06-4143-8c79-1441e7ed6fd7 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:13.467734 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:13.468400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:13.468680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:13.468983 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] PATH_INFO: `/v3/projects/e3015a88384447ce8ee07537d685f3a3` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:13.470144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3015a88384447ce8ee07537d685f3a3)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:13.472127 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:13.519400 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3015a88384447ce8ee07537d685f3a3', 'request_id': 'req-364f9b51-243b-4dcd-951c-2bfe8b44a09b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.528160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3015a88384447ce8ee07537d685f3a3', 'request_id': 'req-364f9b51-243b-4dcd-951c-2bfe8b44a09b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.530367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:13.535905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3015a88384447ce8ee07537d685f3a3', 'request_id': 'req-364f9b51-243b-4dcd-951c-2bfe8b44a09b'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.563483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] The token cache is being invalidate because project e3015a88384447ce8ee07537d685f3a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:49:13.563702 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.564361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.565970 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4447/8934] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:49:13 2022] DELETE /identity/v3/projects/e3015a88384447ce8ee07537d685f3a3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:49:13.570405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:13.572040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:13.586018 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62f0d5-d8cf-459e-bd25-451f1d2b3395 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:13.647474 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:13.648121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:13.648403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:13.648677 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] PATH_INFO: `/v3/users/75e0fcc96f3943a4bec356ed88e30bc0` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:13.649950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e0fcc96f3943a4bec356ed88e30bc0)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:13.651949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:13.705212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e0fcc96f3943a4bec356ed88e30bc0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.705948 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e0fcc96f3943a4bec356ed88e30bc0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.714945 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e0fcc96f3943a4bec356ed88e30bc0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.715770 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e0fcc96f3943a4bec356ed88e30bc0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.728169 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-a457f90c-d933-4f9c-8363-2fd1bd2fb531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e0fcc96f3943a4bec356ed88e30bc0'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:13.740437 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4488/8935] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:49:13 2022] DELETE /identity/v3/users/75e0fcc96f3943a4bec356ed88e30bc0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:49:13.927132 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:13.928771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:13.942394 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f9b51-243b-4dcd-951c-2bfe8b44a09b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.004426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.005089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.005375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.005680 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] PATH_INFO: `/v3/projects/cdfc8554448b4472b2fc0da4dc4dbf1c` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.006814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdfc8554448b4472b2fc0da4dc4dbf1c)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.008921 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.059774 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdfc8554448b4472b2fc0da4dc4dbf1c', 'request_id': 'req-c296fce7-4fb7-4f85-a871-daab6232ad0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:14.068439 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdfc8554448b4472b2fc0da4dc4dbf1c', 'request_id': 'req-c296fce7-4fb7-4f85-a871-daab6232ad0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:14.074605 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdfc8554448b4472b2fc0da4dc4dbf1c', 'request_id': 'req-c296fce7-4fb7-4f85-a871-daab6232ad0d'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:14.086769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] The token cache is being invalidate because project cdfc8554448b4472b2fc0da4dc4dbf1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:49:14.087055 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:14.087764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-c296fce7-4fb7-4f85-a871-daab6232ad0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:49:14.089520 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4448/8936] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:49:13 2022] DELETE /identity/v3/projects/cdfc8554448b4472b2fc0da4dc4dbf1c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:49:14.094926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945cc66-ae86-4948-92fd-a21f49304372 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.095411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945cc66-ae86-4948-92fd-a21f49304372 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.095535 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945cc66-ae86-4948-92fd-a21f49304372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.146720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f945cc66-ae86-4948-92fd-a21f49304372 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:14.147941 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f945cc66-ae86-4948-92fd-a21f49304372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:14.161317 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:14.211593 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4489/8937] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:14.220209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a06621-174e-4976-9e19-4918b483e00a None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.220495 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a06621-174e-4976-9e19-4918b483e00a None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.220766 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a06621-174e-4976-9e19-4918b483e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.242081 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-f8a06621-174e-4976-9e19-4918b483e00a None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:14.243279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-f8a06621-174e-4976-9e19-4918b483e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:14.252194 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4449/8938] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:14.256949 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945cc66-ae86-4948-92fd-a21f49304372 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.268420 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945cc66-ae86-4948-92fd-a21f49304372 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.274016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.274620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.274921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.275214 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.275918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:49:14.276218 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.277858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.289898 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4490/8939] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:49:14.294211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a06621-174e-4976-9e19-4918b483e00a None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.303560 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a06621-174e-4976-9e19-4918b483e00a None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.308726 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.309310 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.309629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.309894 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.310514 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.312455 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.352904 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4450/8940] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:49:14 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:49:14.359179 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.373084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c75122-26ea-4ef4-96cc-7117a881f97b admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.430590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.431103 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.431377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.431651 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.432324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.434307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.456786 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4491/8941] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:49:14 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:49:14.461577 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.472315 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459035bd-d178-4445-8ebb-42c71f4e2dda admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.478066 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.478673 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.479030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.479303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.479957 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.481616 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.492154 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4451/8942] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:49:14.496823 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.510033 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6058c-0989-4dc9-97ea-21f923676cc0 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.515309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f8308-d45f-422a-8d50-41ec5e427723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.515925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f8308-d45f-422a-8d50-41ec5e427723 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.516204 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f8308-d45f-422a-8d50-41ec5e427723 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.516489 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f8308-d45f-422a-8d50-41ec5e427723 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.517144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f8308-d45f-422a-8d50-41ec5e427723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.518836 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f8308-d45f-422a-8d50-41ec5e427723 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.531427 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4492/8943] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:49:14.535856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.546820 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8744987-2c17-4bed-850a-bdf023fd4907 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.552845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.553676 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.553878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.554134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] PATH_INFO: `/v3/projects/88a7205c02914d428fd0d43b7fa61fe9/users/ac0fa1f4eeca4d5e822b603953471c73/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.570001 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a7205c02914d428fd0d43b7fa61fe9, user_id=ac0fa1f4eeca4d5e822b603953471c73, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.574183 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.576589 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:14.591645 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4452/8944] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:49:14 2022] PUT /identity/v3/projects/88a7205c02914d428fd0d43b7fa61fe9/users/ac0fa1f4eeca4d5e822b603953471c73/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:49:14.602008 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d44ea-b64e-4648-8e77-927023a36178 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.602365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d44ea-b64e-4648-8e77-927023a36178 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.602704 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d44ea-b64e-4648-8e77-927023a36178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.633853 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-339d44ea-b64e-4648-8e77-927023a36178 None None] MFA Rules not processed for user `ac0fa1f4eeca4d5e822b603953471c73`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:14.635265 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-339d44ea-b64e-4648-8e77-927023a36178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:14.712561 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4493/8945] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:49:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:14.717428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.785986 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4115-c6d1-4724-b2c7-1fac3cd357f9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.791428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.792049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.792356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.792620 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.793246 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.795243 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.823927 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4453/8946] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:49:14 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:49:14.828393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d44ea-b64e-4648-8e77-927023a36178 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.841322 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d44ea-b64e-4648-8e77-927023a36178 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.851917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:14.902365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.902979 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.903321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.903583 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.905049 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.907104 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.934234 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4494/8947] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:49:14 2022] POST /identity/v3/users => generated 301 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:49:14.938576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.949855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e4960-c505-458b-bf66-cad53b4784e9 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.955527 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.956140 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.956448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.956715 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:14.957372 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:14.959583 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:14.971760 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4454/8948] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:49:14.978610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:14.989664 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d566a1-189a-4056-b922-4ccee290c3a5 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:14.994620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:14.995298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:14.995580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:14.995855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] PATH_INFO: `/v3/projects/69335ed5484f4b1a857951354b13ec9f/users/fb0ad9110d074c4b90c354297eaa48d5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:15.011213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69335ed5484f4b1a857951354b13ec9f, user_id=fb0ad9110d074c4b90c354297eaa48d5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:15.013890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:15.028689 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4495/8949] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:49:14 2022] PUT /identity/v3/projects/69335ed5484f4b1a857951354b13ec9f/users/fb0ad9110d074c4b90c354297eaa48d5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:49:15.033052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:15.043641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de27a9-aac0-459b-99c3-a21d20de7357 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:15.114491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2829a-f86a-45e5-8421-ce7512b8574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:15.115240 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2829a-f86a-45e5-8421-ce7512b8574a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:15.115367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2829a-f86a-45e5-8421-ce7512b8574a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:15.115623 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2829a-f86a-45e5-8421-ce7512b8574a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:15.116259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2829a-f86a-45e5-8421-ce7512b8574a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:15.117919 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2829a-f86a-45e5-8421-ce7512b8574a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:15.128089 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4455/8950] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:49:15.134081 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:15.145561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713c863-f7e7-465a-abaa-c352a024a945 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:15.152072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:15.152740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:15.153016 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:15.153276 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] PATH_INFO: `/v3/projects/69335ed5484f4b1a857951354b13ec9f/users/fb0ad9110d074c4b90c354297eaa48d5/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:15.155511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69335ed5484f4b1a857951354b13ec9f, user_id=fb0ad9110d074c4b90c354297eaa48d5, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:15.158281 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:15.170076 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4496/8951] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:49:15 2022] PUT /identity/v3/projects/69335ed5484f4b1a857951354b13ec9f/users/fb0ad9110d074c4b90c354297eaa48d5/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:49:15.181736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:15.182202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:15.182480 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:15.211747 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] MFA Rules not processed for user `fb0ad9110d074c4b90c354297eaa48d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:15.213937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:15.267149 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:15.288705 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4456/8952] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:49:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:15.304684 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:15.307325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:15.382978 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2016ab-c260-4ede-9933-3fe74acb5a85 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:15.388749 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d13ce1-623a-41f2-9fe9-39c875a63910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:15.389360 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d13ce1-623a-41f2-9fe9-39c875a63910 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:15.389665 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d13ce1-623a-41f2-9fe9-39c875a63910 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:15.389935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d13ce1-623a-41f2-9fe9-39c875a63910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:15.448660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d13ce1-623a-41f2-9fe9-39c875a63910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:15.450348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d13ce1-623a-41f2-9fe9-39c875a63910 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:15.460370 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4497/8953] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:18.272652 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:18.274344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:18.334093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca345cd-4ead-4dbb-973b-53ffadf3cd76 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:18.337681 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:18.338287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:18.338562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:18.338861 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:18.347835 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:18.349303 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:18.360118 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4457/8954] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:18 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:21.334942 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:21.335520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:21.335520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:21.344569 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:21.359890 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] MFA Rules not processed for user `ac0fa1f4eeca4d5e822b603953471c73`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:21.361129 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:21.375137 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4498/8955] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:49:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:21.739435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:21.748386 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb3c2a-4c0b-46e9-a5d4-e53b0a5d44c1 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:21.752061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f9a69-7bc8-4c2a-ad22-34eb28a05557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:21.752656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f9a69-7bc8-4c2a-ad22-34eb28a05557 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:21.752934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f9a69-7bc8-4c2a-ad22-34eb28a05557 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:21.753207 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f9a69-7bc8-4c2a-ad22-34eb28a05557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:21.755006 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-712f9a69-7bc8-4c2a-ad22-34eb28a05557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:21.767838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f9a69-7bc8-4c2a-ad22-34eb28a05557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:21.769214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f9a69-7bc8-4c2a-ad22-34eb28a05557 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:21.780233 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4458/8956] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:22.125221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:22.126863 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:22.215413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e4c54-2706-4164-acf5-d0e7f8be5646 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:22.222388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:22.223012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:22.223290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:22.223559 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:22.225321 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:22.237754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:22.239476 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:22.250154 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4499/8957] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:30.283858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3407c8-a352-42e7-a699-6fef0c9a280f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:30.284448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3407c8-a352-42e7-a699-6fef0c9a280f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:30.284448 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3407c8-a352-42e7-a699-6fef0c9a280f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:30.305779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2f3407c8-a352-42e7-a699-6fef0c9a280f None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:49:30.307003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f3407c8-a352-42e7-a699-6fef0c9a280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:30.316768 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4459/8958] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:49:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:49:30.342088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:49:30.343816 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:49:30.359571 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9088b0-228b-411a-ac28-d48e072f812f service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:49:30.365950 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166101e-d280-406a-a308-02b9a5264062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:49:30.366562 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166101e-d280-406a-a308-02b9a5264062 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:49:30.366837 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166101e-d280-406a-a308-02b9a5264062 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:49:30.367143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166101e-d280-406a-a308-02b9a5264062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:49:30.377153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166101e-d280-406a-a308-02b9a5264062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:49:30.378801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166101e-d280-406a-a308-02b9a5264062 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:49:30.389298 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4500/8959] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:49:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:49:40.299187 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=40 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:40.385122 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=40 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:50:16.205933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a7954-b40b-403d-ab36-d8d5e938f7ff None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:50:16.206518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a7954-b40b-403d-ab36-d8d5e938f7ff None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:50:16.206518 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a7954-b40b-403d-ab36-d8d5e938f7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:50:16.227780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2f7a7954-b40b-403d-ab36-d8d5e938f7ff None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:50:16.229017 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2f7a7954-b40b-403d-ab36-d8d5e938f7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:50:16.239167 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4460/8960] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:50:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:50:16.268089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84280e9f-974e-4ffc-987d-feff2a797665 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:50:16.268741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84280e9f-974e-4ffc-987d-feff2a797665 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:50:16.268741 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84280e9f-974e-4ffc-987d-feff2a797665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:50:16.295563 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-84280e9f-974e-4ffc-987d-feff2a797665 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:50:16.296813 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-84280e9f-974e-4ffc-987d-feff2a797665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:50:16.308058 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4501/8961] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Sep 17 21:50:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:50:16.314483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a7954-b40b-403d-ab36-d8d5e938f7ff None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:50:16.325355 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a7954-b40b-403d-ab36-d8d5e938f7ff None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:50:16.330913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f26c25-1d7f-4927-b2b4-336c108b146c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:50:16.331545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f26c25-1d7f-4927-b2b4-336c108b146c service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:50:16.331802 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f26c25-1d7f-4927-b2b4-336c108b146c service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:50:16.332076 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f26c25-1d7f-4927-b2b4-336c108b146c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:50:16.341501 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f26c25-1d7f-4927-b2b4-336c108b146c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:50:16.343089 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f26c25-1d7f-4927-b2b4-336c108b146c service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:50:16.353225 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4461/8962] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:50:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:50:21.902315 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801dd78-40ec-4586-b87b-ffe6c8dda8f7 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:50:21.902884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801dd78-40ec-4586-b87b-ffe6c8dda8f7 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:50:21.902884 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801dd78-40ec-4586-b87b-ffe6c8dda8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:50:21.928175 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6801dd78-40ec-4586-b87b-ffe6c8dda8f7 None None] MFA Rules not processed for user `ac0fa1f4eeca4d5e822b603953471c73`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:50:21.929921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6801dd78-40ec-4586-b87b-ffe6c8dda8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:50:21.937938 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4502/8963] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:50:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:50:26.349088 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=9 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:50:31.918992 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:50:38.833483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dfb19-dd92-4bdb-8a2c-82f5b561067b None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:50:38.834072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dfb19-dd92-4bdb-8a2c-82f5b561067b None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:50:38.834072 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dfb19-dd92-4bdb-8a2c-82f5b561067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:50:38.857736 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d69dfb19-dd92-4bdb-8a2c-82f5b561067b None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:50:38.858984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d69dfb19-dd92-4bdb-8a2c-82f5b561067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:50:38.876314 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4462/8964] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:50:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:50:38.889636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801dd78-40ec-4586-b87b-ffe6c8dda8f7 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:50:38.909994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801dd78-40ec-4586-b87b-ffe6c8dda8f7 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:50:38.916059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:50:38.916832 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:50:38.917118 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:50:38.917392 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:50:38.927429 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:50:38.929539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:50:38.940580 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4503/8965] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:50:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:50:48.867698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:50:48.936585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:51:12.979612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69dfb19-dd92-4bdb-8a2c-82f5b561067b None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:51:12.991432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69dfb19-dd92-4bdb-8a2c-82f5b561067b None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:51:12.997429 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b9345-3c86-4749-8523-6016d5c539f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:51:12.998061 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b9345-3c86-4749-8523-6016d5c539f2 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:51:12.998956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b9345-3c86-4749-8523-6016d5c539f2 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:51:12.999234 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b9345-3c86-4749-8523-6016d5c539f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:51:13.001389 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-889b9345-3c86-4749-8523-6016d5c539f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:51:13.013754 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b9345-3c86-4749-8523-6016d5c539f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:51:13.015376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b9345-3c86-4749-8523-6016d5c539f2 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:51:13.025425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4463/8966] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:51:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:51:15.215920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:51:15.227933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caabf8-f4a4-4cdb-8280-e06fefb7a60c service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:51:15.234173 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccd079-2bb8-4d07-b21d-16179095d916 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:51:15.234784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd079-2bb8-4d07-b21d-16179095d916 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:51:15.235059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd079-2bb8-4d07-b21d-16179095d916 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:51:15.235327 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccd079-2bb8-4d07-b21d-16179095d916 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:51:15.237257 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-3dccd079-2bb8-4d07-b21d-16179095d916 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:51:15.251831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccd079-2bb8-4d07-b21d-16179095d916 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:51:15.253428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccd079-2bb8-4d07-b21d-16179095d916 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:51:15.265193 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4504/8967] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:51:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:51:23.021388 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:51:25.260843 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:52:13.039907 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7e6b5-9d03-41b0-9b30-f48990c4f2d8 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:13.040842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7e6b5-9d03-41b0-9b30-f48990c4f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:13.041320 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7e6b5-9d03-41b0-9b30-f48990c4f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:13.063041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-3dc7e6b5-9d03-41b0-9b30-f48990c4f2d8 None None] MFA Rules not processed for user `ac0fa1f4eeca4d5e822b603953471c73`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:52:13.064433 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3dc7e6b5-9d03-41b0-9b30-f48990c4f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:52:13.083164 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4464/8968] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:52:13.430439 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a56f-bbb2-4581-9030-f06843cbe564 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:13.431029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a56f-bbb2-4581-9030-f06843cbe564 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:13.431029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a56f-bbb2-4581-9030-f06843cbe564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:13.470364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c142a56f-bbb2-4581-9030-f06843cbe564 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:52:13.471485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c142a56f-bbb2-4581-9030-f06843cbe564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:52:13.488170 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4505/8969] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:52:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:52:13.500233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7e6b5-9d03-41b0-9b30-f48990c4f2d8 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:52:13.510502 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7e6b5-9d03-41b0-9b30-f48990c4f2d8 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:52:13.514318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:52:13.514926 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:13.515202 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:13.515466 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:13.522918 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:52:13.524444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:52:13.532470 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4465/8970] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:52:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:52:18.748680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20440-17a2-4d8e-b6f7-8b6eba5ffed9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:18.749232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20440-17a2-4d8e-b6f7-8b6eba5ffed9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:18.749232 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd20440-17a2-4d8e-b6f7-8b6eba5ffed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:18.770221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-6bd20440-17a2-4d8e-b6f7-8b6eba5ffed9 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:52:18.771645 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-6bd20440-17a2-4d8e-b6f7-8b6eba5ffed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:52:18.778454 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4506/8971] 10.222.0.60 () {58 vars in 1015 bytes} [Sat Sep 17 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:52:18.790625 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:52:18.799771 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d75f91-d02e-4316-ba6c-804f9a432d5b service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:52:18.803901 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:52:18.804525 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:18.804847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:18.805121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:18.813200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:52:18.814933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:52:18.823461 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4466/8972] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:52:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:52:28.764993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=16 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:52:28.819782 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=15 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:52:59.302917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366466b2-ebd8-4ace-b028-51b9ac85bc72 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:59.303545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366466b2-ebd8-4ace-b028-51b9ac85bc72 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:59.303545 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366466b2-ebd8-4ace-b028-51b9ac85bc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:59.324611 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-366466b2-ebd8-4ace-b028-51b9ac85bc72 None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:52:59.325847 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-366466b2-ebd8-4ace-b028-51b9ac85bc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:52:59.334047 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4507/8973] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:52:59.346200 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:52:59.355424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1872c31f-f204-4c11-aca6-358b3a550b88 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:52:59.359982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:52:59.360576 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:59.360862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:59.361146 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:59.368913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:52:59.370378 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:52:59.378500 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4467/8974] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:52:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:52:59.531567 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72904e66-490b-4172-9597-f862c2083c6e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:59.531849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72904e66-490b-4172-9597-f862c2083c6e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:59.532105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72904e66-490b-4172-9597-f862c2083c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:59.552672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-72904e66-490b-4172-9597-f862c2083c6e None None] MFA Rules not processed for user `a824d5df8638409bbaf4d6e8d22d63ca`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:52:59.553933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-72904e66-490b-4172-9597-f862c2083c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:52:59.561322 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4508/8975] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:52:59.578575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:52:59.587809 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c8e43-857d-4fd2-ab68-a085095d4a88 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:52:59.591510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf640d-eb7e-4245-910b-bd594e74f10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:52:59.592111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf640d-eb7e-4245-910b-bd594e74f10b service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:59.592377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf640d-eb7e-4245-910b-bd594e74f10b service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:59.592646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf640d-eb7e-4245-910b-bd594e74f10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:59.599976 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf640d-eb7e-4245-910b-bd594e74f10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:52:59.601838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf640d-eb7e-4245-910b-bd594e74f10b service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:52:59.609527 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4468/8976] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:52:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:52:59.720550 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72904e66-490b-4172-9597-f862c2083c6e None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:52:59.750105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72904e66-490b-4172-9597-f862c2083c6e None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:52:59.756466 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:52:59.757136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:59.757482 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:59.757876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:52:59.760051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:52:59.772968 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:52:59.774598 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:52:59.785065 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4509/8977] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:52:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:52:59.966273 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ef40e-404f-4fd7-be24-7c431d6f5c63 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:52:59.966561 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ef40e-404f-4fd7-be24-7c431d6f5c63 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:52:59.966858 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ef40e-404f-4fd7-be24-7c431d6f5c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:53:00.001290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-528ef40e-404f-4fd7-be24-7c431d6f5c63 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:53:00.002469 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-528ef40e-404f-4fd7-be24-7c431d6f5c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:53:00.023466 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4469/8978] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:53:00.034271 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:53:00.036027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:53:00.048888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c6fd9-15d1-40b2-8a21-90bdc21322ab service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:53:00.053347 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ba0a5-d92c-4e45-be78-56702c395fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:53:00.054012 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba0a5-d92c-4e45-be78-56702c395fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:53:00.054286 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba0a5-d92c-4e45-be78-56702c395fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:53:00.054555 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba0a5-d92c-4e45-be78-56702c395fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:53:00.065692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ba0a5-d92c-4e45-be78-56702c395fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:53:00.067172 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ba0a5-d92c-4e45-be78-56702c395fc3 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:53:00.078555 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4510/8979] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:53:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:53:00.141838 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ef40e-404f-4fd7-be24-7c431d6f5c63 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:53:00.155647 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ef40e-404f-4fd7-be24-7c431d6f5c63 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:53:00.162295 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:53:00.162927 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:53:00.163181 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:53:00.163422 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:53:00.165595 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:53:00.178904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:53:00.180499 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:53:00.190496 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4470/8980] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:53:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:53:10.073434 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=22 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:53:10.186279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=30 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:53:55.532075 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3829ff-8b29-4e4d-9ae0-a653f50401e9 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:53:55.532633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3829ff-8b29-4e4d-9ae0-a653f50401e9 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:53:55.532743 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3829ff-8b29-4e4d-9ae0-a653f50401e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:53:55.552632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-9a3829ff-8b29-4e4d-9ae0-a653f50401e9 None None] MFA Rules not processed for user `064befaa1bd34b89bafad00c81ede856`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:53:55.553925 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9a3829ff-8b29-4e4d-9ae0-a653f50401e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:53:55.562630 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4511/8981] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:53:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:53:55.908988 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:53:55.932497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a3a8-f8e3-4537-b404-3b8181751258 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:53:55.937897 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:53:55.940543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:53:55.941041 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:53:55.941568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:53:55.952169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:53:55.953832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:53:55.964180 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4471/8982] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:53:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:54:01.896068 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04275-f1d7-439a-9d52-35c5835ea836 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:54:01.897027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04275-f1d7-439a-9d52-35c5835ea836 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:54:01.897282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04275-f1d7-439a-9d52-35c5835ea836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:54:01.919191 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-f9f04275-f1d7-439a-9d52-35c5835ea836 None None] MFA Rules not processed for user `ac0fa1f4eeca4d5e822b603953471c73`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:54:01.921066 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f9f04275-f1d7-439a-9d52-35c5835ea836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:54:01.928175 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4512/8983] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:54:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:54:01.971016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:54:01.972606 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:54:02.093203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97ee9c-23f6-4abe-86c6-9f67ceff5626 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:54:02.097630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:54:02.098265 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:54:02.098543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:54:02.098815 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:54:02.100685 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:54:02.123287 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:54:02.124783 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:54:02.136503 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4472/8984] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:54:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:54:03.190500 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04275-f1d7-439a-9d52-35c5835ea836 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:54:03.201795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04275-f1d7-439a-9d52-35c5835ea836 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:54:03.208156 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:54:03.209039 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:54:03.209491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:54:03.210001 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:54:03.211999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:54:03.227663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:54:03.229277 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:54:03.241231 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4513/8985] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:54:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:54:12.131670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=41 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:54:13.236197 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=12 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:54:15.938363 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:54:15.942092 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:54:15.951848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e00a7-0816-43a9-9485-8b3bdab9f77b service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:54:15.955222 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f88970-1c97-4d5d-b727-772877d37ca1 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:54:15.958640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:54:15.959259 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:54:15.960038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:54:15.960038 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:54:15.961338 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4135ba-f379-4735-9d75-e08ec15c18b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:54:15.962107 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4135ba-f379-4735-9d75-e08ec15c18b4 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:54:15.962387 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4135ba-f379-4735-9d75-e08ec15c18b4 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:54:15.962643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4135ba-f379-4735-9d75-e08ec15c18b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:54:15.964682 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ee4135ba-f379-4735-9d75-e08ec15c18b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:54:15.968796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:54:15.970444 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:54:15.978656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4135ba-f379-4735-9d75-e08ec15c18b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:54:15.980185 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4473/8986] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:54:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:54:15.980364 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4135ba-f379-4735-9d75-e08ec15c18b4 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:54:15.991623 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4514/8987] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:54:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:54:25.975853 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:54:25.987221 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:55:42.765697 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:55:42.776049 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f55bf-f2e0-4456-b72e-863abe242daf service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:55:42.779787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:55:42.780413 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:55:42.780704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:55:42.780920 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:55:42.790842 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:55:42.792435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:55:42.802542 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4474/8988] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:55:42 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:55:45.533150 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263fc1-718e-4171-8fdb-84cd4019a7bc None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:55:45.533792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263fc1-718e-4171-8fdb-84cd4019a7bc None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:55:45.533792 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263fc1-718e-4171-8fdb-84cd4019a7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:55:45.553491 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-28263fc1-718e-4171-8fdb-84cd4019a7bc None None] MFA Rules not processed for user `ac0fa1f4eeca4d5e822b603953471c73`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:55:45.554636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-28263fc1-718e-4171-8fdb-84cd4019a7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:55:45.561354 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4515/8989] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:55:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:55:48.191719 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:55:48.200892 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b835-dd80-47a3-b92d-0d18548eb6c4 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:55:48.205406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:55:48.206425 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:55:48.206915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:55:48.207345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:55:48.209551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:55:48.221735 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:55:48.223798 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:55:48.234633 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4475/8990] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:55:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:55:53.895225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28263fc1-718e-4171-8fdb-84cd4019a7bc None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:55:53.909703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28263fc1-718e-4171-8fdb-84cd4019a7bc None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:55:53.915918 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:55:53.916539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:55:53.916822 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:55:53.917091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:55:53.927182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:55:53.928793 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:55:53.939286 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4516/8991] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:55:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:55:58.230093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=12 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:03.935194 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=9 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:19.009607 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:19.060756 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78e0cc-bad8-4245-9967-9b93dbde6135 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:19.067065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:19.067645 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:19.067946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:19.068214 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:19.077060 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:19.079111 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:19.089078 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4476/8992] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:56:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:56:20.192202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:20.236324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa778c-3383-4b32-80b2-aa443110f164 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:20.242192 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:20.242785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:20.243054 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:20.243323 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:20.253065 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:20.254686 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:20.265505 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4517/8993] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:56:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:56:29.084906 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=20 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:30.260662 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=20 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:32.728524 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:32.743360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ef0a8-214a-40d7-9f8e-266d54cb756d service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:32.751579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0350a982-cf88-43db-8be4-9b1f66ebaff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:32.752342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350a982-cf88-43db-8be4-9b1f66ebaff9 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:32.752792 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350a982-cf88-43db-8be4-9b1f66ebaff9 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:32.753046 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0350a982-cf88-43db-8be4-9b1f66ebaff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:32.763571 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350a982-cf88-43db-8be4-9b1f66ebaff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:32.765290 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0350a982-cf88-43db-8be4-9b1f66ebaff9 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:32.776958 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4477/8994] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:56:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:56:39.832403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:39.899263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8e83-da54-45aa-b608-2e2fbd09320c service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:39.904086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:39.904720 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:39.904995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:39.905262 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] PATH_INFO: `/v3/users/ac0fa1f4eeca4d5e822b603953471c73` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:39.906471 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0fa1f4eeca4d5e822b603953471c73)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:39.908413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:39.959331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0fa1f4eeca4d5e822b603953471c73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:39.960325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0fa1f4eeca4d5e822b603953471c73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:39.969718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0fa1f4eeca4d5e822b603953471c73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:39.970371 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0fa1f4eeca4d5e822b603953471c73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:39.983044 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0fa1f4eeca4d5e822b603953471c73'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:39.995278 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4518/8995] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:56:39 2022] DELETE /identity/v3/users/ac0fa1f4eeca4d5e822b603953471c73 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:56:40.000204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:40.000753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:40.000753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:40.031551 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:56:40.032773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:40.096093 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4478/8996] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:56:40.106413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:40.108074 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:40.178402 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7871a0-24d6-449b-bb7d-09ad4472e4ee admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:40.182385 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:40.182990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:40.183269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:40.183532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:40.193022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:40.194520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:40.205030 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4519/8997] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:56:40.380408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:40.381996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:40.395753 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7b40c-e2bb-40fc-92c9-4ef904ff82eb None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:40.401160 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:40.401910 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:40.402189 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:40.402459 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] PATH_INFO: `/v3/projects/88a7205c02914d428fd0d43b7fa61fe9` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:40.403742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a7205c02914d428fd0d43b7fa61fe9)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:40.405929 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:40.462832 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a7205c02914d428fd0d43b7fa61fe9', 'request_id': 'req-3e36a691-f414-4867-af67-455f44dd7bcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.472317 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a7205c02914d428fd0d43b7fa61fe9', 'request_id': 'req-3e36a691-f414-4867-af67-455f44dd7bcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.479292 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a7205c02914d428fd0d43b7fa61fe9', 'request_id': 'req-3e36a691-f414-4867-af67-455f44dd7bcb'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.491704 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] The token cache is being invalidate because project 88a7205c02914d428fd0d43b7fa61fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:56:40.491995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.492668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.494388 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4479/8998] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:56:40 2022] DELETE /identity/v3/projects/88a7205c02914d428fd0d43b7fa61fe9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:56:40.499696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:40.510762 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34514ab-759a-4f39-9f25-3f161af2ae93 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:40.521058 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:40.578027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:40.578740 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:40.579025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:40.579302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] PATH_INFO: `/v3/users/fb0ad9110d074c4b90c354297eaa48d5` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:40.580510 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0ad9110d074c4b90c354297eaa48d5)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:40.582538 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:40.631485 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0ad9110d074c4b90c354297eaa48d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.632063 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0ad9110d074c4b90c354297eaa48d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.641003 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0ad9110d074c4b90c354297eaa48d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.641643 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0ad9110d074c4b90c354297eaa48d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.653902 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-82ebdf3c-4195-415d-9d0f-635c309a0dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0ad9110d074c4b90c354297eaa48d5'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:40.667200 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4520/8999] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 21:56:40 2022] DELETE /identity/v3/users/fb0ad9110d074c4b90c354297eaa48d5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:56:40.962325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:40.964168 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:40.980483 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36a691-f414-4867-af67-455f44dd7bcb admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.042073 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.042749 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.042966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.043238 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] PATH_INFO: `/v3/projects/69335ed5484f4b1a857951354b13ec9f` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.044431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=69335ed5484f4b1a857951354b13ec9f)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.046403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.048570 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:41.098692 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69335ed5484f4b1a857951354b13ec9f', 'request_id': 'req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:41.107779 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69335ed5484f4b1a857951354b13ec9f', 'request_id': 'req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:41.113377 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69335ed5484f4b1a857951354b13ec9f', 'request_id': 'req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:41.124288 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] The token cache is being invalidate because project 69335ed5484f4b1a857951354b13ec9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:56:41.124578 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:41.125211 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-d97af6e6-25a4-4b9d-9197-0cc388b3cc5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:56:41.126730 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4480/9000] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 21:56:40 2022] DELETE /identity/v3/projects/69335ed5484f4b1a857951354b13ec9f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:56:41.132608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321754-982d-4ff9-9a39-5052105319a2 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.133155 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321754-982d-4ff9-9a39-5052105319a2 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.133263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321754-982d-4ff9-9a39-5052105319a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.153802 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-b0321754-982d-4ff9-9a39-5052105319a2 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:56:41.154920 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b0321754-982d-4ff9-9a39-5052105319a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:41.217651 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4521/9001] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:56:41.227777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6841f5-b505-46e5-a3eb-7c294be8d614 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.228167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6841f5-b505-46e5-a3eb-7c294be8d614 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.228545 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6841f5-b505-46e5-a3eb-7c294be8d614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.253777 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-bc6841f5-b505-46e5-a3eb-7c294be8d614 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:56:41.254982 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-bc6841f5-b505-46e5-a3eb-7c294be8d614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:41.263815 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4481/9002] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:56:41.268145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0321754-982d-4ff9-9a39-5052105319a2 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:41.278623 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0321754-982d-4ff9-9a39-5052105319a2 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.284253 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.284855 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.285125 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.285390 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.286072 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:56:41.286363 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.288013 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.300328 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4522/9003] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 21:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:56:41.304845 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6841f5-b505-46e5-a3eb-7c294be8d614 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:41.314670 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6841f5-b505-46e5-a3eb-7c294be8d614 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.320267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.320886 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.321283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.321588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.322215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.324185 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.355006 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4482/9004] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:56:41 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:56:41.359696 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:41.369993 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b91b-e78c-4ec0-b623-5e747aade5fd admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.387128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:41.429341 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.429935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.430208 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.430483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.431531 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.433542 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.461154 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4523/9005] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:56:41 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:56:41.462479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:41.473227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b52190-8225-4339-b57f-5ce3b5cb63b6 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.477884 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.478473 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.478746 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.479082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.479717 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.481411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.491494 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4483/9006] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:56:41.495625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:41.506396 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295b320-949b-4ae9-8f69-a48be597bc65 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.511236 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10af5b-58f0-4e75-ab6b-54b4ff97ac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.511866 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10af5b-58f0-4e75-ab6b-54b4ff97ac61 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.512250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10af5b-58f0-4e75-ab6b-54b4ff97ac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.512524 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10af5b-58f0-4e75-ab6b-54b4ff97ac61 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.513162 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10af5b-58f0-4e75-ab6b-54b4ff97ac61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.514803 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10af5b-58f0-4e75-ab6b-54b4ff97ac61 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.525879 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4524/9007] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:56:41.530249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:41.540999 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14067e59-aca5-4a63-9437-4d81df40f90a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.546044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.546669 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.546941 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.547206 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] PATH_INFO: `/v3/projects/015ec23d11f64d56844d28795ccba61a/users/3d3bf2dacad34c0aa465968f1c273cf9/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.561821 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=015ec23d11f64d56844d28795ccba61a, user_id=3d3bf2dacad34c0aa465968f1c273cf9, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.564510 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.582178 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4484/9008] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:56:41 2022] PUT /identity/v3/projects/015ec23d11f64d56844d28795ccba61a/users/3d3bf2dacad34c0aa465968f1c273cf9/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:56:41.589438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a95825-5bbd-46eb-b0c5-530bff7e1e0c None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.589750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a95825-5bbd-46eb-b0c5-530bff7e1e0c None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.590022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a95825-5bbd-46eb-b0c5-530bff7e1e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.609324 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-87a95825-5bbd-46eb-b0c5-530bff7e1e0c None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:56:41.610594 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-87a95825-5bbd-46eb-b0c5-530bff7e1e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:41.670649 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4525/9009] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:56:41.681338 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:41.683334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:41.743722 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb920e-799d-4dcd-81c0-6afcc2bd6de5 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:41.748093 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:41.748707 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:41.748994 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:41.749267 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:41.758928 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:41.760411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:41.770082 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4485/9010] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:56:45.861625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a66c0d-dee9-44ae-ab70-a7043a22bd10 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:45.862148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a66c0d-dee9-44ae-ab70-a7043a22bd10 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:45.862148 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a66c0d-dee9-44ae-ab70-a7043a22bd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:45.897851 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-d5a66c0d-dee9-44ae-ab70-a7043a22bd10 None None] MFA Rules not processed for user `3d3bf2dacad34c0aa465968f1c273cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:56:45.899157 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d5a66c0d-dee9-44ae-ab70-a7043a22bd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:45.970192 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4526/9011] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:56:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:56:45.975065 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:45.985325 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f8dd0-c3bd-4c43-96ee-6d96b41e603d service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:45.991534 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:45.992157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:45.992432 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:45.992698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:45.993401 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:45.995446 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:46.031078 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4486/9012] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 21:56:45 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:56:46.032280 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a66c0d-dee9-44ae-ab70-a7043a22bd10 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:46.044047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a66c0d-dee9-44ae-ab70-a7043a22bd10 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:46.072199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:46.106733 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:46.107359 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:46.107660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:46.107938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:46.108590 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:46.110628 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:46.150059 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4527/9013] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 21:56:46 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:56:46.154991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:46.179279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc1ce1-64d9-44b5-9564-9523199bca4a admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:46.183936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:46.184539 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:46.184814 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:46.185080 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:46.185758 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:46.187375 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:46.189662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:46.202958 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4487/9014] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:56:46.204128 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:46.218621 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50cfff-391a-474d-a1cb-088ddd0bd30d admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:46.223544 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:46.224216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:46.224493 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:46.224764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] PATH_INFO: `/v3/projects/f99ff4f6290b45f882b66d1f7407010e/users/92eaa391d0484fd8853f341baae9702f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:46.241255 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99ff4f6290b45f882b66d1f7407010e, user_id=92eaa391d0484fd8853f341baae9702f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:46.244047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:46.259343 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4528/9015] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:56:46 2022] PUT /identity/v3/projects/f99ff4f6290b45f882b66d1f7407010e/users/92eaa391d0484fd8853f341baae9702f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:56:46.263419 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:46.274491 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd46f77-1df2-4280-83d9-05f8eaa95d71 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:46.345693 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1f05b-fd78-4c23-a343-5671e361b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:46.346420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1f05b-fd78-4c23-a343-5671e361b87b admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:46.346698 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1f05b-fd78-4c23-a343-5671e361b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:46.346966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1f05b-fd78-4c23-a343-5671e361b87b admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:46.347619 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1f05b-fd78-4c23-a343-5671e361b87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:46.349311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1f05b-fd78-4c23-a343-5671e361b87b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:46.360610 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4488/9016] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 21:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:56:46.364852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:46.376203 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54157-baf5-468a-866f-49b55a2ff373 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:46.381139 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:46.382010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:46.382407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:46.382772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] PATH_INFO: `/v3/projects/f99ff4f6290b45f882b66d1f7407010e/users/92eaa391d0484fd8853f341baae9702f/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:46.384997 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99ff4f6290b45f882b66d1f7407010e, user_id=92eaa391d0484fd8853f341baae9702f, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:46.387824 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:46.399148 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4529/9017] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 21:56:46 2022] PUT /identity/v3/projects/f99ff4f6290b45f882b66d1f7407010e/users/92eaa391d0484fd8853f341baae9702f/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:56:51.019888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1d35a-b9dd-435e-ac1b-4719c7d89616 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:51.020415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1d35a-b9dd-435e-ac1b-4719c7d89616 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:51.020415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1d35a-b9dd-435e-ac1b-4719c7d89616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:51.055671 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-2fe1d35a-b9dd-435e-ac1b-4719c7d89616 None None] MFA Rules not processed for user `92eaa391d0484fd8853f341baae9702f`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:56:51.056996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2fe1d35a-b9dd-435e-ac1b-4719c7d89616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:51.136519 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4489/9018] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:56:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:56:51.154630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:51.157325 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:51.264296 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd3f95-f0d0-4fc1-98fe-c4266c92a29a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:51.269983 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:51.270680 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:51.271017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:51.271348 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:51.331536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:51.333289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:51.343271 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4530/9019] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:56:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:56:52.368479 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:52.369018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:52.369018 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:52.389663 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] MFA Rules not processed for user `3d3bf2dacad34c0aa465968f1c273cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:56:52.390878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:52.398591 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4490/9020] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 21:56:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:56:54.092435 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:56:54.093999 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:54.117561 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:54.171846 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae968f-2c3a-426e-8036-f2cf6e4a6407 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:56:54.178307 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:56:54.178970 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:56:54.179314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:56:54.179586 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:56:54.181422 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:56:54.194876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:56:54.196584 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:56:54.207861 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4531/9021] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:56:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:57:02.384047 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=67 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:57:04.203202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=24 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:57:05.998128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:57:05.999764 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:57:06.072575 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d530-1720-42a0-bca5-71a142042348 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:57:06.077431 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:57:06.078067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:57:06.078342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:57:06.078579 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:57:06.088257 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:57:06.089772 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:57:06.100075 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4491/9022] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:57:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:57:16.095709 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=31 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:58:16.703097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:58:16.713287 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e96a4-54c2-48f7-8291-4b0307ea2c23 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:58:16.718002 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:58:16.718637 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:58:16.718914 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:58:16.719180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:58:16.721225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:58:16.731199 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:58:16.732660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:58:16.740840 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4532/9023] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:58:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:58:26.737617 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:05.038856 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:05.040311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:05.100504 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b327d-84bf-4e7f-8694-b6d23ebab381 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:05.104946 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b10fe-1a5c-49a6-9e94-15d77762ab64 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:05.105603 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b10fe-1a5c-49a6-9e94-15d77762ab64 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:05.105880 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b10fe-1a5c-49a6-9e94-15d77762ab64 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:05.106166 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b10fe-1a5c-49a6-9e94-15d77762ab64 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:05.108078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-b14b10fe-1a5c-49a6-9e94-15d77762ab64 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:05.121334 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b10fe-1a5c-49a6-9e94-15d77762ab64 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:05.122904 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b10fe-1a5c-49a6-9e94-15d77762ab64 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:05.134017 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4492/9024] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:05.324703 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:05.334646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122efc27-0c1e-4c1a-83f3-83c617f9a473 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:05.338750 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:05.339357 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:05.339635 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:05.340329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:05.342377 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:05.356536 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:05.358061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:05.369168 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4533/9025] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:15.129307 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=26 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:15.364689 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:17.905340 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87dcc0-91b8-47d6-a16d-1eba5f767b89 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:17.905924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87dcc0-91b8-47d6-a16d-1eba5f767b89 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:17.905924 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87dcc0-91b8-47d6-a16d-1eba5f767b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:17.970277 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-af87dcc0-91b8-47d6-a16d-1eba5f767b89 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:59:17.971487 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-af87dcc0-91b8-47d6-a16d-1eba5f767b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:18.048098 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4493/9026] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:59:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:59:18.106648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:18.108289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:18.202784 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c5ef9-f99a-4609-8064-45ac590dbdbb service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:18.208518 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d400b9-2101-4129-9721-32780387484e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:18.209126 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d400b9-2101-4129-9721-32780387484e service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:18.209401 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d400b9-2101-4129-9721-32780387484e service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:18.209715 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d400b9-2101-4129-9721-32780387484e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:18.218073 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d400b9-2101-4129-9721-32780387484e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:18.219653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d400b9-2101-4129-9721-32780387484e service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:18.227825 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4534/9027] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:59:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:59:28.044613 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=44 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:28.224529 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=41 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:30.698097 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87dcc0-91b8-47d6-a16d-1eba5f767b89 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:30.708172 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87dcc0-91b8-47d6-a16d-1eba5f767b89 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:30.711744 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:30.712344 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:30.712622 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:30.712891 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:30.714653 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:30.723992 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:30.725505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:30.733412 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4494/9028] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:30.872290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403e3aa-5694-4144-b7f8-3377b22a14a4 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:30.873226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403e3aa-5694-4144-b7f8-3377b22a14a4 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:30.873516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403e3aa-5694-4144-b7f8-3377b22a14a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:30.926413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-c403e3aa-5694-4144-b7f8-3377b22a14a4 None None] MFA Rules not processed for user `724ce8c58c1b473e9e3af18009532c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:59:30.927795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-c403e3aa-5694-4144-b7f8-3377b22a14a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:30.937618 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4535/9029] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 17 21:59:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:59:30.949503 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:30.959488 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45415b0b-0ed8-4162-b1df-0869fad67ea1 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:30.963249 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce9c64a0bb478485c47ebedd5a29f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:30.963973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:30.964312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:30.964640 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:30.975890 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:30.977285 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:30.988635 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4495/9030] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:31.060111 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403e3aa-5694-4144-b7f8-3377b22a14a4 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:31.075449 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403e3aa-5694-4144-b7f8-3377b22a14a4 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:31.081732 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:31.082539 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:31.082849 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:31.083127 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:31.085279 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:31.095821 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:31.097282 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:31.105387 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4536/9031] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:31.183556 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:31.194008 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85022247-a030-485b-973f-fa6f67591c09 service glance] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:31.198667 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:31.199264 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:31.199531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:31.200171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:31.202293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:31.216618 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:31.218125 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:31.229375 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4496/9032] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:31.289972 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:31.291636 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:31.323650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2db7b-7d49-4e58-96ee-3631abe0d8ef service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:31.329827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9169e11-a63e-4b8d-834a-a8c45ed12f18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:31.330438 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9169e11-a63e-4b8d-834a-a8c45ed12f18 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:31.330719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9169e11-a63e-4b8d-834a-a8c45ed12f18 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:31.330974 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9169e11-a63e-4b8d-834a-a8c45ed12f18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:31.332889 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-f9169e11-a63e-4b8d-834a-a8c45ed12f18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:31.344407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9169e11-a63e-4b8d-834a-a8c45ed12f18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:31.346024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9169e11-a63e-4b8d-834a-a8c45ed12f18 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:31.356228 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4537/9033] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:32.552360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:32.554289 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:32.568387 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae784-c1ed-4217-ae8f-6810d1df1419 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:32.573995 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:32.574967 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:32.575242 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:32.575516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:32.577278 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:32.589962 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:32.591568 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:32.602754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4497/9034] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 21:59:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:59:33.653994 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d8389-ad0d-411e-ba43-3e1be483799e None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:33.654472 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d8389-ad0d-411e-ba43-3e1be483799e None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:33.654592 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d8389-ad0d-411e-ba43-3e1be483799e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:33.675405 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-cf9d8389-ad0d-411e-ba43-3e1be483799e None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:59:33.676933 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-cf9d8389-ad0d-411e-ba43-3e1be483799e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:33.684382 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4538/9035] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 21:59:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:59:33.696594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:33.708406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7678f544-e2d4-4c6a-81b6-7ff0d2e23c53 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:33.713635 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:33.714224 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:33.714493 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:33.714757 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:33.722822 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:33.724403 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:33.732413 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4498/9036] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:59:43.669278 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=33 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:43.729335 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=30 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:46.119182 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97127ee3-4226-40eb-8791-9695482807d8 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:46.119694 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97127ee3-4226-40eb-8791-9695482807d8 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:46.119807 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97127ee3-4226-40eb-8791-9695482807d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:46.143427 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-97127ee3-4226-40eb-8791-9695482807d8 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:59:46.144646 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-97127ee3-4226-40eb-8791-9695482807d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:59:46.153501 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4539/9037] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 21:59:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:59:46.165612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:59:46.176406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5e1f5-8872-4bd1-9e52-24530df7074e service swift] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:59:46.181903 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e45185-c058-49ab-8a89-9ce67e8c5356 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:59:46.182516 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e45185-c058-49ab-8a89-9ce67e8c5356 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:59:46.182795 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e45185-c058-49ab-8a89-9ce67e8c5356 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:59:46.183064 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e45185-c058-49ab-8a89-9ce67e8c5356 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:59:46.191580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e45185-c058-49ab-8a89-9ce67e8c5356 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:59:46.193291 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e45185-c058-49ab-8a89-9ce67e8c5356 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:59:46.201966 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4499/9038] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 21:59:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:59:56.135634 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:56.198738 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:00:57.247022 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97127ee3-4226-40eb-8791-9695482807d8 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:00:57.257411 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97127ee3-4226-40eb-8791-9695482807d8 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:00:57.261835 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:00:57.262497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:00:57.262755 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:00:57.263053 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:00:57.265143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:00:57.275632 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:00:57.277080 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:00:57.289634 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4540/9039] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:00:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:01:02.067454 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4265-d4e1-4621-9967-5ed87f1b85f1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:01:02.068003 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4265-d4e1-4621-9967-5ed87f1b85f1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:01:02.068137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a4265-d4e1-4621-9967-5ed87f1b85f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:01:02.088938 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-959a4265-d4e1-4621-9967-5ed87f1b85f1 None None] MFA Rules not processed for user `3d3bf2dacad34c0aa465968f1c273cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:01:02.090239 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-959a4265-d4e1-4621-9967-5ed87f1b85f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:01:02.097919 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4500/9040] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:01:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:01:03.946797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:01:03.956366 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4364783-c973-425c-9aac-21f23c6892d4 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:01:03.960712 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:01:03.961310 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:01:03.961615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:01:03.961882 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:01:03.963817 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:01:03.973532 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:01:03.974990 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:01:03.982795 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4541/9041] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:01:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:01:12.084098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=3 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:01:13.979520 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=12 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:01:15.465952 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a4265-d4e1-4621-9967-5ed87f1b85f1 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:01:15.489157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a4265-d4e1-4621-9967-5ed87f1b85f1 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:01:15.494878 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:01:15.495485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:01:15.495760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:01:15.496227 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:01:15.498048 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:01:15.510979 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:01:15.512612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:01:15.523754 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4501/9042] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:01:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:01:20.227456 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:01:20.240413 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc0cd-7243-4b8d-882d-e3cf61c21c92 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:01:20.245609 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:01:20.246219 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:01:20.246492 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:01:20.246746 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:01:20.248931 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:01:20.261333 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:01:20.263186 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:01:20.274455 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4542/9043] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:01:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:01:23.120641 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:01:23.132864 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff1841-644e-4733-ad31-0c9d7e6bf218 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:01:23.139306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:01:23.139905 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:01:23.140175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:01:23.140440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:01:23.142531 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:01:23.154612 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:01:23.156209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:01:23.166387 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4502/9044] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:01:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:01:30.270326 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:01:33.162424 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=15 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:03:22.292269 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:03:22.312683 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73d901-b7d0-4e1c-8dc1-3bf89c4f7930 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:03:22.316495 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:03:22.317100 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:03:22.317378 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:03:22.317721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:03:22.325721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:03:22.327144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:03:22.336544 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4543/9045] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:03:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:03:25.139415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:03:25.163611 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b1278-a298-46ea-be4e-77f5d5d06500 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:03:25.169760 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:03:25.170367 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:03:25.170646 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:03:25.170915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:03:25.179337 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:03:25.180934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:03:25.190425 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4503/9046] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:03:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:03:32.332716 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=10 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:03:35.186406 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=10 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:17.414897 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:17.438088 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f148aad-a7fa-4709-9975-86f699695a7d service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:17.444353 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:17.444951 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:17.445225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:17.445886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:17.455446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:17.457029 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:17.468180 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4544/9047] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:17.616478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:17.626543 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd89e7-be1f-44c9-84f3-2d5712fd7a5b service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:17.630553 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:17.631233 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:17.631505 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:17.631773 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:17.643205 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:17.644996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:17.657432 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4504/9048] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:18.855030 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:18.865486 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ec0b6-1a91-482d-8cf3-0f85d296cfc4 service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:18.869917 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:18.870511 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:18.870782 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:18.871051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:18.881660 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:18.883061 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:18.894712 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4545/9049] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:21.435312 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:21.444036 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fff29-642c-470e-a140-43ab47718126 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:21.447996 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:21.448588 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:21.448863 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:21.449133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:21.522972 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:21.524438 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:21.543515 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4505/9050] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:24.807017 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:24.829924 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd6b27-acb7-4c85-a784-208ae873a846 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:24.835314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:24.835934 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:24.836213 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:24.836483 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:24.838447 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:24.851243 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:24.852954 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:24.864188 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4546/9051] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:27.768204 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:27.781283 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2cd09-61e1-4b86-bcf7-220021d351b6 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:27.786416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:27.787035 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:27.787311 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:27.787580 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:27.789376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:27.803656 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:27.805345 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:27.816102 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4506/9052] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 22:05:27.842742 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:27.855059 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8753-cefb-4979-9402-802988422ca4 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:27.861935 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:27.862709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:27.863082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:27.863873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] PATH_INFO: `/v3/users/3d3bf2dacad34c0aa465968f1c273cf9` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:27.864819 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3bf2dacad34c0aa465968f1c273cf9)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:27.867389 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:27.917137 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3bf2dacad34c0aa465968f1c273cf9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:27.917943 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3bf2dacad34c0aa465968f1c273cf9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:27.926995 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3bf2dacad34c0aa465968f1c273cf9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:27.927631 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3bf2dacad34c0aa465968f1c273cf9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:27.940051 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3bf2dacad34c0aa465968f1c273cf9'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:27.952681 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4547/9053] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 22:05:27 2022] DELETE /identity/v3/users/3d3bf2dacad34c0aa465968f1c273cf9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 22:05:28.127414 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:28.138634 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f407c8-47bb-451f-bf04-688761d14f75 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:28.204054 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:28.204695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:28.204974 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:28.205270 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] PATH_INFO: `/v3/projects/015ec23d11f64d56844d28795ccba61a` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:28.206478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=015ec23d11f64d56844d28795ccba61a)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:28.208398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:28.256668 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015ec23d11f64d56844d28795ccba61a', 'request_id': 'req-ad348f12-0fb9-44cb-a507-d626a60ec9fa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.259167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:28.266456 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015ec23d11f64d56844d28795ccba61a', 'request_id': 'req-ad348f12-0fb9-44cb-a507-d626a60ec9fa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.273286 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015ec23d11f64d56844d28795ccba61a', 'request_id': 'req-ad348f12-0fb9-44cb-a507-d626a60ec9fa'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.298365 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] The token cache is being invalidate because project 015ec23d11f64d56844d28795ccba61a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 22:05:28.298654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.299468 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.301015 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4507/9054] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 22:05:28 2022] DELETE /identity/v3/projects/015ec23d11f64d56844d28795ccba61a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:05:28.305340 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:28.307024 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:28.321667 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a33c4e-34cc-4aa4-85a5-a3d86b2ffb14 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:28.382659 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:28.383446 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:28.383754 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:28.384031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] PATH_INFO: `/v3/users/92eaa391d0484fd8853f341baae9702f` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:28.385298 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=92eaa391d0484fd8853f341baae9702f)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:28.388808 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:28.420234 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:28.441091 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92eaa391d0484fd8853f341baae9702f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.441796 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92eaa391d0484fd8853f341baae9702f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.450329 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92eaa391d0484fd8853f341baae9702f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.451038 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92eaa391d0484fd8853f341baae9702f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.463159 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-9baf7edc-952a-4a75-8eb3-ba0a89c1b75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92eaa391d0484fd8853f341baae9702f'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.476634 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4548/9055] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 22:05:28 2022] DELETE /identity/v3/users/92eaa391d0484fd8853f341baae9702f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:05:28.658564 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:28.660096 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:28.676765 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad348f12-0fb9-44cb-a507-d626a60ec9fa admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:28.739742 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:28.740376 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:28.740654 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:28.740930 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] PATH_INFO: `/v3/projects/f99ff4f6290b45f882b66d1f7407010e` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:28.742075 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99ff4f6290b45f882b66d1f7407010e)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:28.743998 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:28.801592 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99ff4f6290b45f882b66d1f7407010e', 'request_id': 'req-96046f79-ff59-4229-afa5-d5977ac7b7ee'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.809293 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99ff4f6290b45f882b66d1f7407010e', 'request_id': 'req-96046f79-ff59-4229-afa5-d5977ac7b7ee'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.814936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99ff4f6290b45f882b66d1f7407010e', 'request_id': 'req-96046f79-ff59-4229-afa5-d5977ac7b7ee'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.826221 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] The token cache is being invalidate because project f99ff4f6290b45f882b66d1f7407010e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 22:05:28.826506 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.827128 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-96046f79-ff59-4229-afa5-d5977ac7b7ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:05:28.828639 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4508/9056] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 22:05:28 2022] DELETE /identity/v3/projects/f99ff4f6290b45f882b66d1f7407010e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:05:28.834152 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fda81-9e28-47c8-8534-0b2495b65723 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:28.834417 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fda81-9e28-47c8-8534-0b2495b65723 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:28.834692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fda81-9e28-47c8-8534-0b2495b65723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:28.885912 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-7e6fda81-9e28-47c8-8534-0b2495b65723 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:05:28.887136 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7e6fda81-9e28-47c8-8534-0b2495b65723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:28.947982 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4549/9057] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:05:28.957194 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a040cf2-23a0-4180-a755-079106396b20 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:28.957513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a040cf2-23a0-4180-a755-079106396b20 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:28.957785 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a040cf2-23a0-4180-a755-079106396b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:28.978297 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8a040cf2-23a0-4180-a755-079106396b20 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:05:28.979497 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8a040cf2-23a0-4180-a755-079106396b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:28.988040 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4509/9058] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 22:05:28.992570 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fda81-9e28-47c8-8534-0b2495b65723 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:29.002886 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fda81-9e28-47c8-8534-0b2495b65723 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:29.008777 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:29.009393 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.009699 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.009964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.010625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 22:05:29.010891 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:29.012497 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:29.024727 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4550/9059] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 17 22:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 22:05:29.029745 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a040cf2-23a0-4180-a755-079106396b20 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:29.042045 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a040cf2-23a0-4180-a755-079106396b20 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:29.049171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:29.049973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.050258 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.050533 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.051209 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:29.053156 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:29.100084 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4510/9060] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 22:05:29 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 22:05:29.101233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:29.111642 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7e9e8-cb18-4976-abdb-3c32baeb4dc7 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:29.172921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:29.173576 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.173852 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.174114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.174744 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:29.176721 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:29.195953 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:29.206179 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4551/9061] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 22:05:29 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 22:05:29.207218 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:29.218130 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9147d8-7ceb-4075-9e8c-8deded9c9dd0 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:29.222823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:29.223436 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.223712 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.223984 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.224629 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:29.226411 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:29.239867 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4511/9062] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 22:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 22:05:29.242225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:29.252885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5c462-b418-4799-a441-c9f60b36921c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:29.257735 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd832d4-ce3d-4a4e-9c28-983bac3ccaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:29.258343 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd832d4-ce3d-4a4e-9c28-983bac3ccaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.258620 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd832d4-ce3d-4a4e-9c28-983bac3ccaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.258921 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd832d4-ce3d-4a4e-9c28-983bac3ccaf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.259597 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd832d4-ce3d-4a4e-9c28-983bac3ccaf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:29.261212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd832d4-ce3d-4a4e-9c28-983bac3ccaf4 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:29.272023 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4552/9063] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 22:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 22:05:29.276229 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:29.286353 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8183e-9c04-4da6-8027-64d65d638538 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:29.291382 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:29.292025 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] REQUEST_METHOD: `PUT` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.292300 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.292567 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] PATH_INFO: `/v3/projects/3881a625eb9c4ad9bc95e1a8033a5b3d/users/d46ef9b682704558999055a3b8f71f25/roles/78bf7fb4f8e947ba99f38ca1691e6393` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.301796 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:29.309078 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3881a625eb9c4ad9bc95e1a8033a5b3d, user_id=d46ef9b682704558999055a3b8f71f25, role_id=78bf7fb4f8e947ba99f38ca1691e6393)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:29.312342 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:29.326802 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4512/9064] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 22:05:29 2022] PUT /identity/v3/projects/3881a625eb9c4ad9bc95e1a8033a5b3d/users/d46ef9b682704558999055a3b8f71f25/roles/78bf7fb4f8e947ba99f38ca1691e6393 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:05:29.332086 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0723ace-bf3b-42f3-a497-2095279ccd08 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.332373 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0723ace-bf3b-42f3-a497-2095279ccd08 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.332653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0723ace-bf3b-42f3-a497-2095279ccd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.352407 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-e0723ace-bf3b-42f3-a497-2095279ccd08 None None] MFA Rules not processed for user `fef6fab9131d4c1e89e0107005eb3caa`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:05:29.353596 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-e0723ace-bf3b-42f3-a497-2095279ccd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:29.419249 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4553/9065] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:05:29.429718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:29.431476 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:29.497630 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d5fce-3c72-41a5-9de3-65ec1ffb628b admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:29.502398 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:29.503044 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:29.503323 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:29.503594 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:29.513137 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:29.514639 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:29.524390 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4513/9066] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:33.743965 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc501b06-c896-4a2d-b3a0-0efc2161e955 None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:33.744516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc501b06-c896-4a2d-b3a0-0efc2161e955 None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:33.744516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc501b06-c896-4a2d-b3a0-0efc2161e955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:33.774084 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-dc501b06-c896-4a2d-b3a0-0efc2161e955 None None] MFA Rules not processed for user `d46ef9b682704558999055a3b8f71f25`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:05:33.775455 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-dc501b06-c896-4a2d-b3a0-0efc2161e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:33.846015 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4554/9067] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:05:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:05:33.853530 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:33.860991 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def2c31-ced9-4178-8557-ed62b1830e44 service neutron] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:33.865893 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:33.866597 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:33.866936 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:33.867408 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] PATH_INFO: `/v3/projects` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:33.868135 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:33.870203 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:33.899981 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4514/9068] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 17 22:05:33 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 22:05:33.904272 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc501b06-c896-4a2d-b3a0-0efc2161e955 None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:33.916178 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc501b06-c896-4a2d-b3a0-0efc2161e955 None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:33.977031 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:33.977442 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:33.977764 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:33.978032 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] PATH_INFO: `/v3/users` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:33.978692 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:33.980801 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:34.006870 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4555/9069] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 17 22:05:33 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 22:05:34.011121 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:34.022513 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613d229-c4d9-4768-a597-73926465db01 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:34.027171 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:34.027778 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:34.028052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:34.028428 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:34.029196 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:34.030848 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:34.042251 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4515/9070] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 22:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 22:05:34.046626 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:34.057674 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606b924-23a4-49bf-868e-404f9b239d81 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:34.063071 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:34.063838 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:34.064180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:34.064519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] PATH_INFO: `/v3/projects/94b9d62918c542ecb2e2734d93455984/users/6ff7f4604f3642a0a1e91009c668ca41/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:34.067475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:34.142606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b9d62918c542ecb2e2734d93455984, user_id=6ff7f4604f3642a0a1e91009c668ca41, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:34.142606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:34.142606 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4556/9071] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 22:05:34 2022] PUT /identity/v3/projects/94b9d62918c542ecb2e2734d93455984/users/6ff7f4604f3642a0a1e91009c668ca41/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:05:34.142819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:34.142819 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15267d-d708-452b-8869-400f15626c0d admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:34.203954 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb91db-ee63-49a2-a990-3cef62bd4ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:34.204582 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb91db-ee63-49a2-a990-3cef62bd4ebe admin admin] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:34.204862 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb91db-ee63-49a2-a990-3cef62bd4ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:34.205133 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb91db-ee63-49a2-a990-3cef62bd4ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:34.205851 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb91db-ee63-49a2-a990-3cef62bd4ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:34.207484 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb91db-ee63-49a2-a990-3cef62bd4ebe admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:34.218992 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4516/9072] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 17 22:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 22:05:34.223697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:34.235625 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da646c8-739d-49c1-b4de-bbb402278e9a admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:34.240798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:34.241709 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] REQUEST_METHOD: `PUT` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:34.243028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:34.243028 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] PATH_INFO: `/v3/projects/94b9d62918c542ecb2e2734d93455984/users/6ff7f4604f3642a0a1e91009c668ca41/roles/6583e7a6ce674f6da4f19e31fe025509` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:34.244423 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b9d62918c542ecb2e2734d93455984, user_id=6ff7f4604f3642a0a1e91009c668ca41, role_id=6583e7a6ce674f6da4f19e31fe025509)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:34.247384 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:34.259073 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4557/9073] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 17 22:05:34 2022] PUT /identity/v3/projects/94b9d62918c542ecb2e2734d93455984/users/6ff7f4604f3642a0a1e91009c668ca41/roles/6583e7a6ce674f6da4f19e31fe025509 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:05:38.674256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a947b57-1bfc-4856-9699-72512aee24a2 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:38.674769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a947b57-1bfc-4856-9699-72512aee24a2 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:38.674837 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a947b57-1bfc-4856-9699-72512aee24a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:38.706026 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-1a947b57-1bfc-4856-9699-72512aee24a2 None None] MFA Rules not processed for user `6ff7f4604f3642a0a1e91009c668ca41`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:05:38.707252 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-1a947b57-1bfc-4856-9699-72512aee24a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:38.757179 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:38.786271 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4517/9074] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:05:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:05:38.802702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:38.804737 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:38.896440 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59370-633c-4ee1-af2c-29a2360aa61c admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:38.921582 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c08cc-a388-4090-8285-24959178ecef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:38.922237 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c08cc-a388-4090-8285-24959178ecef service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:38.922519 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c08cc-a388-4090-8285-24959178ecef service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:38.922791 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c08cc-a388-4090-8285-24959178ecef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:38.985339 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c08cc-a388-4090-8285-24959178ecef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:38.987233 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c08cc-a388-4090-8285-24959178ecef service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:38.999436 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4558/9075] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 22:05:40.101031 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:40.101662 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:40.101768 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:40.125373 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] MFA Rules not processed for user `d46ef9b682704558999055a3b8f71f25`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:05:40.126732 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:40.134826 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4518/9076] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:05:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 22:05:40.462986 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c08cc-a388-4090-8285-24959178ecef service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:40.464780 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-636c08cc-a388-4090-8285-24959178ecef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:40.527772 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c08cc-a388-4090-8285-24959178ecef service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:40.531719 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:40.532380 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:40.532656 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:40.532927 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:40.535010 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:40.546374 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:40.547839 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:40.551180 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:40.560986 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4559/9077] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:40.916294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:40.917925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:40.997157 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b326-aaa8-401a-b29f-e95de49da10f None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:41.002134 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:41.002807 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:41.003087 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:41.003360 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:41.005201 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:41.017151 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:41.018888 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:41.029158 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4519/9078] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:50.561718 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:50.647301 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:50.659027 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d163-68ff-4796-9d2a-f4987fa713f8 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:50.663202 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:50.663812 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:50.664093 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:50.664365 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:50.674648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:50.676117 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:50.686478 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4560/9079] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:05:51.024925 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=45 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:52.665235 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:05:52.682969 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92337f2-8a9c-4e7f-8578-840c42ad7746 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:05:52.688728 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:05:52.689356 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:05:52.689666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:05:52.689944 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:05:52.692420 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:05:52.708554 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:05:52.711150 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:05:52.724012 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4520/9080] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:05:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:06:00.681795 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:06:02.719030 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=6 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:06:25.001025 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:06:25.043896 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68015c06-808b-428f-8e12-1cba05fdbe75 service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:06:25.049697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90b82f-bdbe-461e-8fbc-2ed626e7cb24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:06:25.050304 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90b82f-bdbe-461e-8fbc-2ed626e7cb24 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:06:25.050580 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90b82f-bdbe-461e-8fbc-2ed626e7cb24 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:06:25.050858 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90b82f-bdbe-461e-8fbc-2ed626e7cb24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:06:25.052785 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-9e90b82f-bdbe-461e-8fbc-2ed626e7cb24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:06:25.065057 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90b82f-bdbe-461e-8fbc-2ed626e7cb24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:06:25.066671 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90b82f-bdbe-461e-8fbc-2ed626e7cb24 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:06:25.077142 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4561/9081] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:06:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:06:30.168361 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:06:30.170083 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:06:30.221477 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b2c89-7ca1-45b3-bd73-78da67805797 service nova] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:06:30.227857 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:06:30.228440 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:06:30.228695 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:06:30.228987 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:06:30.230956 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:06:30.242855 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:06:30.244447 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:06:30.254186 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4521/9082] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:06:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:06:35.073121 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=20 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:06:40.250318 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=20 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:07:54.282964 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b09ce-1097-405e-b68a-f3d4313a755f None None] REQUEST_METHOD: `POST` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:07:54.283475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b09ce-1097-405e-b68a-f3d4313a755f None None] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:07:54.283475 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b09ce-1097-405e-b68a-f3d4313a755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:07:54.314225 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.auth.core [None req-179b09ce-1097-405e-b68a-f3d4313a755f None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:07:54.315464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-179b09ce-1097-405e-b68a-f3d4313a755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:07:54.382670 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4562/9083] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 22:07:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:07:54.434435 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:07:54.435883 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:07:54.526847 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f372d7-0167-4510-84cb-1c6020cf50b2 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:07:54.532306 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864593-6339-41b2-a122-4ccdd62afeb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:07:54.532896 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864593-6339-41b2-a122-4ccdd62afeb2 service swift] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:07:54.533169 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864593-6339-41b2-a122-4ccdd62afeb2 service swift] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:07:54.533434 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864593-6339-41b2-a122-4ccdd62afeb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:07:54.541678 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864593-6339-41b2-a122-4ccdd62afeb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:07:54.543279 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864593-6339-41b2-a122-4ccdd62afeb2 service swift] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:07:54.551640 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4522/9084] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 22:07:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 22:07:55.502089 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b09ce-1097-405e-b68a-f3d4313a755f None None] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:07:55.524144 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b09ce-1097-405e-b68a-f3d4313a755f None None] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:07:55.530146 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '724ce8c58c1b473e9e3af18009532c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:07:55.530901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:07:55.531290 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:07:55.531648 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:07:55.534105 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:07:55.547736 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:07:55.550090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:07:55.562101 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4563/9085] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:07:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:08:03.856427 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d816d2-0586-4790-96a4-1fbee1719cc1 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:08:03.856913 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d816d2-0586-4790-96a4-1fbee1719cc1 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:08:03.857086 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d816d2-0586-4790-96a4-1fbee1719cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:08:03.879099 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-50d816d2-0586-4790-96a4-1fbee1719cc1 None None] MFA Rules not processed for user `d46ef9b682704558999055a3b8f71f25`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:08:03.880336 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-50d816d2-0586-4790-96a4-1fbee1719cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:08:03.887979 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4523/9086] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 17 22:08:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:08:04.576226 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:08:04.590468 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b309cf-98e5-40fb-b45a-7510715a588e service nova] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:08:04.597653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:08:04.598289 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:08:04.598566 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:08:04.598888 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:08:04.600606 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:08:04.613916 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:08:04.615633 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:08:04.629000 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4564/9087] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:08:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:08:05.862478 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7b47-2a45-441e-a940-f12778c1b4ae None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:08:05.862932 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7b47-2a45-441e-a940-f12778c1b4ae None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:08:05.863029 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7b47-2a45-441e-a940-f12778c1b4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:08:05.883843 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-8ebb7b47-2a45-441e-a940-f12778c1b4ae None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:08:05.885123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-8ebb7b47-2a45-441e-a940-f12778c1b4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:08:05.893200 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4524/9088] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 22:08:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:08:05.905386 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:08:05.918130 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54e6e2-2d7a-4e61-a03a-182ee7276082 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:08:05.924608 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:08:05.925309 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:08:05.925650 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:08:05.925928 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:08:05.935239 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:08:05.936872 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:08:05.947127 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4565/9089] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 22:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 22:08:14.141966 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7b47-2a45-441e-a940-f12778c1b4ae None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:08:14.153256 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7b47-2a45-441e-a940-f12778c1b4ae None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:08:14.158294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34d730-7efa-4f64-8816-bae11479f596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:08:14.158990 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34d730-7efa-4f64-8816-bae11479f596 service neutron] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:08:14.159314 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34d730-7efa-4f64-8816-bae11479f596 service neutron] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:08:14.159666 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34d730-7efa-4f64-8816-bae11479f596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:08:14.161602 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3c34d730-7efa-4f64-8816-bae11479f596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:08:14.184574 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34d730-7efa-4f64-8816-bae11479f596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:08:14.186098 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34d730-7efa-4f64-8816-bae11479f596 service neutron] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:08:14.194589 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4525/9090] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:08:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:08:15.942901 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=51 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:08:24.190942 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=57 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:08:28.302831 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:08:28.320629 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd754fe-91b0-480f-a5f9-c2ce4791dc00 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:08:28.326145 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a824d5df8638409bbaf4d6e8d22d63ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:08:28.326883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:08:28.327174 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:08:28.327453 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:08:28.329841 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:08:28.341308 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:08:28.342876 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:08:28.354398 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4566/9091] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:08:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:08:38.350672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:09:56.906937 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829e186-6a58-42b5-8a72-c6b47cd09eea None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:09:56.907586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829e186-6a58-42b5-8a72-c6b47cd09eea None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:09:56.907586 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829e186-6a58-42b5-8a72-c6b47cd09eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:09:56.969870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-c829e186-6a58-42b5-8a72-c6b47cd09eea None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:09:56.971167 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-c829e186-6a58-42b5-8a72-c6b47cd09eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:09:56.989679 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4526/9092] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 17 22:09:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:09:57.029798 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:09:57.051787 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f26ea-ce34-45b0-b62a-6fc88d1fd3bc service neutron] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:09:57.056697 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:09:57.057293 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:09:57.057610 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:09:57.057883 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:09:57.065904 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:09:57.067842 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:09:57.076149 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4567/9093] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 22:09:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 22:10:06.310730 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85255bd5-e491-4a4a-83ed-ef2a4b16ef91 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:06.311250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85255bd5-e491-4a4a-83ed-ef2a4b16ef91 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:06.311250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85255bd5-e491-4a4a-83ed-ef2a4b16ef91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:06.330934 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-85255bd5-e491-4a4a-83ed-ef2a4b16ef91 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:10:06.332148 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-85255bd5-e491-4a4a-83ed-ef2a4b16ef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:10:06.339707 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4527/9094] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 22:10:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:10:06.351939 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:10:06.362717 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9c076-ddc3-4b3d-8a8a-570c07798aa1 service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:10:06.367977 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:10:06.368578 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:06.368857 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:06.369132 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:06.377242 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:10:06.378873 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:10:06.387173 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4568/9095] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 22:10:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 22:10:11.321958 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa608a9d-3b6f-46f5-90ac-a45c4112af06 None None] REQUEST_METHOD: `POST` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:11.322430 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa608a9d-3b6f-46f5-90ac-a45c4112af06 None None] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:11.322540 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa608a9d-3b6f-46f5-90ac-a45c4112af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:11.345392 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.auth.core [None req-aa608a9d-3b6f-46f5-90ac-a45c4112af06 None None] MFA Rules not processed for user `a090c2f1086243a19fd4f5ade6d7560b`. Rule list: `[]` (Enabled: `True`). {{(pid=99166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 22:10:11.347082 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-aa608a9d-3b6f-46f5-90ac-a45c4112af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:10:11.358288 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4528/9096] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 17 22:10:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 22:10:11.370971 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:10:11.382241 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396bbb1-1040-4e3f-9f30-d8d734b9d11a service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:10:11.387829 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8eea4641bfd940fe8a94ec79484ef4d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:10:11.388428 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:11.388700 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:11.388969 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:11.397871 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:10:11.399516 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:10:11.408049 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4569/9097] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 17 22:10:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 22:10:21.337787 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=28 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:21.404403 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=22 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:23.182615 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa608a9d-3b6f-46f5-90ac-a45c4112af06 None None] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:10:23.248062 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa608a9d-3b6f-46f5-90ac-a45c4112af06 None None] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:10:23.252696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:10:23.253331 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:23.253642 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:23.253915 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] PATH_INFO: `/v3/users/d46ef9b682704558999055a3b8f71f25` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:23.255119 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46ef9b682704558999055a3b8f71f25)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:10:23.257058 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:10:23.307294 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46ef9b682704558999055a3b8f71f25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.319718 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46ef9b682704558999055a3b8f71f25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.327933 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46ef9b682704558999055a3b8f71f25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.328677 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46ef9b682704558999055a3b8f71f25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.329402 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46ef9b682704558999055a3b8f71f25'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.342164 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4529/9098] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 22:10:23 2022] DELETE /identity/v3/users/d46ef9b682704558999055a3b8f71f25 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:10:23.530090 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:10:23.545138 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ab0e7-8984-4c94-a177-6da477f924eb service swift] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:10:23.606506 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:10:23.607143 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:23.607419 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:23.607690 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] PATH_INFO: `/v3/projects/3881a625eb9c4ad9bc95e1a8033a5b3d` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:23.610381 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3881a625eb9c4ad9bc95e1a8033a5b3d)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:10:23.612332 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:10:23.660207 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3881a625eb9c4ad9bc95e1a8033a5b3d', 'request_id': 'req-7f126418-bdcf-4305-96b1-4af348dd60fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.666630 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3881a625eb9c4ad9bc95e1a8033a5b3d', 'request_id': 'req-7f126418-bdcf-4305-96b1-4af348dd60fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.675216 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3881a625eb9c4ad9bc95e1a8033a5b3d', 'request_id': 'req-7f126418-bdcf-4305-96b1-4af348dd60fe'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.686388 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] The token cache is being invalidate because project 3881a625eb9c4ad9bc95e1a8033a5b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 22:10:23.686672 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.687331 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.688859 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4570/9099] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 22:10:23 2022] DELETE /identity/v3/projects/3881a625eb9c4ad9bc95e1a8033a5b3d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:10:23.693762 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:10:23.695016 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:10:23.708395 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b535b-e82e-4f61-8da3-5742a3509e26 admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:10:23.767485 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:10:23.768123 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:23.768415 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:23.768689 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] PATH_INFO: `/v3/users/6ff7f4604f3642a0a1e91009c668ca41` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:23.769973 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff7f4604f3642a0a1e91009c668ca41)` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:10:23.771870 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:10:23.793823 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:23.823215 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff7f4604f3642a0a1e91009c668ca41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.834859 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff7f4604f3642a0a1e91009c668ca41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.843416 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff7f4604f3642a0a1e91009c668ca41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.844052 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff7f4604f3642a0a1e91009c668ca41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.844769 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.notifications [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff7f4604f3642a0a1e91009c668ca41'} {{(pid=99166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:23.860521 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4530/9100] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 17 22:10:23 2022] DELETE /identity/v3/users/6ff7f4604f3642a0a1e91009c668ca41 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 22:10:24.035463 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:10:24.037513 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:10:24.051153 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f126418-bdcf-4305-96b1-4af348dd60fe admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:10:24.119302 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef6fab9131d4c1e89e0107005eb3caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8633d9055db49bfa42074c66a0581c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:10:24.119938 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:10:24.120212 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:10:24.120480 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] PATH_INFO: `/v3/projects/94b9d62918c542ecb2e2734d93455984` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:10:24.121653 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b9d62918c542ecb2e2734d93455984)` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:10:24.123585 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:10:24.161082 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:24.177052 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b9d62918c542ecb2e2734d93455984', 'request_id': 'req-107e0734-de79-48f8-89d8-6d92eac0f109'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:24.183825 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b9d62918c542ecb2e2734d93455984', 'request_id': 'req-107e0734-de79-48f8-89d8-6d92eac0f109'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:24.193702 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b9d62918c542ecb2e2734d93455984', 'request_id': 'req-107e0734-de79-48f8-89d8-6d92eac0f109'} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:24.204923 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] The token cache is being invalidate because project 94b9d62918c542ecb2e2734d93455984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 22:10:24.205210 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:24.205856 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.notifications [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 22:10:24.207444 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4571/9101] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 17 22:10:24 2022] DELETE /identity/v3/projects/94b9d62918c542ecb2e2734d93455984 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 22:10:33.857175 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:34.201926 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:11:28.983562 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:11:28.985917 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:11:29.076067 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792fec2-d891-466b-96c5-da62d319159c admin admin] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:11:29.143519 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:11:29.144144 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:11:29.144426 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:11:29.144696 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:11:29.146601 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.fernet_utils [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:11:29.159163 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:11:29.160780 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:11:29.171073 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4531/9102] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:11:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:11:34.138885 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:11:34.140797 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:11:34.157097 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e0734-de79-48f8-89d8-6d92eac0f109 admin admin] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:11:34.163530 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:11:34.164140 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:11:34.164395 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:11:34.164661 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:11:34.166523 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.fernet_utils [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 22:11:34.180464 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:11:34.182114 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:11:34.192694 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4572/9103] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:11:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:11:39.166683 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=63 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:11:44.188047 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=6 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:16:33.954232 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] Authenticating user token {{(pid=99166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:16:34.007220 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf9a08-cbcf-4c98-a600-7a607c0987b6 service placement] Validating token access rules against request {{(pid=99166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:16:34.013197 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3f0d9-1e83-4421-86c1-30d68794bddc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:16:34.013964 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3f0d9-1e83-4421-86c1-30d68794bddc service placement] REQUEST_METHOD: `GET` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:16:34.014250 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3f0d9-1e83-4421-86c1-30d68794bddc service placement] SCRIPT_NAME: `/identity` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:16:34.014508 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3f0d9-1e83-4421-86c1-30d68794bddc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:16:34.023617 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3f0d9-1e83-4421-86c1-30d68794bddc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:16:34.025225 ncdv-master-1 devstack@keystone.service[99166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3f0d9-1e83-4421-86c1-30d68794bddc service placement] RBAC: Authorization granted {{(pid=99166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:16:34.038254 ncdv-master-1 devstack@keystone.service[99166]: [pid: 99166|app: 0|req: 4532/9104] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:16:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:16:40.141250 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] Authenticating user token {{(pid=99167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 22:16:40.194663 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bea26-f34c-4fa2-97b5-0f1cc1918b41 service placement] Validating token access rules against request {{(pid=99167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 22:16:40.200441 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57141003-517e-4dec-8e0e-41e46590b015 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '064befaa1bd34b89bafad00c81ede856', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 22:16:40.201040 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141003-517e-4dec-8e0e-41e46590b015 service placement] REQUEST_METHOD: `GET` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 22:16:40.201314 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141003-517e-4dec-8e0e-41e46590b015 service placement] SCRIPT_NAME: `/identity` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 22:16:40.201615 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141003-517e-4dec-8e0e-41e46590b015 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 22:16:40.210263 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57141003-517e-4dec-8e0e-41e46590b015 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 22:16:40.211827 ncdv-master-1 devstack@keystone.service[99167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57141003-517e-4dec-8e0e-41e46590b015 service placement] RBAC: Authorization granted {{(pid=99167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 22:16:40.221612 ncdv-master-1 devstack@keystone.service[99167]: [pid: 99167|app: 0|req: 4573/9105] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 17 22:16:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 22:16:44.033805 ncdv-master-1 devstack@keystone.service[99166]: DEBUG dbcounter [-] [99166] Writing DB stats keystone:SELECT=24 {{(pid=99166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:16:50.217745 ncdv-master-1 devstack@keystone.service[99167]: DEBUG dbcounter [-] [99167] Writing DB stats keystone:SELECT=24 {{(pid=99167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}